Best and most complete possible identifying information on every line (first name + last name or full name, address, city, state, postcode, email, phone)
Relevant dates in ISO 8601 YYYY-MM-DD
Statuses, product names, etc. (in plain text not numeric codes)
One CSV per compressed file
Optional but recommended
Date in filename ISO 8601 YYYY-MM-DD
Compressed with zip, gz, bz2
Decide what you're going to upload
For example, every day, you upload 2 files:
...which contains a single CSV file, orders.csv
...every line of which has name, address
...which contains a single CSV file, customers.csv
...every line of which has name, address, email
Extra fee applies
Multiple CSVs per file
CSVs without headers
Tab separated or fixed width
Joining in identifying information from other files
Putting 2 CSVs in a single ZIP file
...instead, upload 2 ZIP files with 1 CSV each
Uploading an orders file with no identifying information, only an order ID
...instead, put name, address, etc. on every order line before you upload!
Encrypting before zipping
...instead, zip before encrypt
1. Send via Amazon S3 (our bucket)
First, we will create a bucket on our end.
To provision access to a bucket, Faraday will need an AWS username like:
Note: when you upload, you will need to set your ACL to:
We'll also need for you to put the following in your IAM user's
inline policy (example showing if the bucket name was "acme_to_faraday"):
2a. Send via Amazon S3 (your bucket)
Upload your files to a bucket you control.
Then, provision us access to a folder by defining a bucket policy:
(in the example below, it is for a folder
to_faraday/ in the bucket
Then, we have to add the bucket name to the IAM user here https://console.aws.amazon.com/iam/home?region=us-east-1#/users/stagecraft-download_s3
2b. S3 upload with SFTP user
We will require your SSH public key. They look like this:
Needless to say, they are NOT secret - that's why they're called "public" keys.
(Do not EVER send anybody, including Faraday, your SSH private key.)
If you don't have an SSH keypair, you can generate one like this https://www.digitalocean.com/docs/droplets/how-to/add-ssh-keys/create-with-openssh/.
Inputting details in our app:
You can navigate to: https://app.faraday.io/sources and hit "Create source"
Fill out the fields here:
File glob: to_faraday/$file_name_YYYYMMDD.csv
AWS bucket: acme-faraday-interchange
AWS region: us-east-1
Connecting and sending data:
Using your favorite SFTP client:
Username: varies by client (for example,
Password: not enabled, you must utilize a SSH public key as explained above
Let's pretend you work for Acme, Inc. Your SSH private key file is
/path/to/acme_rsa (remember, this is a secret, don't sent it to us) and the username given to you by Faraday is
You will upload files like this:
$ sftp -i /path/to/acme_rsa firstname.lastname@example.org
Connected to email@example.com
$ sftp> put helloworld.csv.gz to_faraday/helloworld-2019-01-15.csv.gz
Uploading helloworld.csv.gz to /faraday-acme/to_faraday/helloworld-2019-01-15.csv.gz
You should be set!
A few notes:
You won't be able to download or even see files once you send them to us - that's a security measure called a "blind dropbox."
You need to upload it to the
to_faraday/folder as shown above.
See above for requirements on file format, etc.
The most secure option is to encrypt your file with GPG (aka PGP) before you send it to us. See https://knowledgebase.faraday.io/help-and-support/optimize/how-to-encrypt-files-with-gpg.