- Best and most complete possible identifying information on every line (first name + last name or full name, address, city, state, postcode, email, phone)
- Relevant dates in ISO 8601 YYYY-MM-DD
- Statuses, product names, etc. (in plain text not numeric codes)
- CSV format
- Comma delimited
- String quoted
- Headers included
- UTF-8 encoded
- One CSV per compressed file
Optional but recommended
- Date in filename ISO 8601 YYYY-MM-DD
- Compressed with zip, gz, bz2
- Encrypted with gpg (https://knowledgebase.faraday.io/help-and-support/optimize/how-to-encrypt-files-with-gpg)
Decide what you're going to upload
For example, every day, you upload 2 files:
...which contains a single CSV file, orders.csv
...every line of which has name, address
...which contains a single CSV file, customers.csv
...every line of which has name, address, email
Extra fee applies
- Multiple CSVs per file
- CSVs without headers
- Tab separated or fixed width
- Joining in identifying information from other files
- Putting 2 CSVs in a single ZIP file
...instead, upload 2 ZIP files with 1 CSV each
- Uploading an orders file with no identifying information, only an order ID
...instead, put name, address, etc. on every order line before you upload!
- Encrypting before zipping
...instead, zip before encrypt
1. Send via Amazon S3 (our bucket)
First, we will create a bucket on our end.
To provision access to a bucket, Faraday will need an AWS username like:
Note: when you upload, you will need to set your ACL to:
We'll also need for you to put the following in your IAM user's
inline policy (example showing if the bucket name was "acme_to_faraday"):
2a. Send via Amazon S3 (your bucket)
Upload your files to a bucket you control.
Then, provision us access to a folder by defining a bucket policy:
(in the example below, it is for a folder
to_faraday/ in the bucket
Then, we have to add the bucket name to the IAM user here https://console.aws.amazon.com/iam/home?region=us-east-1#/users/stagecraft-download_s3
2b. S3 upload with SFTP user
We will require your SSH public key. They look like this:
Needless to say, they are NOT secret - that's why they're called "public" keys.
(Do not EVER send anybody, including Faraday, your SSH private key.)
If you don't have an SSH keypair, you can generate one like this https://www.digitalocean.com/docs/droplets/how-to/add-ssh-keys/create-with-openssh/.
Inputting details in our app:
You can navigate to: https://app.faraday.io/sources and hit "Create source"
Fill out the fields here:
File glob: to_faraday/$file_name_YYYYMMDD.csv
AWS bucket: acme-faraday-interchange
AWS region: us-east-1
Connecting and sending data:
Using your favorite SFTP client:
- Port: 22
- Username: varies by client (for example,
- Password: not enabled, you must utilize a SSH public key as explained above
Let's pretend you work for Acme, Inc. Your SSH private key file is
/path/to/acme_rsa (remember, this is a secret, don't sent it to us) and the username given to you by Faraday is
You will upload files like this:
$ sftp -i /path/to/acme_rsa firstname.lastname@example.org
Connected to email@example.com
$ sftp> put helloworld.csv.gz to_faraday/helloworld-2019-01-15.csv.gz
Uploading helloworld.csv.gz to /faraday-acme/to_faraday/helloworld-2019-01-15.csv.gz
You should be set!
A few notes:
- You won't be able to download or even see files once you send them to us - that's a security measure called a "blind dropbox."
- You need to upload it to the
to_faraday/folder as shown above.
- See above for requirements on file format, etc.
- The most secure option is to encrypt your file with GPG (aka PGP) before you send it to us. See https://knowledgebase.faraday.io/help-and-support/optimize/how-to-encrypt-files-with-gpg.