How to send data to Faraday

Check the format and required information

First, please see http://knowledgebase.faraday.io/help-and-support/data-sources/what-fields-are-required-when-sending-data-to-faraday

Decide what you're going to upload

For example, every day, you upload 2 files:

orders_2018-09-29.zip.gpg
...which contains a single CSV file, orders.csv
...every line of which has name, address
customers_2018-09-29.zip.gpg
...which contains a single CSV file, customers.csv
...every line of which has name, address, email

Extra fee applies

Multiple CSVs per file
CSVs without headers
Tab separated or fixed width
Joining in identifying information from other files

Common mistakes

Putting 2 CSVs in a single ZIP file
...instead, upload 2 ZIP files with 1 CSV each!
Uploading an orders file with no identifying information, only an order ID
...instead, put name, address, etc. on every order line before you upload!
Encrypting before zipping
...instead, zip before encrypt!

Send via Amazon S3 (our bucket)

If you want to upload to a bucket we control, just contact your CSM and they'll get it set up. We will need an AWS username like arn:aws:iam::$YOUR_AWS_ACCOUNT_ID:user/acme_to_faraday and then we will give it access to a bucket.
When you upload, be sure to set your ACL to bucket-owner-full-control .

Send via Amazon S3 (your bucket)

Upload your files to a bucket you control.
Then give us access to a folder (in the example below, I use the folder to_faraday/ in the bucket acme-faraday-interchange ) by defining a bucket policy:


{
  "Version": "2012-10-17",
  "Id": "FaradayAccessToBucket",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::113233973114:user/stagecraft-download_s3"
      },
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::acme-faraday-interchange"
      ]
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::113233973114:user/stagecraft-download_s3"
      },
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Resource": "arn:aws:s3:::acme-faraday-interchange/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::113233973114:user/stagecraft-download_s3"
      },
      "Action": [
        "s3:GetObject",
        "s3:GetObjectAcl",
        "s3:GetObjectVersion"
      ],
      "Resource": "arn:aws:s3:::acme-faraday-interchange/*"
    }
  ]
}

Then, we have to add the bucket name to the iam user here https://console.aws.amazon.com/iam/home?region=us-east-1#/users/stagecraft-download_s3

Upload with SFTP

First things first, we need your SSH public key. They look like this:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdtiZZsY6tR0JUWD1OtnkSFOK9wE3kiq5zNBwRKobpCsuSE3/vHZr20MeqhP7HhHvN/mlO81xvY9Hz4AfmslJWJAxw0kDbUj9Gz639FlnalUlNcMkSoG2iSYn/krB/6YG76Eb64CF14X+9aLKgyKrwgOc90AF0aD22Ig3ddsj0HW1YyxB0xH3NQVrdALQME+FILA+Fv20Ihmu8I/en8pdAcFa3vQDpE5aWc36w2/0xKU7luTwXjvJV0bOuDqMIe84KU4oHB0JrhnLTVxUDl+lEsmjZJqj7FBVV8Wg8Wqk0t3Iz1+4k2U68VPpCIY2m9fXkoh/rFJUYT4rRP+lwhJmt

Needless to say, they are NOT secret - that's why they're called "public" keys.
Do not EVER send us (or anybody else) your SSH private key.
If you don't have an SSH keypair, you can generate one like this https://www.digitalocean.com/docs/droplets/how-to/add-ssh-keys/create-with-openssh/. Be sure to store the private key safely - you will need it to connect in the future.
Go ahead and paste your public key into https://app.faraday.io/settings/integrations

Using your favorite SFTP client:

Host: s-9733f4cc1a6049ed9.server.transfer.us-east-1.amazonaws.com
Port: 22
Username: varies by client (for example, acme1 )
Password: not enabled, you must use an SSH public key as explained above
Folder: to_faraday/

Let's pretend you work for Acme, Inc. Your SSH private key file is /path/to/acme_rsa (remember, this is a secret, don't sent it to us) and the username given to you by Faraday is acme1 .
You will upload files like this:

$ sftp -i /path/to/acme_rsa acme1@s-9733f4cc1a6049ed9.server.transfer.us-east-1.amazonaws.com
Connected to acme1@s-9733f4cc1a6049ed9.server.transfer.us-east-1.amazonaws.com.
sftp> put helloworld.csv.gz to_faraday/helloworld-2019-01-15.csv.gz
Uploading helloworld.csv.gz to /faraday-acme/to_faraday/helloworld-2019-01-15.csv.gz

A few notes:

You won't be able to download or even see files once you send them to us - that's a security measure called a "blind dropbox."
You need to upload it to the to_faraday/ folder as shown above.
See above for requirements on file format, etc.
The most secure option is to encrypt your file with GPG (aka PGP) before you send it to us. See https://knowledgebase.faraday.io/help-and-support/optimize/how-to-encrypt-files-with-gpg.