Skip to content

SFTP Delivery

Send Workbench can deliver files to external parties via SFTP. This is typically used for bordereau exports, scheduled data feeds to Lloyd's systems, and bulk document delivery to coverholders or managing agents.

The SFTP Out service (riskwire-sftp-out) is an Apache Camel-based gateway that reads from an SQS queue and transfers files to a configured SFTP endpoint.

Note

Exact pipeline component names (e.g. SEND_SFTP), attribute keys, and named-connection structures below should be confirmed against your current core platform version and environment profile — SFTP integrations are frequently customised per client and the contract between the pipeline and the SFTP Out service has evolved over time.

How SFTP delivery works

Pipeline (trigger) → SQS → SFTP Out (Camel) → Remote SFTP Server

A pipeline step publishes the file (and its destination metadata) to the SQS queue. The SFTP Out service picks up the message and establishes an SFTP connection to deliver the file. On success, the delivery is audited and the pipeline continues.

SFTP connection configuration

SFTP connection details are set in pde.dev.json (local) or the DynamoDB environment config (production):

{
  "sftpOut": {
    "host": "sftp.example.com",
    "port": 22,
    "username": "workbench-delivery",
    "password": "${SFTP_PASSWORD}",
    "remoteDirectory": "/inbound/send",
    "knownHosts": "/etc/ssh/known_hosts"
  }
}
Field Description
host Hostname or IP of the remote SFTP server
port SFTP port (default 22)
username SFTP account username
password SFTP password — inject via environment variable
remoteDirectory Default remote directory for file delivery
knownHosts Path to SSH known_hosts file for host key verification

Warning

SFTP credentials are sensitive. Always inject passwords via environment variables or AWS Secrets Manager. Never store them in the config repository.

Key-based authentication

For production environments, prefer SSH key authentication over password authentication:

{
  "sftpOut": {
    "host": "sftp.example.com",
    "port": 22,
    "username": "workbench-delivery",
    "privateKeyPath": "/secrets/sftp_rsa",
    "privateKeyPassphrase": "${SFTP_KEY_PASSPHRASE}",
    "remoteDirectory": "/inbound/send"
  }
}

The private key file must be accessible to the SFTP Out service container. In production this is mounted via AWS Secrets Manager or a Kubernetes secret.

Triggering SFTP delivery from a pipeline

SFTP delivery is triggered by a SEND_SFTP pipeline component. A typical pipeline step looks like:

{
  "name": "Deliver Bordereau to Coverholder",
  "componentType": "SEND_SFTP",
  "attributes": {
    "remoteDirectory": "/inbound/coverholder/{{coverholderCode}}",
    "filename": "bordereau_{{reportingPeriod}}.xlsx"
  }
}

The remoteDirectory and filename fields support Handlebars-style placeholders referencing risk or declaration data from the pipeline context.

S3-to-SFTP pipeline

The core config repository includes a reusable pipeline for pulling a file from S3 and delivering it to SFTP:

config/core/pipelines/s3_pull_to_sftp.json

This is the standard pattern for bordereau report delivery. The pipeline:

  1. Retrieves the file from the specified S3 path
  2. Connects to the configured SFTP server
  3. Transfers the file to the specified remote directory
  4. Audits the delivery

Reference this pipeline from your action or scheduled job rather than building the SFTP logic from scratch.

Scheduled SFTP delivery

SFTP exports are often scheduled (e.g. monthly bordereau delivery). Scheduled pipelines are configured using the schedule-column-mapping configs in the core config repository and triggered by a Cron-based pipeline step.

For bordereau-specific scheduling, see Bordereau Ingestion.

Multiple SFTP destinations

If your implementation requires delivery to more than one SFTP server (e.g. a different server per coverholder), you can define multiple named SFTP configurations and reference the appropriate one in each pipeline:

{
  "sftpConnections": {
    "coverholder_abc": {
      "host": "sftp.abc-coverholder.com",
      "port": 22,
      "username": "send-feed",
      "password": "${SFTP_ABC_PASSWORD}"
    },
    "lloyds_bureau": {
      "host": "sftp.lloyds.com",
      "port": 22,
      "username": "send-feed",
      "privateKeyPath": "/secrets/lloyds_rsa"
    }
  }
}

Reference the named connection in your pipeline component:

{
  "componentType": "SEND_SFTP",
  "attributes": {
    "connectionName": "lloyds_bureau",
    "remoteDirectory": "/send/upload",
    "filename": "{{filename}}"
  }
}

File format considerations

The SFTP Out service is format-agnostic — it transfers whatever file is in the SQS message payload. The file must be generated by a preceding pipeline step (e.g. a document generation step or a data export step) before the SFTP component runs.

Common file types delivered via SFTP: Excel bordereau files (.xlsx), CSV exports, PDF policy documents, XML data feeds.

Local development

For local testing, use an SFTP test container. A lightweight option:

sftp-test:
  image: atmoz/sftp
  command: testuser:testpass:::upload
  ports:
    - "2222:22"

Configure your local SFTP Out connection to use localhost:2222 and verify file delivery by SSHing into the test container.

See also