English (English)

Scheduled File Upload via SFTP

One of the channels for uploading data to ABBYY Timeline is SFTP. You can upload files to Timeline automatically on a schedule or manually using any standard SFTP client that does not require read access to the destination folder. For example, you can use the free OpenSSH utility for Linux, macOS, and Windows and WinSCP for Windows and macOS.

Timeline configures the SFTP destination and shares the location and credentials with the client for the scheduled file push. The data upload from the SFTP server is then an automated process where the files are automatically picked up and processed to a project or repository.

Important. Before you get started, make sure you're aware of the limitations when using scheduled file upload via SFTP. For details, see Limitations when uploading via SFTP below.

Basic steps to use file upload if use Timeline in the cloud through an ABBYY SAAS instance

This instruction will help you set up a scheduled data upload if you are using Timeline through an ABBYY SAAS instance in the cloud. To configure scheduled data upload if use Timeline instance deployed on-premises on Linux or Windows systems, go to Basic steps to use scheduled file upload if Timeline instance deployed on-premises

  1. Generate your public key (which everybody is allowed to know) and a private key (which you keep secret and do not give to anybody).
    Please refer to your SFTP client documentation for instructions or examples below.
  2. Contact Timeline support at support@abbyy.com and send the public key.
    Support will send you a username and SFTP server name in response.
  3. Create a manifest.json file.
    This manifest in JSON format contains instructions on uploading data files to Timeline.
    For detailed instructions on creating a manifest file, see Manifest file.
  4. ZIP together the manifest and your data file(s) and upload the ZIP to the SFTP server.
    See Data Requirements for the requirements that your uploaded data must meet.
    All files to be uploaded must be listed in a single manifest.json file and zipped along with it. manifest.json must be at the root of ZIP.

    Examples of ZIPs

  1. Timeline will send an email notification with the results of data uploading and processing to the email specified in the manifest.
    Notification will also have documents attached to them with additional information about the reported upload events.
    More detailed information about each upload event is also available in the History of the corresponding project or repository (depending on where you uploaded your data).

Basic steps to use file upload if Timeline instance deployed on-premises

This instruction will help you set up a scheduled data upload if you are using a Timeline instance deployed on-premises on Linux or Windows systems. When installing on-premises, you can use the background-upload feature that involves a folder that is monitored for files copied there. Whenever a new ZIP file is detected in that folder, the application grabs it and interprets it as an uploaded archive.

  1. Create a manifest.json file.
    This manifest in JSON format contains instructions on uploading data files to Timeline.
    For detailed instructions on creating a manifest file, see Manifest file.
  2. ZIP your data file(s).
    See Data Requirements for the requirements that your uploaded data must meet.
  3. Copy the manifest.json and ZIP to the destination folder on the server running Timeline:
    1. If Timeline deployed on-premises on Linux, this is the folder specified in STORAGE/sftp variable in the .env file.
      Default value: /opt/timeline/storage
    2. If Timeline deployed on-premises on Windows, this is the $INSTALLDIR/storage/sftp folder.

Limitations when uploading via SFTP

Use upload via SFTP as scheduled file upload, not as real-time data upload.

  1. Keep time interval between uploads since it takes some time to upload and process data.
  2. It is recommended to upload files no more than once per 5 minutes. The upload time may increase when uploading large files and executing a large number of operations in a repository.
  3. A project and repository are unavailable during the data upload and post-processing.
    Timeline allows you to select multiple sources to upload data, but it is allowed to use only one data source to upload at a time. In case uploads initiated from different sources overlap in time, only the upload initiated first will be successful.
    Examples:
    1. While data is being uploaded to a repository with the preconfigured Load into project operation, an overlapped upload attempt to the same project may fail because the project will be unavailable until Load into project operation initiated by the repository completes.
    2. While a user uploads data to the project via the Timeline website, the upload via SFTP to the same project may fail due to the project being unavailable until the user-initiated upload completes. Same for a repository. The repository is unavailable for upload via SFTP until the upload to the same repository, initiated manually by the user on the Timeline website is completed.

Example of generating a public key

The instruction below shows how to generate a public key using the PuTTY application. Instructions for your SFTP client might be different from the example.

To generate a public key on a Windows machine:

To generate a public key on a MAC or Linux machine:

22.09.2023 8:59:47

Usage of Cookies. In order to optimize the website functionality and improve your online experience ABBYY uses cookies. You agree to the usage of cookies when you continue using this site. Further details can be found in our Privacy Notice.