The key file required for the data upload is the manifest file. This file contains information on what and where to upload. The file is in JSON format. It could be created in any simple text editor or using variety of online JSON editors.
Important.
- The manifest file must be named manifest.json since Timeline accepts only manifest files with this name.
- All files to be uploaded must be listed in a single manifest.json file and zipped along with it.
- Place manifest.json only in the root of the ZIP file, and data files can be placed in subfolders.
- The manifest file must contain either repositoryToken or projectToken but not both.
For instructions on setting up the manifest, select one of the options below, depending on where you plan to upload the data.
Manifest with the repositoryToken to upload data to a repository
The table below describes the nodes that are in the manifest file.
| Node |
Description |
| repositoryToken |
Encrypted strings identifying the repository you are uploading data to.
To obtain this string:
- Select View > Repository > Details > Data Sources.
- Check Scheduled file upload.
- Click Show and copy token to clipboard.
|
| email |
A list of email addresses, separated by semi-column.
The reports about successful or failed uploads will be sent to the specified emails. Timeline also always sends an email to the repository owner.
|
| commands |
The ordered list of various commands which will be executed once the upload package is received and unzipped.
Commands has actions and optional arguments.
- upload
Parses the file(s) and puts the data into the table.
Arguments:
- file – name of the file from the ZIP.
- table – name of repository table into which the data from file(s) will be uploaded. If the table doesn’t exist, it will be created.
- keepOldData – if true, Timeline keeps the old data in the selected table.
Note. The argument is optional.
- clone
Copies a repository table and all data in it.
Arguments:
- src – name of the table to be copied.
- dst – name of the table into which the data from the file will be copied. If the table doesn’t exist, it will be created.
- To-do
Executes a to-do list from the repository.
You can create to-do lists within a Timeline Repository, which are essentially a list of data transformation functions. These typically end in uploading that data to a project. If a data file does not require any transformation, the to-do list can specify that it be uploaded directly to a project using the predefined project table mapping. For more information about To-do list, see ETL in the Cloud.
Important. To upload file(s) to a project you must first perform a manual upload and map the data. Timeline will automatically save a mapping after the data file has been successfully uploaded to a project. When uploading files of the same type, Timeline maps the uploaded data automatically. If no previously saved mapping is found the upload fails.
- Arguments:
- table – name of the table on which the to-do list will be executed.
- list – name of an existing to-do list.
|
Manifest with the projectToken to upload data to a project
The table below describes the nodes that are in the manifest file.
| Node |
Description |
| projectToken |
Encrypted strings identifying the project you are uploading data to.
To obtain this string:
- Select Project > Details > Data Sources.
- Check Scheduled file upload.
- Click Show and copy token to clipboard.
Important. If you expect to upload large datasets to your project, be sure to turn on Automatically switch to Large Data Mode in Project > Details > Data Sources.
|
| email |
A list of email addresses, separated by semi-column.
The reports about successful or failed uploads will be sent to the specified emails. Timeline also always sends an email to the project owner.
|
| commands |
A command to be executed once the upload package is received and unzipped.
Command has action and optional arguments.
- upload
The action parses file(s) and puts the data to the existing project using the mapping predefined for this project.
Arguments:
- file – name of the file from the ZIP.
- clearProject – if true, old data from the project will be deleted before new data is loaded.
Note. The argument is optional.
- mapNewColumnsAsAttributes – if true, all optional columns except the mandatory columns Timeline ID, Timestamp, and Event Name columns are added to the project as attributes.
Note. The argument is optional.
Example. Use this argument if you plan to upload a file with more optional columns in the table than defined in the preconfigured mapping. When upload, data from the new columns upload as new attributes. If you omit the mapNewColumnsAsAttributes argument from the manifest file, any optional columns that were not previously mapped will not be uploaded.
Important. To upload file(s) to a project you must first perform a manual upload and map the data. Timeline will automatically save a mapping after the data file has been successfully uploaded to a project. When uploading files of the same type, Timeline maps the uploaded data automatically. If no previously saved mapping is found the upload fails.
|
Examples of manifest
The following manifest file takes file events.csv from the ZIP, parses it into the existing repository table mytable, then copies the table into yourtable (an existing data in the table is kept), and then applies To-do list mylist to it. Report will be sent to the repository owner, yourname@company.com, and yourcolleague@company.com.
The next example parses the file events.csv and uploads it to a project identified by the token. Old data from the project is deleted.