Data Requirements
Process analysis allows you to assess specific events in a chain of structured business processes.
Some examples of business processes include receiving orders, invoicing, shipping products, updating employee information, or servicing a customer. Business processes occur at all levels of an organization’s activities and include events that the customer sees and events that are hidden in information systems. The term also refers to the mix of all the separate steps toward the final business goal.
What kind of data does ABBYY Timeline require?
ABBYY Timeline assumes the existence of an event log where each event refers to a case, an activity, and a point in time. An event log can be seen as a collection of cases and a case can be seen as a trace/sequence of events.
Event data may come from a wide variety of sources:
- a database system (e.g., patient data in a hospital)
- a comma-separated values (CSV) file or spreadsheet
- a transaction log (e.g., a trading system)
- a business suite/ERP system (SAP, Oracle, etc.)
- a message log (e.g., from IBM middleware)
Example Data Set
| Datetime | OrderID | Workflow Step | Employee | Location | Day Of Week | Shift |
|---|---|---|---|---|---|---|
| 2016-02-07 10:50:00+00 | 100012 | Order Received | Thomas Moore | Philadelphia | Thurs | 3rd |
| 2016-02-08 00:49:00+00 | 100012 | Existing Customer Check | Thomas Moore | Philadelphia | Thurs | 3rd |
| 2016-02-08 08:03:00+00 | 100012 | Stock Check | Thomas Moore | Philadelphia | Thurs | 3rd |
| 2016-02-09 11:40:00+00 | 100012 | Order Hold | Thomas Moore | Philadelphia | Thurs | 3rd |
| 2016-02-11 01:39:00+00 | 100012 | Stock Packed | Thomas Moore | Philadelphia | Thurs | 3rd |
| 2016-02-11 17:37:00+00 | 100012 | Goods Shipped | Thomas Moore | Philadelphia | Thurs | 3rd |
| 2016-02-13 14:30:00+00 | 100012 | Goods Accepted | Thomas Moore | Philadelphia | Thurs | 3rd |
| 2016-02-14 17:42:00+00 | 100012 | Order Complete | Thomas Moore | Philadelphia | Thurs | 3rd |
Data File Requirements for ABBYY Timeline
The most common mechanism to load your data into the ABBYY Timeline platform is via a CSV file. This document describes the required structure of the file, along with some tips and tricks related to the generation of the file.
File Structure
The data should be placed into a comma-separated file. Each row in the file represents an event – a record that something happened to a specific object at a particular time. The file must have three mandatory columns and can include any number of optional columns. All columns can have arbitrary names, as no naming rules are imposed, however no two columns should have the same name.
The mandatory columns are:
- TimelineID
A column for some identifier of an object you want to track over time. This could be an Order ID, Claim ID, Patient Encounter Number, Support Ticket Number, and so on. - Timestamp
A column for the timestamp showing when something happened in the life of the object. This column generally contains a date and time (see format description below). If a date with no time is provided, midnight (12:00 AM, 00:00:00) will be used. - Event Name
A column describing what happened to the object at that time – Order Submitted, Patient Departed, Adjuster Assigned, Ticket Escalated etc.
Optional columns
In addition to the mandatory columns, the file can contain any number of additional columns which will be used as dimensional attributes. You can filter by these fields, group and break down by them, or use them as additional information when analyzing the processes.
Order of Records
The order of the records in the file doesn’t matter, with one exception. If several records, related to the same object, have the exact same timestamp, the application will keep them in the same order they were placed in the file.
File Sample
| TimelineID | Timestamp | Event name | Employee | Location |
|---|---|---|---|---|
| A | 1/16/2017 7:20:15 | Student Applied | John | Boston |
| A | 3/10/2017 16:54:10 | Student Accepted | Mary | Boston |
| A | 4/11/2017 15:04:00 | Bill Generated | Ann | Charlotte |
| B | 2/1/2017 9:15:00 | Student Applied | John | Boston |
| B | 3/2/2017 16:20:05 | Student Accepted | Mary | Boston |
Here, A and B are the identifiers of the traceable objects, in our example – Students. Event names are Student Applied, Student Accepted etc.
In a CSV file, it looks like
TimelineID;Timestamp;Event name;Employee;Location
A;1/16/2017 7:20:15;Student Applied;John;Boston
A;03.10.2017 16:54;Student Accepted;Mary;Boston
A;04.11.2017 15:04;Bill Generated;Ann;Charlotte
B;02.01.2017 9:15;Student Applied;John;Boston
B;03.02.2017 16:20;Student Accepted;Mary;Boston
Common Issues
The most common issues with data files are a bad date/time format and a broken file format.
Date/Time Format
Make sure you save timestamps in the file in one of the following formats:
- 1/6/2017 7:20:15
- 1/6/2017 7:20:15 AM
- 2017-01-06 7:20:15
- 2017-01-06T7:20:15Z
Note: The default Time Format in Excel doesn’t include seconds. In order to save seconds into a CSV file, switch to Custom Formats in Excel, select "m/d/yyyy h:mm" and change it to "m/d/yyyy h:mm:ss".
File Format
If the values in any field include commas, the format of the file may break. To avoid this, make sure to specify double quotes as the string qualifier. Microsoft Excel does it automatically, however, some tools like MS SQL Export Wizard require manual settings.
The file should be Locale English (United States) and US ASCII or UTF-8 encoded.
9/22/2023 8:59:47 AM