Jobs & Actions

In Flatfile, a Job represents a large unit of work performed asynchronously on a resource such as a file, WorkbookAnalogous to a database..., or SheetA collection of fields.... The Jobs workflow provides visibility into the status and progress of your Jobs, allowing you to monitor and troubleshoot the data processing pipeline.

For example, a Job could represent:

• a data export
• a remapping of data from one sheet to another
• an action in the UI that triggers a Job

​

Types of Jobs

There are 4 types of Jobs:

1. Extracting data from a file (moving data from a file to a workbook)
2. Mapping data from one workbook to another (moving data from a workbook to a Workbook)
3. Deleting data from a Workbook (operating on data in a single Workbook)
4. Exporting data from a Workbook

All of these operate on (potentially) large amounts of data and complete asynchronously; you don’t want to wait around for them, so we notify you when they’re done.

​

The Anatomy of a Job

Flatfile Jobs follow a standard structure:

• type - Workbook, file, Sheet
• operation - export, extract, map, delete
• source - id for the data source
• destination? - id for the data target (if any)
• config - a configuration object specific to the operation
• trigger - manual or immediate

In addition, Jobs track their execution state with:

• status - PLANNING, READY, EXECUTING, COMPLETED, FAILED
• parts - how many parts the Job was broken into
• processed - how many of those parts have been processed so far

​

Using Jobs

An action in the UI can trigger a Job by POST-ing to the /jobs API endpoint. If the trigger is immediate the Job will run immediately; otherwise it can be triggered with a POST to the /jobs/:id/execute endpoint.

​

Job Lifecycle Events

Jobs emit the following events throughout their lifecycle that you can listen for:

• job:waiting - emitted after a Job is created but before it begins executing
• job:started - emitted right before a Job starts execution
• job:updated - emitted when a Job’s progress is updated
• job:failed - emitted when a Job has failed
• job:completed - emitted when a Job has completed successfully
• job:deleted - emitted when a Job has been deleted

​

Building with Jobs

How you build your UI is up to you, but here is one possibility:

1. Create a component that is used to trigger a Job
2. When the Job is triggered, POST to the /jobs endpoint
3. When the Job is complete, the server will send a job:completed event
4. When the job:completed event is received, update the UI to reflect the new state