Jobs & Actions
In Flatfile, a Job represents a large unit of work performed asynchronously on a resource such as a file, WorkbookAnalogous to a database..., or SheetA collection of fields.... The Jobs workflow provides visibility into the status and progress of your Jobs, allowing you to monitor and troubleshoot the data processing pipeline.
For example, a Job could represent:
- a data export
- a remapping of data from one sheet to another
- an action in the UI that triggers a Job
Types of Jobs
There are 4 types of Jobs:
- Extracting data from a file (moving data from a file to a workbook)
- Mapping data from one workbook to another (moving data from a workbook to a Workbook)
- Deleting data from a Workbook (operating on data in a single Workbook)
- Exporting data from a Workbook
All of these operate on (potentially) large amounts of data and complete asynchronously; you don’t want to wait around for them, so we notify you when they’re done.
The Anatomy of a Job
Flatfile Jobs follow a standard structure:
type
- Workbook, file, Sheetoperation
- export, extract, map, deletesource
- id for the data sourcedestination?
- id for the data target (if any)config
- a configuration object specific to the operationtrigger
- manual or immediate
In addition, Jobs track their execution state with:
status
- PLANNING, READY, EXECUTING, COMPLETED, FAILEDparts
- how many parts the Job was broken intoprocessed
- how many of those parts have been processed so far
Using Jobs
An action in the UI can trigger a Job by POST-ing to the /jobs
API endpoint.
If the trigger
is immediate
the Job will run immediately;
otherwise it can be triggered with a POST to the /jobs/:id/execute
endpoint.
Job Lifecycle Events
Jobs emit the following events throughout their lifecycle that you can listen for:
job:waiting
- emitted after a Job is created but before it begins executingjob:started
- emitted right before a Job starts executionjob:updated
- emitted when a Job’s progress is updatedjob:failed
- emitted when a Job has failedjob:completed
- emitted when a Job has completed successfullyjob:deleted
- emitted when a Job has been deleted
Building with Jobs
How you build your UI is up to you, but here is one possibility:
- Create a component that is used to trigger a Job
- When the Job is triggered, POST to the
/jobs
endpoint - When the Job is complete, the server will send a
job:completed
event - When the
job:completed
event is received, update the UI to reflect the new state