Setup headless file feeds
File feeds are one of the most common ways to exchange batches of data on a regular basis. One key challenge for a traditional file feed, however, is its inflexibility. If anything changes about the way source data is extracted, or the destination schema is configured, the feed is all but guaranteed to break.
Flatfile offers an alternative way to think about file feeds: as adaptable data connections that leverage a human in the loop, when necessary.
How it works
Because of Flatfile’s Event-driven architecture, it’s easy to set up a SpaceA micro-application... as a destination for any incoming files. You can write a cron job, Lambda function, or Python script that targets the Space with incoming files.
Extract data from files
You can listen for
file:uploaded and subsequently trigger any custom extraction logic. Extractions are JobLarge asynchronous work... that process asyncronously.
(Note: CSVs uploaded to Flatfile will always be immediately extracted to a raw Workbook.)
Mapping in Flatfile is a unique type of Job. Each mapping execution is driven by a Job Plan that is progressively improved over time. After the first couple of Files are mapped to a particular BlueprintA Data Definition Language..., Flatfile will automatically suggest field and category-level mappings. While listening for an extraction Job to complete, you can poll the Space for the most recently used Job Plan and then execute a mapping Job.
Automate validation and transformation
Similarly, validation and pre-configured transformation operations can run immediately on
Automate an egress Job
Finally, you can configure a Job to run immediately after any validation / transformation is completed in a Workbook.
You could choose to egress all data regardless of its validity. Another option is to only egress
valid data and notify the appropriate user that attention is needed on any
(Suggestion: You can also have the Job set a timestamp for the most recent data sync using the
This gives you a consistent point-in-time reference for your syncing logic.)
To find the additional Flatfile integration paths that may work better for your business, explore our other core paths: