Build custom file processing plugins to handle unique data formats and transform files into structured data
@flatfile/util-extractor
utility, which provides a standardized framework for file processing:
file:created
event and processes your files.
Component | Purpose | Required |
---|---|---|
File Extension | String or RegExp of supported file extension(s) | ✓ |
Extractor Type | String identifier for the extractor type | ✓ |
Parser Function | Core logic that converts file buffer to structured data | ✓ |
Options | Configuration for chunking, parallelization, and customization | - |
file:created
event fires.pipe
or .psv
extension that look like this:
.pipe
or .psv
extension.
.multi
or .sections
extension that look like this:
.bin
or .dat
extension. Due to the nature of binary format, we can’t easily present a sample import here.
.custom
extension that look like this, while transforming dates and amount values:
.pipe
or .special
extension that look like this:
Parameter | Type | Description |
---|---|---|
fileExt | string or RegExp | File extension to process (e.g., ".custom" or /\.(custom|special)$/i ) |
extractorType | string | Identifier for the extractor type (e.g., “custom”, “binary”) |
parseBuffer | ParserFunction | Function that converts Buffer to WorkbookCapture |
options | Record<string, any> | Optional configuration object |
Option | Type | Default | Description |
---|---|---|---|
chunkSize | number | 5000 | Records to process per batch |
parallel | number | 1 | Number of concurrent processing chunks |
debug | boolean | false | Enable debug logging |
parseBuffer
function receives additional options beyond what you pass to Extractor
:
Option | Type | Description |
---|---|---|
fileId | string | The ID of the file being processed |
fileExt | string | The file extension (e.g., “.csv”) |
headerSelectionEnabled | boolean | Whether header selection is enabled for the space |
WorkbookCapture
object:
Flatfile.RecordData
format:
Type | Description | UI Effect |
---|---|---|
error | Validation error | Red highlighting, blocks submission |
warning | Warning message | Yellow highlighting, allows submission |
info | Informational message | Blue highlighting |
fileExt
configuration