Overview - Flatfile Developer Docs

Flatfile plugins extend your data processing capabilities with pre-built, reusable modules for common data operations. They handle everything from file extraction and data transformation to validation and external integrations, allowing you to build sophisticated data workflows quickly.

Open Source

All Flatfile plugins are open source and available on GitHub at github.com/FlatFilers/flatfile-plugins.

Plugin Categories

🔄 Transform

Plugins for data transformation, validation, and cleaning.

Record Hook

Run custom logic on individual data records in real-time

Autocast

Automatically cast values to appropriate data types

Constraints

Extend schemas with external validation constraints

Dedupe

Remove duplicate records via sheet-level actions

📤 Extractors

Plugins for parsing and extracting data from various file formats.

Delimiter Extractor

Parse delimited files (CSV, TSV, etc.)

XLSX Extractor

Extract data from Excel spreadsheets

JSON Extractor

Parse JSON files and convert to tabular data

PDF Extractor

Extract structured data from PDF documents

📋 Schemas

Plugins for converting external schemas to Flatfile format.

JSON Schema Converter

Convert JSON Schema to Flatfile Blueprint

OpenAPI Schema Converter

Convert OpenAPI schema to Flatfile Blueprint

SQL DDL Converter

Convert SQL DDL to Flatfile Blueprint

YAML Schema Converter

Convert YAML Schema to Flatfile Blueprint

📤 Export

Plugins for exporting processed data to external systems.

Export Workbook

Export data from Flatfile to various formats

Webhook Egress

Send data to external webhooks

Export Delimited ZIP

Export workbooks as delimited files in ZIP archives

Export Pivot Table

Generate pivot tables from sheet data

🔧 Core

Essential plugins for Flatfile operations and configuration.

Space Configure

Configure Flatfile spaces with workbooks and settings

Job Handler

Handle Flatfile jobs with custom logic

View Mapped

Show only mapped columns in post-mapping view

Rollout

Automatically roll out workbook changes

✅ Validation

Specialized validation plugins for data quality.

Validate Email

Comprehensive email address validation

Validate Phone

Phone number formatting and validation

Validate Date

Date format normalization and validation

Validate Number

Number validation and formatting

🔗 Enrich

Plugins for data enrichment using external APIs.

Geocode

Geocode addresses using Google Maps API

Sentiment Analysis

Analyze sentiment of text fields

Text Summarization

Summarize and extract key phrases from text

What3Words

Convert What3Words addresses to standard addresses

Getting Started with Plugins

Installation

Install plugins using npm in your Flatfile project:

npm install @flatfile/plugin-record-hook @flatfile/plugin-autocast

Basic Usage

Add plugins to your listener:

import { FlatfileListener } from '@flatfile/listener'
import { recordHook } from '@flatfile/plugin-record-hook'
import { autocast } from '@flatfile/plugin-autocast'

const listener = FlatfileListener.create(async (client) => {
  // Add autocast for automatic type conversion
  client.use(autocast())
  
  // Add record hook for custom validation
  client.use(
    recordHook('customers', (record) => {
      // Custom record processing logic
      const email = record.get('email')
      if (email && !isValidEmail(email)) {
        record.addError('email', 'Invalid email format')
      }
      return record
    })
  )
})

Plugin Configuration

Most plugins accept configuration options:

import { configureSpace } from '@flatfile/plugin-space-configure'
import { dedupePlugin } from '@flatfile/plugin-dedupe'

client.use(
  configureSpace({
    workbooks: [workbookConfig],
    space: {
      name: "Customer Import",
      namespace: "customers"
    }
  })
)

client.use(
  dedupePlugin({
    sheetSlug: 'customers',
    dedupe: {
      key: 'email',
      keep: 'first',
      custom: {
        operation: 'dedupe-customers',
        label: 'Remove Duplicate Customers'
      }
    }
  })
)

Plugin Development

Creating Custom Plugins

You can create custom plugins for reusable functionality:

import { FlatfilePlugin } from '@flatfile/listener'

export function myCustomPlugin(options: any): FlatfilePlugin {
  return (client) => {
    // Plugin initialization logic
    client.filter({ job: 'sheet:myCustomOperation' }, (configure) => {
      configure.on('job:ready', async (event) => {
        // Plugin functionality
        const { jobId } = event.context
        
        try {
          await client.jobs.ack(jobId, {
            info: 'Running custom operation...'
          })
          
          // Your custom logic here
          await performCustomOperation(options)
          
          await client.jobs.complete(jobId, {
            outcome: { message: 'Custom operation completed successfully' }
          })
        } catch (error) {
          await client.jobs.fail(jobId, {
            outcome: { message: `Operation failed: ${error.message}` }
          })
        }
      })
    })
  }
}

// Usage
client.use(myCustomPlugin({ customOption: 'value' }))

Plugin Best Practices

Single Responsibility: Each plugin should handle one specific functionality
Configuration: Accept configuration options for flexibility
Error Handling: Implement robust error handling and logging
Documentation: Provide clear documentation and examples
Testing: Include comprehensive tests for your plugin

Popular Plugin Combinations

Basic Data Import

// Essential plugins for most data import scenarios
client.use(autocast())
client.use(recordHook('sheet-name', validateRecord))
client.use(dedupePlugin({ sheetSlug: 'sheet-name' }))

Advanced Validation

// Comprehensive data validation
client.use(autocast())
client.use(recordHook('customers', validateCustomer))
client.use(validateEmail({ sheetSlug: 'customers', fieldKey: 'email' }))
client.use(validatePhone({ sheetSlug: 'customers', fieldKey: 'phone' }))
client.use(validateDate({ sheetSlug: 'customers', fieldKey: 'birthdate' }))

Schema Conversion

// Convert external schemas to Flatfile
client.use(convertJsonSchema({ source: 'api-schema.json' }))
client.use(convertOpenApiSchema({ source: 'swagger.yaml' }))

Data Enrichment

// Enrich data with external services
client.use(enrichGeocode({ 
  sheetSlug: 'addresses',
  addressField: 'full_address'
}))
client.use(enrichSentiment({
  sheetSlug: 'feedback', 
  textField: 'comments'
}))

Plugin Repository

All Flatfile plugins are open source and available on GitHub:

Main Repository: github.com/FlatFilers/flatfile-plugins
NPM Organization: @flatfile
Plugin Marketplace: flatfile.com/plugins

Contributing

We welcome contributions to the Flatfile plugin ecosystem:

Bug Reports: Report issues on the GitHub repository
Feature Requests: Suggest new plugins or enhancements
Pull Requests: Contribute code improvements or new plugins
Documentation: Help improve plugin documentation

Support

Documentation: Each plugin includes detailed documentation
Community: Join our Slack community for support
GitHub Issues: Report bugs or request features on GitHub
Professional Support: Contact our team for enterprise support

​Open Source

​Plugin Categories

​🔄 Transform