When embedding Flatfile, this plugin should be deployed in a server-side listener. Learn more

@flatfile/plugin-dedupe


This plugin dedupes records in a sheet via a sheet level custom action.



Event Type:


listener.on('job:ready')

​
Parameters

jobOperation
string
required

The jobOperation parameter specifies the name of the job operation to match on.

opt.on
string

The on parameter specifieds which field key to match on.

opt.keep
'first' | 'last'

The keep option lets you choose whether to keep the first or last duplicate record.

opt.custom
function

The custom parameter accepts a custom dedupe function. This will override the keep parameter.

opt.debug
boolean

The debug parameter lets you toggle on/off helpful debugging messages for development purposes.

​
API Calls

  • api.records.get
  • api.jobs.ack
  • api.records.delete
  • api.jobs.fail
  • api.jobs.complete

​
Imported NPM Packages

​
Usage

An action with the operation name of “dedupe-email” must be configured on a Sheet in order to the plugin to be triggered.

install
npm i @flatfile/plugin-dedupe
import
import { dedupePlugin } from "@flatfile/plugin-dedupe";

// ... inside the Sheet configuration
"actions": [
  {
    "operation": "dedupe-email",
    "mode": "background",
    "label": "Dedupe emails",
    "description": "Remove duplicate emails"
  }
]
// ...

// common usage
// Keep the last record encountered (from top to bottom) based on the`email` field key.
// Must have a Sheet level action specified with the operation name `dedupe-email`
listener.use(
  dedupePlugin("dedupe-email", {
    on: "email",
    keep: "last",
  })
);

// user specified dedupe function using RemedaJS
// must return a list a record id's for deletion
listener.use(
  dedupePlugin("dedupe-email", {
    custom: (records) => {
      let uniques = new Set();
      return R.pipe(
        records,
        R.reduce((acc, record) => {
          const { value } = record.values["email"];

          if (uniques.has(value)) {
            return [...acc, record.id];
          } else {
            uniques.add(value);

            return acc;
          }
        }, [] as Array<string>)
      );
    },
  })
);

​
See the code

@flatfile/plugin-dedupe

@flatfile/plugin-autocast@flatfile/plugin-export-workbook