Creating virtual fields dynamically

With the addVirtualField method, you have the ability to create or hide a field programmatically that isn't in the file, but what if we want to see if a field exists before we try to create it? What if we want to create a field only if another field exists? While this simple method is the way to create a new field during an import, you need to know what's happening within the data during the import so that you can tell if a field needs to be created. This is where Portal's Step Hooks come into play.

By using Step Hooks, we have to ability to see what headers exist in the data set and what they are matched to. This information can be used to create a field.

note

These two things are not enabled by default on every account. To enable these features, please reach out to our support team at support@flatfile.com.

Example

Let's visualize this with a code example. In this scenario, let's say that we want to have a full_name field in our data that consists of a first and a last name. Some of your users might bring you a file with a full name in it, which is your ideal scenario, but other users might be bringing you a file with a first_name and a last_name field. If your desired output is a full name, you can use addVirtualField() with registerStepHook() to see if you have a full_name field or if you need to create one. You can then use Data Hooks and a little bit of JavaScript to combine them into the full_name field. Let's assume our data looks like what is in the below table.

First name	Last name
Marshall	Erickson
Winston	Schmidt
Jim	Brockmire
Stanley	Hudson

With this in mind, we can setup our schema to have the ability to take in a first name and last name or full name field.

const importer = new FlatfileImporter("license_key", {
  type: "import type",
  fields: [
    { key: "full_name", label: "Full Name" },
    { key: "first_name", label: "First Name" },
    { key: "last_name", label: "Last Name" },
  ],
});

Now that we have our schema defined, we can use Step Hooks to determine whether or not we should be creating a full name field. Keep in mind that while it is possible to reformat and run code inside the registerStepHook() callback, it is expected that this callback returns a boolean value. In order to proceed to the next step of the importer, we will want to return true in the callback. Inside this callback, we will use the addVirtualField() method to create a full_name field if one isn't mapped initially.

importer.registerStepHook("review", ({ headers_matched }) => {
  if (
    !headers_matched.find((v) => v.matched_key === "full_name") &&
    headers_matched?.find((v) => v.matched_key === "first_name") &&
    headers_matched.find((v) => v.matched_key === "last_name")
  ) {
    importer.addVirtualField(
      {
        label: "Full Name",
        key: "full_name",
        description: 'Only create this field when columns "First Name" and "Last Name" are matched',
      },
      {
        order: 1,
      },
    );
  }
  return true;
});

Now that we have determined if need to create the full_name value and have created it (if necessary), we can use the registerRecordHook() method to combine the values from first_name and last_name into the full_name field.

importer.registerRecordHook((record, index, mode) => {
  let out = {};
  if (record.first_name && record.last_name && mode === "init") {
    out.full_name = {
      value: record.first_name + " " + record.last_name,
      info: [
        {
          message: "combined first and last name into full name",
          level: "info",
        },
      ],
    };
  }
  return out;
});

You would now have your full_name field populated with full names despite the fact that the source data came in with first and last name separated.

tip

Want to play around with this example? We have this code working in a code playground here.