A complete guide to importing data with Data Hooks

We all know how important it is to quickly get customers to the " and that successfully onboarding customers plays a huge role in that process. It sounds obvious, but what many companies don't fully recognize is that the data itself is one of the most important parts of customer data onboarding.

To deal with data import challenges, product teams are saddled with building out complex CSV importers, while customer success teams frequently have to fix customer data manually in order to make it work within the product.

Flatfile's Data Hooks® are a useful data healing element for reformatting, validating and correcting data automatically during the data file import process. 

This means that your users won't need to go in manually and correct data themselves. 

The challenges of importing data

Importing data might be a nuisance or even a nightmare, depending on your product and your customers. Most software products are completely unusable without customer data. 

However, importing this data in a timely manner can be a struggle for both your customers and your company. 

Challenges for customers

If you don't think data onboarding is a problem, think again. Importing CSV or Excel files isn't anyone's idea of a good time. 

In our State of Data Onboarding survey, we discovered that 23% of companies reported that it takes weeks or months to import their data and that 96% have run into problems when importing data.

For customers, importing data can be time-consuming, unintuitive and confusing. They often need to put serious effort into prepping their data before it's ready to upload. And data importers don't usually give useful troubleshooting tips when something goes wrong. 

Worse still, fixing data requires users to leave the application, hunt for the issue in their file, and then return to the platform to re-upload it, which is a less-than-ideal experience. 

Challenges for product, customer success, and engineering teams

Today, the majority of companies importing data use an internally built tool, manual data import processes, or a combination of both. 

That means that the in-house team is responsible for data importing. They must build a quality data importer that offers a great user experience, and they also have to step in with manual assistance when that data importer inevitably fails or when an individual customer struggles to make good use of it. 

Some companies even have full-time employees who are devoted entirely to preparing and onboarding customer data. This manual work isn't a necessity: It represents a lack of efficient, scalable data onboarding systems.

Data Hooks is a specific feature within Flatfile's data onboarding solution that allows your team to address the data import problems that slow everyone down.

What are Data Hooks?

Data Hooks allow for more fringe use cases of data healing and validation using Flatfile's data onboarding platform. When used properly, they can be used for things like automatically reformatting area codes or country codes, removing special characters, validating emails against external data and really anything a developer can write in JavaScript.

At Flatfile, we're continually building more data healing and validation functionality, but Data Hooks covers anything that's not yet included in our core product. 

There are two types of Data Hooks:

• Field hooks are for columns. Field hooks run validation on a particular column of data during the matching and importing process

• Record hooks are for rows. Record hooks run validation on each row of data and can be used for single-field, multi-field, or cross-field validation

Use cases for Data Hooks

In our Data Hooks documentation, you'll find a growing library of code snippets you can use to handle a variety of data validation or transformation needs, including:

Reformatting specific value formats: As an example of single-field validation, a CSV file might state "zip," whereas your data schema calls this the "zip code." The simple Data Hook snippet would allow this to be automatically mapped, and the customer could review and accept the mapping. 

Splitting specific value formats: Let's say a customer has a source file with many full names on it, but your data schema only allows the first name and last name fields. A Data Hook for field splitting ensures that data imported with Flatfile's data onboarding platform is input correctly and with the minimal amount of involvement from your customer.

Validating data with your server: You wouldn't want customers uploading duplicate contact records to your CRM, for example. You can use a Data Hook to validate data with your server so that during the import process, a callback function denotes which email addresses or phone numbers already exist in the database.

Cross-field validation: You may want to require a column of data only if other data isn't included. For example, if the phone number and mailing address aren't present, then an email address is required for that row of data to be uploaded.

How do Data Hooks address important issues with data onboarding?

The other good news is that we can see which Data Hooks are being used most frequently, so we know what to add to Flatfile's data onboarding platform next.

But it's not about us. It's about your customers and your team.

For your customers

Customers using Flatfile's data onboarding platform and the Data Hooks snippets you've implemented from our library will enjoy a much easier importing flow. They won't have to leave your application to make the necessary changes to their files that might be required during troubleshooting. 

That means no context switching. The entire data import process becomes smoother and faster.

For your product, customer success, and engineering teams

Companies using Data Hooks to solve common data importing problems save time for their customers and themselves. 

Customer success teams can focus less on helping customers import data and more on building relationships to drive real value. Similarly, software engineers won't get roped into cleaning up a customer's data file; instead, they can focus on their current sprint and build out their core product.

Data Hooks allows your team to scalably address data import challenges - instantly increasing efficiency!

Editor's note: This post was originally published in 2021 and has been updated for comprehensiveness.