Subscribe now to stay up-to-date
Product updates & features
Kelly Johnson
Published 10/24/2024
In data transformation, mapping is where every migration project begins — and often where frustration escalates. Without the right tools, navigating through the complexity of this process can slow your project to a crawl.
The Flatfile team has been vigorously working to solve this problem. After several experiments and epiphanies, we revamped a key platform feature — Advanced Mapping — to transform this critical process. Thanks to this breakthrough, we removed the guesswork and hassle from data migration.
We unveiled this new tool on the inaugural day of our Launch Week event, a four-day conference that marks Flatfile's largest feature release ever.
In the session, Engineering Lead Damon Banks expertly explained how Advanced Mapping will expand your options for mapping data, saving time and cost and streamlining your team’s workflow.
Read our session recap to get key highlights of what you can expect with Advanced Mapping.
Flatfile retooled mapping from the ground up to make it more intuitive and flexible. Advanced Mapping offers enhanced control and visibility over how your data is mapped – meaning faster, smarter workflows with fewer headaches.
One of the more notable features of Advanced Mapping is Groups. Previously, basic mapping only offered a left-to-right assignment without a simple mechanism to track what was mapped or what remains – Groups solved this problem.
When you use Advanced Mapping, you could see as many as five new groupings — Recommendations, Exact Matches, Previously Mapped, Unmapped, and Empty Fields — that will make organizing, tracking and refining your mapping tasks easier.
Here’s a quick breakdown of what comprises Groups:
Recommendations is our advanced AI-mapping engine trained on billions of decisions that provides incredibly accurate mapping suggestions.
Exact Matches auto-suggests simple matches such as “first name” and “last name.”
Previously Mapped is an AI-driven feature that learns your preferences. Anything you previously matched is retained, so you can use it the next time you map schemas.
Unmapped helps your team understand what's done and what’s left to do when mapping large data sets. With this group type, you no longer have to constantly check to see if mapping is complete.
Empty Fields shows you any empty columns without information and groups them together so you can clearly gauge the accuracy of your incoming data.
Here's a preview of the Recommendations grouping
Take a look at the Previously Mapped grouping
Groups will give you a holistic view of your data migration – giving you confidence that nothing is overlooked and quickly identifying any fields that need additional scrutiny.
Another exciting development in Advanced Mapping is the enhanced Data Preview feature. This feature will help your team have a better understanding of how information is mapped between the source data and the destination data.
By removing simple data samples and placing them in tabular form, your team will see the data exactly as it will appear in its final form, simplifying the validation process.
“Think about that for a minute: No more testing mapping to see desired outputs. You can now quickly see if what you have done is what’s needed. This instant feedback is going to be your best friend in data mapping, and we’re very proud of it,” said Banks.
What’s more, you could only view 10 records previously. Now, you can view up to 1,000 records at a time – a massive upgrade for the feature.
This expanded view offers a richer context for how data is being mapped, providing more clarity and confidence in the process.
Are you working with large and complex datasets or what we call Wide File Ergonomics? Advanced Mapping alleviates the endless scroll nightmare your team may experience and other administrative burdens accompanying navigating gigantic files.
We incorporated new features like Scroll to Field, Search and Auto Save into Advanced Mapping to streamline the experience and ensure every field is accounted for and accurately mapped.
Here’s how it works:
Scroll to Field Mapping: When you hover over a field, Data Preview will auto-scroll to it. You’ll now have real-time previews of your source data and destination data, so you don't have to scroll to the field; it auto-scrolls for you.
Search: Your team can now search for any column, and data preview will dynamically show the result.
Auto Save: Lost work is now a thing of the past. With Auto Save, you can be confident that you can save your work and pick up where you left off after you close a browser or shut down your computer.
“We've tested this new Wide File Ergonomics on large schemas of well over 1,000 fields, and the results are simply amazing. Data Mapping is efficient, and the outcome is greater accuracy,” said Banks.
As a result, you won’t waste time searching for specific data points or making adjustments to wide datasets.
Advanced Mapping goes beyond simple field-to-field matching. Where the old mapping uses a manual approach, the new mapping relies on data transformation rules. Today, you can manipulate your data as part of the mapping process.
Do you need to split full names into first and last names or combine multiple data points into a single field? No problem. Now, you can customize how your data is transformed – giving you full flexibility to match your destination schema.
Banks offered this example: “You now have five new options for mapping your data. Plus, we removed the restriction of mapping only one source field to one destination field. Meaning, if you need to map a phone number to both primary phone and primary contact, you can do it!”
Take a look at the five new key rules:
Combine Fields: Allows you to merge multiple source fields into one destination field.
Single Value: Lets you set a constant value for the destination field.
First available value: Provides flexibility by mapping the first non-empty value from a list of source fields to a destination field.
List rule: Gives you the power to aggregate multiple source fields into an array within a single destination field.
Split Rule: Separates a single source field into multiple destination fields based on delimiters or patterns. The Split rule uses AI to understand the content of the source field, and then it uses the context of all available destination fields to suggest the best way to split your data with accuracy.
Here's what you'll see when using the Split Mapping Rule
Banks added that “by leveraging these powerful rules, advanced mapping is going to empower you to handle even the most complex data transformation much easier, no more detours into Excel for complex formulas or manual data. Everything you need is right at your fingertips, making data mapping more efficient and less error prone.”
Your team will be inspired to adjust data directly within Flatfile, eliminating the need for cumbersome pre-processing steps.
Advanced Mapping isn’t just a new feature; it's the foundation for how Flatfile will continually improve data mapping. Additionally, it’s one of the most meaningful tools for removing latency from data transformation, and we’re dogmatic about reducing this friction to accelerate your projects.
This tool and the robust features within it will evolve with every use, providing more options and better outcomes for even the most complex data scenarios.
As we announce more updates to the Flatfile platform, you will see enhancements that will continue to simplify and expedite the mapping process.
Watch the recording of our Advanced Mapping session, which included a demo. Get a sneak preview of the tool in action.
Smarter, better data transformation