It has become critically important to digitize and preserve valuable assets from government agencies, libraries, museums, historical societies, and other organizations. Discoverygarden inc. is a company dedicated to preserving cultural and mission critical digital content. Its customers include such prestigious organizations as UCONN , Baseball Hall of Fame, and UNESCO. Located in Prince Edward Island, Canada, the company has preserved over a hundred million files so far.
In this post, we uncover:
Dealing with data import challenges with multiple data types
Evaluating the make/buy data import decision
Moving forward with Flatfile's data import solution
“Our focus is on building solutions for long term preservation and stewardship of data whether it was born digital or whether it is photos and physical assets that have been digitized,” said TJ Lewis, COO at discoverygarden inc. “We’re enabling our clients to preserve, store, and easily access these assets over the long term.”
One of the key differentiators for discoverygarden inc. is the open source nature of their platform which is built on Islandora, a free, open-source digital asset management framework.
“There are a number of proprietary systems out there,” said Lewis, “but with open source software we have the advantage of being able to easily extend our platform to include custom functionality for our clients.”
discoverygarden inc. performs about 125 data migration projects a year, typically working with structured metadata. However there are thousands of different metadata schemas and emerging description frameworks in use today which poses a challenge when it comes to importing customer data.
“Since we deal with many types of customers and many sources of data, it’s very common to encounter poorly structured or sanitized data,” explained Lewis. “We’re always having to perform deep analysis to find problems prior to data importing and having to map client data into structured schemas. That’s typically a painful process.”
Lewis estimated that it often takes weeks to train a customer on how to map and import their data. To streamline the process, discoverygarden inc. has implemented the Flatfile data importer into its new Software as a Service offering, dgspaces.
“Flatfile provides us with an extremely user-friendly way to transfer the mapping and validation process to the client before it gets to us,” said Lewis. “That is a huge time saver.”
The development team at discoverygarden inc. have built multiple iterations of a data import solution using Drupal but decided that the Flatfile data importer was a better choice for our product offerings.
“We built a CSV importer that had mapping functionality years ago, but it wasn’t widely adopted,” said Lewis. “We also built a spreadsheet ingest module but that still required developer intervention before the client could actually upload their data. For dgspaces, we decided it was going to take much longer to build a data importer ourselves. We decided it was much easier to implement the solution that Flatfile offered.”
The Flatfile data importer provides a much more user-friendly solution that results in a better customer experience, according to Lewis.
“It’s been difficult and time consuming to troubleshoot problems that arise with client data uploads,” said Lewis. “With Flatfile’s data importer we can quickly review the file that the customer uploaded and diagnose the problem. It streamlines the process for our support team to go in and help the client. That’s a really nice addition for us.”
Flatfile also greatly reduced the time the discoverygarden inc. team spent on the data importing process.
“There are several people involved when it’s time to import data,” said Lewis. “Our data architect does an initial analysis of the client files and recommends any cleanup that must be done and develops mapping to our data model. It’s a great deal of manual work on our end. Flatfile’s data importer eliminated this arduous process since now the clients can do the mapping, validation and cleanup on their own.”
As part of their service projects, discoverygarden inc. is often tackling custom development such as connecting a product API to pull data out or building a custom feature or workflow for a client.
Flatfile will be working with discoverygarden inc. to streamline these projects by ingesting database files and acting as a staging database repository that stitches files together. The discoverygarden inc. team will also be able to hook into an API to do the same thing.
“The best thing about the Flatfile data importer is the ease of use for our clients to be able to map and validate a file themselves,” Lewis concluded. “That process prevents data errors from being imported into the system.”
“Additionally Flatfile has an impressive client and customer success team behind the scenes to ensure that everything is performing the way it should!”
discoverygarden inc. was founded on the principle of global standards and dedicated to evolving industry leading preservation platforms. Incorporated in 2010 to support the Islandora Open Source Digital Asset Management project that had been founded three years earlier at The University of Prince Edward Island. Ongoing collaboration led to a rewrite of Islandora to allow for greater growth and integration with other software systems. Over the years we have evolved and taken the digital asset management solution beyond its initial niche market in academia and branched into a number of worldwide industries. For more information, visit https://www.discoverygarden.ca
Flatfile's mission is to remove barriers between humans and data. With AI-assisted data onboarding, they eliminate repetitive work and make B2B data transactions fast, intuitive, and error-free. Flatfile automatically learns how imported data should be structured and cleaned, enabling customers and teams to spend more time using their data instead of fixing it. Flatfile has transformed over 300 million rows of data for companies like ClickUp, Blackbaud, Benevity, and Toast. To learn more about Flatfile’s data onboarding platform email firstname.lastname@example.org or get started for free at flatfile.com.