Flatfile helps discoverygarden accelerate digital preservation with a simplified data onboarding process

The challenge

It has become critically important to digitize and preserve valuable assets from government agencies, libraries, museums, historical societies, and other organizations. Discoverygarden inc.  is a company dedicated to preserving cultural and mission critical digital content. Its customers include such prestigious organizations as UCONN , Baseball Hall of Fame, and UNESCO. Located in Prince Edward Island, Canada, the company has preserved over a hundred million files so far. 

In this post, we uncover:

1. Dealing with data import challenges with multiple data types

2. Evaluating the make/buy data import decision

3. Moving forward with Flatfile's data import solution

“Our focus is on building solutions for long term preservation and stewardship of data whether it was born digital or whether it is photos and physical assets that have been digitized,” said TJ Lewis, COO at discoverygarden inc. “We’re enabling our clients to preserve, store, and easily access these assets over the long term.”

One of the key differentiators for discoverygarden inc. is the open source nature of their platform which is built on Islandora, a free, open-source digital asset management framework. 

“There are a number of proprietary systems out there,” said Lewis, “but with open source software we have the advantage of being able to easily extend our platform to include custom functionality for our clients.”

Dealing with data import challenges with multiple data types

discoverygarden inc. performs about 125 data migration projects a year, typically working with structured metadata. However there are thousands of different metadata schemas and emerging description frameworks in use today which poses a challenge when it comes to importing customer data. 

“Since we deal with many types of customers and many sources of data, it’s very common to encounter poorly structured or sanitized data,” explained Lewis. “We’re always having to perform deep analysis to find problems prior to data importing and having to map client data into structured schemas. That’s typically a painful process.”

Lewis estimated that it often takes weeks to train a customer on how to map and import their data. To streamline the process, discoverygarden inc. has implemented the Flatfile data importer into its new Software as a Service offering, dgspaces.

“Flatfile provides us with an extremely user-friendly way to transfer the mapping and validation process to the client before it gets to us,” said Lewis. “That is a huge time saver.”

Evaluating the make/buy data import decision

The development team at discoverygarden inc. have built multiple iterations of a data import solution using Drupal but decided that the Flatfile data importer was a better choice for our product offerings.

“We built a CSV importer that had mapping functionality years ago, but it wasn’t widely adopted,” said Lewis. “We also built a spreadsheet ingest module but that still required developer intervention before the client could actually upload their data.  For dgspaces, we decided it was going to take much longer to build a data importer ourselves. We decided it was much easier to implement the solution that Flatfile offered.”

The Flatfile data importer provides a much more user-friendly solution that results in a better customer experience, according to Lewis.

“It’s been difficult and time consuming to troubleshoot problems that arise with client data uploads,” said Lewis. “With Flatfile’s data importer we can quickly review the file that the customer uploaded and diagnose the problem. It streamlines the process for our support team to go in and help the client. That’s a really nice addition for us.”

Flatfile also greatly reduced the time the discoverygarden inc. team spent on the data importing process.

“There are several people involved when it’s time to import data,” said Lewis. “Our data architect does an initial analysis of the client files and recommends any cleanup that must be done and develops mapping to our data model. It’s a great deal of manual work on our end. Flatfile’s data importer eliminated this arduous process since now the clients can do the mapping, validation and cleanup on their own.”

Moving forward with Flatfile's data import solution

As part of their service projects, discoverygarden inc. is often tackling custom development such as connecting a product API to pull data out or building a custom feature or workflow for a client. 

Flatfile will be working with discoverygarden inc. to streamline these projects by ingesting database files and acting as a staging database repository that stitches files together. The discoverygarden inc. team will also be able to hook into an API to do the same thing.

“The best thing about the Flatfile data importer is the ease of use for our clients to be able to map and validate a file themselves,” Lewis concluded. “That process prevents data errors from being imported into the system.”

“Additionally Flatfile has an impressive client and customer success team behind the scenes to ensure that everything is performing the way it should!”

About discoverygarden inc.

discoverygarden inc. was founded on the principle of global standards and dedicated to evolving industry leading preservation platforms. Incorporated in 2010 to support the Islandora Open Source Digital Asset Management project that had been founded three years earlier at The University of Prince Edward Island. Ongoing collaboration led to a rewrite of Islandora to allow for greater growth and integration with other software systems. Over the years we have evolved and taken the digital asset management solution beyond its initial niche market in academia and branched into a number of worldwide industries. For more information, visit https://www.discoverygarden.ca

About Flatfile

Flatfile, the pioneer of AI-assisted data exchange, provides companies and their software development teams with the easiest, fastest and safest way to build the ideal data file import experience for their users. Hundreds of companies such as ADP, AstraZeneca, Square and Sage rely on Flatfile’s extensible, API-first platform to create enterprise-scale, intelligent, file-based data import solutions, increase developer productivity, reduce cost and improve data quality.