Overview

If you receive an external data and need to include it and merge it with your existing data, you will find yourself facing two major problems:

  • the external dataset may not have the same headings as the existing one.
  • imagine if your new data has information on 200,000 businesses that you need to merge, doing it manually would not only be time-consuming but also nearly impossible.

It essentially automates the process of merging the two data sets. Our ETL technology enables the mapping of the most similar headings to merge different structured data. Once these similar headings identified, it is up to the user to approve or not the mapping suggestion before merging them.

Features

  • Automated one to one mapping using our data model generator.
  • Mapping of data attributes based on similarity scoring.
  • Retrieval of mapping proposals based on data content and values.
  • Data transformation features include: split, concatenation, duplicate, replace, remove and « keep data only if » scenarios to ensure that the loaded data is properly mapped.

Benefits

  • Interaction with the customer: whilst our algorithm is the one to suggest a mapping, the user is free to decide whether it fits its expectations or not.
  • Quality control: when merging the two data, the algorithm ensures that the headings are the most similar ones possible.
  • Our component called 2OS integrates an AI algorithm: no need for a configuration or a workflow designed in the 2OS platform. This solution can be directly used through the ETL function of the Studio, or it is possible to integrate the solution in your proper process of Data management.