Quick and Easy Migration to Azure Data Factory from a Legacy ETL

Migrating from a legacy ETL tool to Azure Data Factory is extremely complex and requires huge amounts of tedious manual work in order to properly prepare for it. Read how Octopai’s automated data lineage and discovery are game-changers in this painstaking process.

The BI Challenge

The BI team must understand which processes to migrate to Azure Data Factory as not all are still relevant. Focusing on the processes that are actually in use can significantly reduce the migration load, and later, the maintenance and cost given that ADF charges by usage. The BI team must answer the following questions to ensure a pain-free, cost-effective migration:

 

  • Which processes are no longer in use?
  • Are there processes that were needed for a one-time or short-term purpose and are no longer relevant?
  • Has the data underlying the process changed in some way, and can no longer be relied upon?
  • Are there processes based on legacy data that is no longer being updated, and therefore not useful anymore
  • Which processes completely duplicate one or more other processes?
  • Which dependencies need to be created between the new processes? Accurate data many times relies on the upstream processes running in the correct sequence. The timing of each step needs to be analyzed and set accordingly.
  • And finally, once the migration is complete, how can we make sure that all of the necessary processes successfully migrated?

How BI Worked Before Octopai

BI groups are accustomed to taking a manual approach when it comes to migrating BI systems and answering the above questions. This would normally mean going through the portfolio, process by process, in order to:

 

  • Find and eliminate processes that simply aren’t needed anymore
  • Identify redundancies, obsolete, or unreliable data sources and targets
  • Target ETL processes that fail to capture relevant data or combine it ineffectively with other data

It can require endless amounts of detective work, combing through logs and job schedules in disparate systems and examining stored procedure code, report definitions, and more.. The process usually takes many months. And because the company’s data landscape is naturally always changing, the BI team ends up chasing a moving target.

How Octopai Empowers BI Groups

Octopai’s automated data lineage and discovery platform takes the painful manual work and inevitable errors out of the preparation process for migrating to Azure Data Factory. How?

 

  • The visual data lineage tools can show which ETL processes are relevant. Specifically, which processes rely on data sources that are obsolete, questionable, or nonexistent as well as targets that are simply no longer in use; and more.
  • ETL processes can be identified and analyzed to ensure they are generating the expected output in the right sequence.
  • The BI team can easily see the relationships between different data sources, greatly simplifying the migration process.

Octopai is the
first BI Intelligence platform in the industry with the unique capability of visualizing the Azure Data Factory pipelines’ full column-level, source-to-target traceability. This is shown through different data transformations at the most detailed level. As with all other tools that Octopai supports, enterprises can now view complete end-to-end data lineage, from original source systems through Azure Data Factory all the way to reporting.

Value to the Organization

  • Significantly reduced time, effort, and error in preparing for migrating to Azure Data Factory
  • Assurance that only the ETL processes needed are migrated leading to significant cost savings to the enterprise since, as mentioned above, ADF charges per usage.
  • Faster execution of the migration project, enabling businesses to take advantage of Azure Data Factory much sooner.

And what happens after the migration to Azure Data Factory is complete? Your BI group will be able to:

 

  • Easily access column-level data lineage within ADF to ensure that everything migrated successfully.
  • Enable employees that are not yet familiar with ADF and its processes to visualize the lineage without in-depth ADF training.
  • Visualize complete end-to-end data lineage from ADF through to reporting, automatically, in seconds for ongoing day-to-day BI challenges such as impact analysis, root cause analysis, and more.

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.

Become a Partner

Well isn’t this exciting?! Thank you for thinking of Octopai! Please complete all form fields accurately so we can properly assess your request.

Partner Info
End Customer Info
End Customer Info (if known)