Out with the old, in with the new.
That means you should leave that old legacy system in the dust and migrate to a brand-spanking-new cloud-based data system as soon as possible, right?
Well, maybe.
Thereâs no denying the immense benefits that cloud computing can bring to data management: easier scalability, reduced cost of IT operations, improved performance⊠the list goes on.
But in order to reap those enviable benefits, you have to do something not-so-enviable:
Migrate.
âMigrateâ is a nice way of saying: Move. Displace. Uproot the data foundations of your organization and try to replant them.
Data migrations arenât simple, but if you make smart preparations and conduct the migration wisely, youâll be able to celebrate with a bottle of champagne sooner than you think.
4 critical elements to enterprise data migration success
1) Plan it out
If you migrate without a plan, and it actually goes well, you should thank your lucky stars. But since lucky stars are generally frowned upon as a risk management strategy, we highly recommend you plan out your cloud migration process.
Two types of planning are important:
- The high-level plan: migration goals
- The local-level plan: designing a migration to achieve those goals
The high-level plan: migration goals
Why are you migrating?
Donât just point to the above list of migration benefits and say, âAll of those!â
Take the time to consider your reasons, your goals, and your priorities among those goals. Get input from all the different departments of your organization that will be impacted by the data migration to the cloud.
If you skip this step (or do a half-hearted job), youâre leaving far too much to those lucky stars, just as you would in a physical move. Letâs say, for instance, that you suffer horribly from seasonal allergies and currently reside in Scranton, Pennsylvania (the âallergy capitalâ of the United States, according to the Asthma and Allergy Foundation of America). You want to relocate, and the number one thing on your mind is escaping that constant sneezing and those itchy eyes.
You see that Denver, Colorado ranks in the top 10 least challenging places to live with seasonal allergies. You do some research and are attracted by the scenic views, the recreational activities (no, not just the recreational substances) and the cultural opportunities.
But you somehow overlook the fact (maybe because youâre at the peak of fall allergy season) that you hate shoveling out your driveway and sidewalks from Scrantonâs annual 40 inches of snow.
And guess what? After your move to Denver, you find yourself in a winter season where you need to shovel out 80 inches of snow! (Denverâs average is 60 inches annually, so maybe itâll be better next year.) Maybe you shouldâve considered Seattle.
While such an extreme oversight is unlikely when it comes to a physical move (I mean, come on, who doesnât picture Denver with snow-covered slopes?), itâs more of a risk when it comes to a digital or operational move.
Listing and prioritizing your organizationâs goals (and anti-goals, i.e. what you want to avoid) is the foundation of a successful cloud data migration.
The local-level plan: designing a migration to achieve those goals
Once you have a high-level plan, itâs time to get down into the local picture. How are you going to pick a cloud infrastructure, design an ideal data environment and orchestrate a move to support those goals?
An important stage of this local-level planning is constructing a map of your current data environment and data flow – and then optimizing it. How would you ideally redesign your environment and data flow to achieve your goals? You can first do a no-holds-barred, data dream environment, and then (inevitably) tweak it to be more realistic and in line with your organizationâs resources.
Creating a detailed map of your data landscape is prohibitive (and a huge time-suck) if you do it manually. Thatâs where automated data lineage can speed your cloud migration process along, by creating a comprehensive map of the data flow through your environment in as little as a day. (Okay, it *could* take a few days – if you have a massive environment.)
Once you have a map, you can see exactly where your data flow and processes are tripping you up from achieving your goals, and you can design how an ideal flow and environment would look.
Get feedback and buy-in from your stakeholders (so you can be sure you didnât overlook 60 annual inches of snow). Then use that approved design as your guiding light in structuring your migration.
2) Donât move junk
This may sound obvious. After all, why would you pay to move junk and then pay to store it?
But when youâre looking at an overwhelming project like a cloud migration, it is tempting to simplify the process wherever you can.
So whatever data you can migrate as-is (especially if youâre doing a lift and shift migration, or even a âlift, tinker and shiftâ migration), you may want to just do it and relegate sorting through it until a later point in time, after the migration.
Resist the temptation.
Sorting through your data assets and processes has a direct impact on the ROI and success of your migration.
I know someone who was born and grew up in the United States but never lived there permanently as an adult. She went to college abroad⊠and decided she wanted to stay abroad to live. Her âmoveâ (if you can even call it that) was seamless, and consisted of placing in her suitcase whatever clothing or childhood items she wanted from her parentsâ home during her vacation visits back.
Her parents, on the other hand, joined her years later as an almost-retired couple who had lived their entire adult lives in the United States – and their move was a move. They had, naturally, accumulated the kind of âstuffâ you amass when you live in the same general geographic area for decades.
But now they were moving overseas, requiring a shipping container where you pay per cubic foot of space. They were moving from a three-story private home into a three-bedroom apartment; if they moved all their possessions âas-is,â theyâd have to rent storage space – money out of their pockets PLUS limited accessibility to their possessions.
So it was time to downsize.
It took time. It took work. But making the move to a new country lighter, freer, with only the possessions they really wanted and needed, had positive impacts financially and logistically. Getting settled and feeling at home in your new environment takes less time if you have less stuff.
So even if youâre doing a lift and shift migration – and all the more so if youâre revising, rebuilding or replacing your current data management system – invest the time and effort to go through your current assets and processes and see what should really be making the move.
And it may not be as much time and work as you expect. If you can leverage automated data lineage to trace the path of every data asset end-to-end through your systems, from source, to ETL, in the database, through to analytics and reporting, youâll have an invaluable, visual map in a short time. Youâll be able to see clearly which assets and processes are valuable, which are discardable, and which need to be tweaked or fixed.
With the full picture at your fingertips, you can wisely pack for a successful move.
3) Learn the language and culture
Similarities can be deceiving.
A British citizen who moves to the United States thinks that the language will be the least of his difficulties. That is, until he is half under his bed, searching for a missing sock, and asks his roommate to hand him a torch. Instead of said roommateâs flashlight, he gets a lecture about fire safety. And then, when he asks around for a place to get braces, heâs directed to an orthodontist instead of the nearest clothing store that carries suspenders.
Oops.
Your legacy data systems, no matter how similar they seem to the cloud data systems to which you are migrating, will have some cultural differences. Be prepared.
These differences might be as deceptively simple as a syntax change. For example, Instr () in Oracle db needs to be written as Position () in Snowflake. That oneâs actually not so bad, because Snowflake just wonât support the Instr () query structure, and youâll realize thereâs something wrong.
But what about this query: select concat(‘SSS’,null) from dual? Because of the difference with how they process the CONCAT function, Oracle db will return SSS, while Snowflake will return null.
Ask for a torch, and instead of getting a flashlight, you get handed the burning brand they use to light the main fire at the Olympics. And you may not even realize until you use it to search under your bed for your missing sock.
Your bad.
To avoid wildfires in your data environment, do thorough research on the differences between your legacy systems and your new cloud systems. Train anyone who will have interaction with the new system on the nature of these differences – before your ETL, reporting system or database migration to the cloud.
Data analysts who will be running queries across Snowflakeâs pay-for-compute-time platform, for example, need to understand the cost and impact of different types of queries. They need to be briefed on the kind of queries NOT to run – especially if those were standard queries on your legacy system.
An ounce of prevention is worth a pound (is that lb.? or ÂŁ?) of cure.
4) Minimize day-to-day disruption
The last time I moved apartments, I found myself still (barely) awake at midnight, dragging myself around looking for a box containing some critical items. (My spouse, of course, did not consider the items as critical as I did and saw no need to extricate himself from the bed to help me search.) Needless to say, I was NOT in a good mood.
Cranky employees tend to have a negative impact on your ROI. If you can maintain your employeesâ ability to easily find and access the data they need, even in the midst of your migration, the migration will go much more smoothly. (Or at least it will feel like itâs going more smoothly, which often amounts to the same thing.)
An automated data catalog can be an invaluable aid to keeping data assets and their location at your employeesâ fingertips. The data catalog itself is a tool that organizes all the data assets in a companyâs data landscape, for each one including definitions, descriptions, ratings, responsible individuals, and more, making it simple to search for, identify and access the data you need for any given purpose.
When a data catalog solution is automated, it will automatically extract the data assets from across the different software in your data landscape, including ETLs, databases, and reporting tools. An automated solution can collect, analyze, and organize â and then build or populate the data catalog by itself.
An automated data catalog will not only self-create but will also self-update, routinely reviewing all metadata within your data landscape and updating your data catalog accordingly. At no time is this more important than during a migration. Your automated data catalog can review both your on-prem data environment and your new cloud environment and integrate both into your catalog. Frequent automated updates will make the current location of any data asset just a quick search and click away. (Similarly, if your end goal is a hybrid environment, a comprehensive automated data catalog can help keep everything straightforward and accessible, no matter where itâs located.)
Result: no service disruptions. Your data-dependent employees may not even realize youâre in the midst of a migration.
Shout it (c)loud!
Youâve done it! Youâve made it successfully to the cloud!
Well, probably youâve only made it to the end of this blog post, but envisioning success is supposed to be helpful for athletes, so why not for data experts?
Try this: envision yourself taking that bottle of champagne and celebrating the launch of your new cloud-based data management system!
Did it work? Now go get âem, champ – get that data to the cloud!