Next Pathway can help accelerate your cloud migration initiatives, regardless of where you are in your program
Our workload migration technology accelerates the migration of the most complex legacy applications and databases to the cloud, while keeping costs down and getting to market faster
Our solution ensures all consuming users, reports and applications are addressed to minimize disruption to the business, while refactoring, repointing or rebuilding applications to take advantage of compute in the cloud
With large and complex ETL pipelines built-up over time, it’s hard to identify which pipelines are relevant as part of a migration project. Our solution accelerates both the identification of high-priority pipelines, and a path to modernizing them to work within the cloud
Our Data Platform Modernization solution addresses the early stages for a company considering cloud platforms, and reconciling it against their legacy, on-prem environments. We help customers build hybrid or cloud-only platforms that are cost effective and high-performant to provide value-adding benefits to their business
Overall, our solution accelerated the company’s end to end migration by 40%
“Our early estimates for code conversion, through a manual/offshore approach, were coming to 1.5 years. This was a non-starter for us. Leveraging Next Pathway’s SHIFT™Translator, we not only accelerated the entire migration timeline, but it allowed our teams to focus on other value-adding tasks to get us cloud-ready.”
A US retail giant decided that a move to the cloud – and off their legacy data warehouse – was the best way to manage their data, perform complex analytics, and lower TCO going forward. However, the migration posed a number of challenges, including:
- EDW Code Translation:
- Over 2.5 MM lines of K-Shell scripts, SQL, and embedded SQL within ETL had to be converted to the target cloud environment.
- Complex Teradata functions – like BTEQ – that did not have native support within Snowflake had to be addressed.
- Migrate Downstream Consuming Applications:
- 450 reports on various BI tools had to be triaged to determine which could be consolidated, and decommissioned.
We employed the SHIFT™ Migration Suite of Apps to help automate each part of the migration effort.
- Via SHIFT™ Translator and Tester, all 2.5 MM lines of code was translated to the cloud native dialect with 100% accuracy within 3 weeks
- Deployed SHIFT™ Interpreter to handle all BTEQ functions at run-time
- Translated all embedded SQL within the 450 reports within 1 week via SHIFT™ Translator.
Downstream Consuming Application Migration
A large financial services company was completing a large data warehouse migration to the cloud, but wasn’t sure how best to migrate their downstream consuming applications to the cloud platform. Over 1500 applications were consuming data from the legacy warehouse, with thousands of reports consumed across hundreds of users. The company wanted to avoid a one-to-one migration to control growing compute costs and inefficient consumption patterns.
We employed our SHIFT™ Crawler to:
- Develop a complete inventory of all code/objects within the legacy data warehouse
- A set of network maps to define the lineage and dependencies within all downstream consuming applications
- An integrated network map to trace the consuming applications back to their tables in the legacy warehouse
This allowed us to consolidate the number of downstream consuming applications based on similar patterns identified, identify which downstream applications could simply be repointed or those that required refactoring, and define the appropriate reporting data model in the cloud.
"Using SHIFT™ Crawler, we quickly identified duplicate and redundant reports that could be consolidated, reducing our overall number of downstream consuming applications by roughly 30%."
Using SHIFT™ Crawler, we identified all in-scope DataStage ETL jobs within 1 week by automatically scanning over 62 million lines of code.
A large bank needed to triage over 62 million lines of DataStage ETL code to identify which pipelines needed to be migrated from the legacy data warehouse to the cloud. In addition to the large volumes, the company had no visibility into which pipelines fed which applications, and the types of transformations happening throughout the data flow. Not knowing which pipelines were relevant for the legacy data warehouse was a huge operational risk to other applications reliant on those data feeds.
Using SHIFT™ Crawler, we identified all in-scope DataStage ETL jobs within 1 week – and quickly built a lineage view to determine read/write dependencies throughout the application.
This allowed the migration program to continue without risk to other systems, but also accelerated testing cycles by knowing how to efficiently orchestrate the execution of the pipeline groups to compare data results between the cloud and legacy environments, prior to cut-over.
Data Platform Modernization
A global insurance company had ambitions to move to the cloud, but had a large investment in an on-prem Hadoop enterprise data lake and wasn’t sure how to reconcile. All data was already loaded to the lake, and being consumed by many lines of business. Further, some business units were beginning to launch departmental cloud solutions that circumvented existing enterprise data governance controls.
We implemented a unique reference architecture that allowed for a self-service consumption pattern for all business units to move data from the on-prem lake to the cloud. Further, we implemented an enterprise data model that would allow on-prem data to be modeled in such a way to satisfy most enterprise consumption requirements, and enable consumers to conduct their specific business transformations within the cloud – lowering compute costs on the enterprise data lake.