Migration of 3 million data records with zero downtime

Migration of 3 million data

The Client

The client is one of the world’s largest multinational telecommunications conglomerate with headquarters in USA. Its range of services includes networking, mobility, unified communications, cable services and VoIP.

Business Need

The multinational telecommunications conglomerate needed to decommission some of the legacy application like HP ALM, Rally, and JIRA and migrate key business processes and data for customer analytics from its legacy data platform to central IBM Jazz CLM platform. The migration was part of a strategic mandate to consolidate on a single data platform and leverage the platform, which closely aligns to the organization’s business needs, planned operational efficiencies and single version of truth for data analytics. The existing software development tools and data management system did not allow for easy and efficient data access for management reporting.

Overall objective is to have a more accessible and comprehensive view of its business data – a 360-degree viewing ability, which would enable it to create and capitalize on increased organizational productivity. This involved an enterprise-wide initiative and the client needed a technology partner to re-engineer the existing customer data from the existing application to a new central CLM application. In addition, the organization identified a large cost incurred by the company on thousands multiple tools license on the heterogeneous applications used by various teams across the organization.

Kovair Role

Skillfully migrating terabytes of data from thousands of projects requires deep understanding of data migrations and the methodologies and pitfalls of such an undertaking. The organization searched for an entity with the experience and know-how to build and deliver on a viable and cost effective solution that would see the organization through the migration process for multi-thousands of projects. The organization found this expertise at Kovair Software with its hybrid version of the QuickSync product that utilized both the QuickSync and Omnibus capabilities on the Kovair Platform.

Kovair Flexible Architecture

The client reviewed several other vendors but the need was to have a platform which is flexible enough to mold as per their business needs seamlessly. The primary features like supporting one to many computed data routing to multiple attributes across multiple projects in the IBM Jazz were a requirement.

Challenges

The relevant data were stored in multiple applications project repositories that did not allow visibility to management in other units. The client’s existing application system included more than 2000 application projects and the challenge was to migrate and perform ongoing sync over the live data without a single minute of downtime. The teams using the existing tools will be moved in a phased manner to the target tools. Since the projects are having millions of data accumulated over the years, the data quality was questionable. With so many projects and 1000’s of simultaneous users accessing live data along with the migration process requires meticulous planning and precision in execution. Since data is business and client cannot afford to lose business, the data quality was of utmost importance. Apart from typical data quality issues, the existing system was also dependent on several other applications and calling programs. This posed added complexity to the migration exercise. Over and above the data volume, the window for data migration was a great challenge as for example 200,000 records of data needed to be migrated in two to three days. Kovair did migrate at a rate of few hundred migrations per minute across multiple work areas with less than 0.1% errors.

The Solution

The Kovair team developed and evaluated 3 technical options to address the migration of the 2000 application projects. After further evaluation and technical discussion, the most suitable option that was identified is a Hybrid solution which is an extension of the Kovair flagship product “The Omnibus Integration Platform” and the Kovair QuickSync product. The solution was prototyped and was tested few hundred times on several test beds. The solution was architected and detailed demos had been provided to the Client’s technical and management leadership

The Kovair project team devised a comprehensive implementation road map which addressed issues such as cost estimates and milestones for the interface migration and technical analysis of the chosen solution. In the first release, data from one of the major existing customer repositories – HPALM was migrated to the new enterprise component. The team also developed several small applications that operated on top of the enterprise component. The first release was completely managed and developed by Kovair and it can rightfully claim a very successful and flawless data migration execution.

Statistics and Achievements

  • Over 1.6 million data migrated from 1500 projects of Rally
  • Over 1.8 million data migrated from 960 applications of HPALM
  • Migration transaction rates are around 300+ records per minute.
  • More than 200 Windows boxes took part in migration simultaneously
  • Migration followed by data synchronization executed while hundreds of end users are working with their respective tools simultaneously.
  • Entire migration has been executed in over 4 months with zero downtime from end users
  • Several brainstorming sessions held with IBM Jazz, Rally and other technical teams
  • Kovair in-house remote monitoring tool for monitoring hundreds of workstations for migration
  • The platform was built to support SSL, TLS1.1 and TLS 1.2
  • Less than 0.1% reported issues.

Live Data Statistics

Rally to IBM RTC and IBM RQM

Benefits

  • The upfront investment in tools and talent as well as costs of manpower of implementing a data management strategy can seem high. When posed against the costs associated with poor data quality, erroneous information, and uniformed decision making, the benefits are very high. The data shows that companies that have implemented central data repositories have generated more revenue and saved more money than companies who have not made the investment.
  • The client may have achieved savings by negotiating tool costs from one vendor versus multiple vendors besides achieving efficiencies in management visibility and reporting.
  • Unified CLM repository for 360 degree reporting
  • Full collaboration between heterogeneous Teams
  • Easy and fluent collaboration and integration
  • The client’s enterprise-wide IT objective of gaining an adaptable, central 360-degree view of organizational data was fulfilled and the client could move without much trepidation to newer integrated tools.
  • The benefits of the new system were quick to manifest across the enterprise and resulted in far greater responsiveness to business changes. This initiative also enabled the client to explore more revenue generating and revenue planning opportunities.
  • Analyzing trends in order to make informed decision is made simple because a central repository stores large amounts of past data, providing a historical intelligence aspect to decision making

Results

The data migration automation application was streamlined and developed in few weeks, saving the organization from having to rely on the legacy applications with massive license cost. With a Hybrid application to automate the process, the time invested was trimmed to weeks.

Finally, the data migration strategy project took place during a major transformation at the organization. As the organization was busy refining the way it approaches a core aspect of its business, it was relieved not to have to manage the project teams or the analytics data migration project. Ultimately a deep experience with data and analytics enabled the customer to focus its resources in the right areas and avoid disruption during a time of major change.