Salaryman IT Ltd - Infrastructure and Database Migrations















Overview of DBMM - Database Migration Methodology

DBMM is a methodology I developed to encapsulate the best practices and solutiuons I have worked with in the major projects I have worked on to date.

This document offers an overview of the DBMM. The full documentation set describes much more detail and includes supporting script and document templates. Please contact me if you are interested in learning more.

I developed DBMM for use in large scale database migration projects such as Service Migrations, Platform Migrations, Business Restructuring or any other scenario where there is a requirement to re-host large numbers of databases or their contents. DBMM encapsulates the knowledge, experience and techniques acquired in major Database Migration projects for a number of global corporations.

The methodology identifies the phases and tasks that need to be followed to prepare for the migrations and to execute them. It identifies the deliverables that should be produced throughout and provides document and script templates where appropriate.

At the heart of the methodology is the idea of a repeatable Factory Process, which will be followed for every single database migration in the project. Using this approach ensures delivery of databases into the target environments to consistent standards and predictable timescales, it avoids re-work, it minimises the scope for issues during the migrations and the use of standard script and document templates ensures high productivity whilst providing great flexibility to react to changes in scope or project schedules.

DBMM follows a three-phase approach. Phase 1 aims at designing the Factory Process and its supporting document and script templates, Phase 2 is about execution of that process for the dozens, hundreds or thousands of databases that need to be migrated and Phase 3 covers project closure activities.

Phase 1 - Preparation

In Phase 1 we design the Factory Process. To do that we need to understand what the source systems look like, how the target environment will be designed and configured, what standards, governance and administrative procedures pertain and what procedures from the Migration Project have already been defined, so we undertake a number of Discovery tasks to gather this information. Where appropriate it may then be necessary to develop the information, standards and procedures further, so this phase includes some Definition tasks to address that. For example, in some projects the target database standards will be very well defined, but in others, particularly where we are migrating to database release or an architecture that is new to an organisation, we may find that standards for that are ill-defined, so it may be necessary to work with the engineering and/or architecture teams to get the standards completed to a sufficient level of detail.

Once this analysis and definition of the environments, standards and processes is completed we will be able to define the Factory Process, which will be used as the basis for the subsequent Database Migrations.

Also in this phase we will define a Compliance Policy and a Database Migration Principles document.

The Compliance Policy documents what standards the migrating database will be expected to comply with and which not. For example, a common question is where existing databases and applications do not meet security standards, is it the responsibility of the Migration Project to apply those standards as databases and schemas are migrated, or do we migrate "as is" and leave non-compliance for another project to address? Addressing this question as a matter of policy at the outset avoids disputes and delay when we come to execute the migrations.

Database Migration Principles document will spell out the rules governing the migrations and the target system builds. It will address issues such as how initialization parameters should be defined, capacity planning and any other details not already defined by standards documents. For example, which init.ora values need to be carried over from legacy databases, how big do we make file systems/volumes in the new environment compared to what was assigned in the legacy databases, in a VMWare environment how many database instances to assign to each Virtual Guest, etc.

Major outputs from this phase will include the fully defined Factory Process, the Compliance Policy and the Database Migration Principles document.

Phase 2 - Execution

In Phase 2 we will execute the database migrations according to the schedule defined by the parent Migration Project plans. In a big project that typically means we expect to migrate the databases and their schemas over a period of time, often at weekends to avoid business impact. A "Migration" in this context will mean the move of one or more databases or database schemas for a particular application, or group of applications.

For each Migration we will follow a standard cycle of steps defined in the Factory Process developed in Phase 1. It will use pre-defined document and script templates and produce standard document outputs as evidence and for audit purposes. The diagram above shows a typical cycle of Plan-> Analyze->Develop->Test->Execute. These tasks may vary somewhat from one project to the next, depending on the particular business environment, but they are typical of what will be undertaken during the Execution Phase for each Migration...

Plan

The high level plan will come from the parent Migration Project and will feed DB migration requirements into the database migration schedule. Notification of an impending migration would normally come a number of weeks before the migration is planned to allow the necessary lead times for ordering and building the target database environments, etc. When a notification is received for a migration it will trigger the Factory Process to begin.

Analyze

The high level plans will typically identify lists of applications that need to be moved, rather than databases, so analysis will need to confirm what databases, schemas and user accounts need to be migrated for each application instance in the schedule. Once the database components are identified we will then undertake detailed investigation of the existing environment so we know what is moving, how big it is, what non-standard DB requirements it has, etc. We will also analyze the applications to gather the information we need to enable us to plan the migration, such as application availability requirements, peak processing periods, current performance metrics, etc. Once completed this gives us the information needed to provide a specification for the target database, if that is yet to be built, to select the appropriate migration method and to put some outline schedules in place.

Develop

Based on the analysis we will decide on an appropriate migration approach. For Oracle databases in most cases we will use Logical Backups (export/import or datapump), since they offer the most flexibility, but applications with demanding availability requirements may need a different approach such as Dataguard or storage replication technologies. Once a migration approach is decided the supporting scripts and documentation will be prepared and any changes required to the source systems will be requested.

At this point we will also request that the platform team(s) configure the target database servers for the Migration, if that hasn't yet been completed. The target databases and schemas will be created, where they don't already exist, and Oracle binaries installed where necessary.

The outputs from this stage for each Migration will include an outline schedule for the proposed migration window and a Migration Manual, which is a step-by-step playbook of the tasks to be executed. The Migration Manual will also include a Backout Plan - the tasks required to revert or migrate back to the legacy environment if the Migration needs to be reversed for some reason.

Test

We will want to rehearse the migration of production databases wherever possible. In some cases there may be technical or opertaional reasons that prevent a rehearsal, but is highly recommended to complete at least one since it will prove the plan, the process and the supporting scripts, and importantly it will provide metrics to so we can accurately schedule the tasks for the live Migration. We will use the rehearsal results to create a detailed schedule for the Migration and to update the Migration Manual to include precise timings for the major tasks and to completely document work-arounds or fixes for any issues experienced during the rehearsal.

For example, it is not unusual to experience bugs and other features when using Logical Backups, especially when using Original Export/Import, but since that is a mature technology for which fixes are well established and well documented then we should be able to identify fixes and work-arounds and embed them into the Migration Manual and supporting scripts for a particular migration.

Execute

Finally we will execute the live Migration for each database or set of databases. By the time we get to this stage it should be a matter of simply following the steps defined in the Migration Manuals. If we have completed the preceding steps correctly we will have very high confidence regarding the success and duration of the database migration activities. Outputs from this process might include logs and other audit information, updates to the configuration management system, handover to the appropriate database support teams and we may submit decommissioning requests for the legacy servers or databases.

Phase 3 - Closure

Phase 3 is a clean up exercise in which we will complete handover of any documents and procedures to the relevant Support Organizations, remove any database and other user accounts that were created for the migration and also clean up any database server file systems, sharepoints, etc that were used as a work area for the migration project.

Copyright 2013 David Orchard Consulting Limited. All rights reserved. Tel +44 (0) 7753249705, http://www.oraclemigrations.com

©2012