Data Migration Challenges
Often an upgrade of an existing software or the acquisition of a new software will involve some type of data migration. Unless the data volume is very small and can be easily accomplished with manual entry, data migration will pose some challenges depending on various factors. The good news is all of these challenges can be overcome with the right approach.
Understand The Source Data
Common sources of data that will be migrated are spreadsheet or an existing database. Time must be provided to properly audit and analyze the source data in order to identify issues. The challenge is that often the new target system will have a specific design and data requirements that are missing in the existing data source. Other issues can include inaccurate data fields, fields that are no longer used or multiple fields representing the same thing but that have evolved or a series of rule changes. Understanding both the new system design and the source data layout is crucial to being able to develop a roadmap for how to handle and remap the existing data into the new design.
Migration Process
Once the source data has been analyzed, audited and the issues identified, the migration process can be mapped out and the appropriate program automation tools applied to the process.
Program tools to assist and automate the data migration process are indispensable. These can be programs specifically written to upload the source data into a new database as well as provide validation and reconciliation of the data as it moves from old to new system. These programs will ensure that records are not lost and account for records transferred as well as those rejected or ignored.
The program tools will also validate and recode data as required if there are duplicates or data that is meaningless because it was entered free-form with no system validation.
These programs once written and adapted the data needs make it easier to rerun samples of data to ensure the results are as expected. Since the data volume will often prohibit a record by record review, data can be cross-checked and queried programmatically for accuracy and for identifying any exceptions.
A Successful Result
A successful migration will ensure that the data transferred is usable and accurate once it arrives in the new software destination. Although a data migration will add additional time and effort to achieve, taking short cuts or skipping steps can severely undermine the entire software implementation at a later point, when it’s too late. Often time pressures exist making this step more difficult and challenging which may result in some compromises. For example, determining how far back the data migration should go or if data is unusable, can a balance forward approach be applied instead. The balance of time, resources and cost will impact the chosen method.#datamigration
