Summary : As a B-end product manager working for many years, you will encounter more or less new system replacement or system reconstruction and upgrade scenarios, data migration is a very important part of the system switching process, and the success of data migration directly affects the stable operation of the subsequent system. Different switchover scenarios will face different data migration problems, and a comprehensive data migration plan can guide the smooth implementation of data migration work.

Summary of data migration in the process of switching between old and new systems for B-end products

As a B-end product manager working for many years, you will encounter more or less new system replacement or system reconstruction and upgrade scenarios, data migration is a very important part of the system switching process, and the success of data migration directly affects the stable operation of the subsequent system. Different switchover scenarios will face different data migration problems, and a comprehensive data migration plan can guide the smooth implementation of data migration work.

Source:https://www.woshipm.com rel="nofollow"

01 Data migration scenario

Enterprise businessman found that the original system cannot meet their business needs, need to develop a new system to replace the original system, and the historical data in the original system or ongoing business data need to be reflected in the new system, the new system needs to complete the data migration from the old system to the new system before the online.

02 System switching mode

The system switching mode will affect the selection of data migration mode. When switching between old and new systems, there are several common operation modes:

1. After confirming the online time of the new system, the new and old systems will run in parallel. The new system will be used for new orders, and the old system will be used for in-transit orders. After all in-transit orders of the old system are completed, the historical data will be migrated to the new system at one time.

2. After confirming the online time of the new system, the old system will be deactivated and all data and functions will be switched to the new system.

3. In the case that the new system has a large adjustment and the business process has also undergone a large-scale adjustment, the online time of the selected year or month will also be selected for switching. After switching, the business data will not be migrated, and the historical data will be retained in the old system for viewing.

03 Data migration content

1. Data:

1. Basic data: the main data supporting each module and each business, such as customer, asset, project, product, vehicle, tool information, etc.

2. Dictionary data: global dictionary data of the system.

3. User data: user account, organization, role, authority, etc.

4. Business data: data generated in the process of business, such as order data, transaction data, bill data, operation data, and flow data of each link. Business data also includes:

5. Historical data: data that has been settled in business or generated at other completion nodes.

6. In-transit data: when the system is upgraded or switched, the business is also running normally. There will be a lot of in-transit data in the system, such as unfinished order data and in-transit waybill data; The processing of in-transit data needs special attention.

2. Association relationship between data: Relational data is associated with data through ID or primary key, and the association relationship between data needs to be re-corresponded during migration.

3. Files: Some attachments, pictures and other data are directly associated with business data, and need to be restored and connected. Some separately stored files can be directly transferred to the new server, and corresponding URLs need to be stored.

04 Migration method

Many factors affect the selection of migration methods: the size of migration data, the length of transmission, and whether the business can be suspended.

1. Offline migration:

1. Excel export-import: import data into the new system through the method of export-import Excel. The exported data can be imported after adjustment according to the data format of the new system;

2. Database synchronization: through SQL or database synchronization tool, the fields of the new system and the old system are corresponded and imported into the new system in batches;

2. Online migration:

1. Interface transmission, which can realize data synchronization between new and old systems by defining and writing interfaces. In the case that the switching between new and old systems requires two systems in parallel, interface transmission can realize two-way synchronization between new and old systems. In the process of enterprise information construction, there will be a scenario of outsourcing to self-research. When the project is uncontrollable, try to achieve one-time system switching and upgrading to avoid system parallelism.

2. Database synchronization: In the face of a project with a large amount of migrated data and a long time-consuming time, you can use the database migration tool, etc., to migrate some data to the new system in full + incremental way, full migration of data to the new system, and then perform the automatic incremental migration or manual incremental migration every day. Before the system switch, you should complete the last incremental backup.

05 Data verification

Data verification is required during and after migration to ensure the accuracy, integrity and consistency of data, such as the number of data, data content, and no incremental data generated during migration. Data migration generally requires multiple rehearsals and multiple verifications to ensure the success rate of data migration when the formal system is officially switched.

Detailed verification cases need to be prepared for verification, similar to test cases:

1. The change frequency of basic data and data dictionary is relatively low, so data synchronization can be carried out in advance, which belongs to the environment preparation before going online, and can be synchronized to the new system in advance and complete the test;

2. After synchronizing the completed business history data, verify whether the data is correct and complete through the query, export, BI and other related functions;

3. For unfinished business in-transit data, additional verification of the business operation process is required. According to the process stage of the data, the next step of the business forward and reverse business operation is carried out to ensure the smooth completion of in-transit data.

06 Conclusion

In recent years, the traditional industry has carried out information construction, completed multiple projects from outsourcing to self-research, and experienced multiple data migrations, including relational databases, non-relational databases, production systems, and office systems. I try to summarize some experience of data migration, although there are still many shortcomings, I can grow a little every day to encourage myself.

Welcome to discuss together.