Magento Data Migration Process from Magento 1 to Magento 2

Migrating from Magento 1 (M1) to a new Magento 2.x (M2) store is a multi-step process, involving a lot of debugging and tracking down migration errors.
It is not something that most store owners can handle on their own. They will need an experienced Magento developer in order to have a successful migration.

The typical migration from M1 to the latest version of M2 requires two migrations:
1) The initial dry run – identifying issues and fixing them so data migration can occur. It is also the starting point for any new development that’s part of the migration.
This includes adding new functionality thru installing a new M2 theme or extensions, and adding new blocks based on client business needs.
It can take anywhere from 1-3 months to complete development, before moving on to live migration.

2) Go live migration where the live store is in maintenance mode and we are migrating data before going live with M2.
Combined, these two data migrations will take from 14-20 hours, depending on complexity of the site.

There are five basic steps to migrating data from a M1 store to a M2 store.

Step 1 – Magento live store review
The migration process starts with a review of the M1 store that is going to migrated to a new M2 store.
Depending on the condition of the M1 database, there may be issues with some of the database tables, such as url rewrites.
Depending on the url key of each product in the database, there may be more than one product with the same url key. This causes a conflict in M1 and to resolve the conflict,
Magento creates a new rewrite in the core_url_rewrite table to address these conflicts. If you have a lot of products that have duplicate url keys, this database table can grow to over 3 million rows.
Before using live site DB for migration purposes, we will address these url rewrite issues by creating a local copy of your live site and then fixing all these duplicate url keys via a spreadsheet import.
Once that is done, we truncate the core_url_rewrite table, save all the products in store in bulk, and then reindex the url rewrite index. This can reduce the core_url_rewrite table to less than 5,000 rows.
We also look at store views that are not being used and delete them to make the transfer of data as clean as possible.

Step 2- Install most recent version
We install the most recent version of M2 in root directory and then check DB and files for any issues before beginning the migration process.

Step 3- Install and Configure M2 migration tool.
We install the migration tool that’s made for your M2 version via composer and then once that’s completed we create the configuration and mapping files that will be used for migrating data between your M1 store and a clean M2 store.
You can create these files from within the M1 module folders. For example if your M1 version is, then use the following commands:
cd vendor/magento/data-migration-tool/etc/opensource-to-opensource/
cp config.xml.dist config.xml
cp map.xml.dist map.xml
This should create two files that will be used for your migrtion. Make sure to change the config.xml to reflect the path to the new map file.

Next, create a source DB on your server, as well as making a copy of your M2 store as a backup in case you run into any issues.
Once the source DB has been created, import the sanifized database from your M1 store into the source DB.
Then configure the migration tool to set source and target Database credentials so the migration tool can talk to both your M1 and M2 databases and transfer data.
Do this by adding a source and destination block in config.xml file (see download at bottom of page for reference).
You will want to change name and username to match with your db credentials. You should now be ready to migrate data between the two databases.

Step 4- Migrate settings and data
Run an initial configuration setting check using the migrate settings command from root directory:
bin/magento migrate:settings
This migrates settings to the new M2 store. This process takes only a minute or two to complete.

Once that step is done, do a test migration, which checks whether or not all the DB tables in M1 and M2 match up properly.
This almost always identifies tables in M1 Database that do not exist in M2 Database, usually as a result of extensions that have created tables in the M1 database.
There may also be fields in a number of tables that don’t exist in M2 as well. All of these differences between the two databases must be resolved via a map.xml file which is part of the migration tool.
For example, your M1 database may have a table named aw_blog. To avoid transferring any data from that M1 table, we need to add an ignore directive (see download at bottom of page for reference).
Once all the tables that don’t exist in M2 are ignored, move on to the fields that don’t exist.
For example in the sales_flat_order_item table, there is a field called verdor_name. That was created by a M1 extension and we don’t want to transfer that either.
Use an ignore directive to handle that field (see download at bottom of page for reference).

Once all the mapping of documents and fields is complete, rerun the migrate command and everything in the M1 DB will transfer to the M2 DB.
Once migration has run to complete with no errors, login to your M2 store and see that all the data from M1 has been migrated to M2 store.
If everything is there, move on to the next step.

Step 5 Post Migration
Since the Db of our M2 store now reflects what’s in our M1 store, transfer all the media folders/files to new M2 store.
Use ssh to connect to M1 store and go to the html/media/catalog folder and zip up the category and product folders with the following commands:
tar -czf product.tar.tgz product
tar -czf category.tar.tgz category

Once they are compressed, download them and then upload to the new M2 store in html/pub/media/catalog/ folder.
Once uploaded, expand them with the following commands:
tar -zxvf product.tar.tgz
tar -zxvf category.tar.tgz
When they have been expanded, delete them since they are no longer needed.
You may also need to upload manually all the wysiwyg images into new M2 store in order to have all images showing on front end.

Next step is to reindex M2 store via command line, clearing cache, and deploying static content, using the following commands:
php -f bin/magento indexer:reindex
php bin/magento cache:flush
php bin/magento setup:static-content:deploy -f

Once these processes are done, you should be able to login to your new M2 store and it should have all the data migrated from your old store.
On the front end, you should see all the categories migrated, all the products migrated and all the content migrated.

If this is your go live migration, you will also want to install your new theme, all the extensions that were needed, as well as any theme files that were created or modified during development.
You will also want to modify any content pages or static blocks that were created/ modified as part of your dry run development process.

For reference, I have included all the migration commands that are typically used during a migration (see download).

Migration Commands (17)
Good luck!

This entry was posted in Magento, Magento 2, Uncategorized. Bookmark the permalink.