This is a continuation of articles as they appear here for a discussion on the Azure Data Platform: 

This article focuses on the troubleshooting of import and migration errors.

 

Both Azure Data Factory and Data Migration tool flag errors that need to be corrected prior to performing a migration to Azure. Warnings and errors can be received when preparing to migrate.  After correcting each error, the validations can be run again to verify resolution of all errors.

 

One of the first preparation steps in the task summary for importing data from on-premises must be to complete a dry-run of the end-to-end import before scheduling the production import. The collection of data sets must be determined, assigned and mapped next. Putting this information together on how the source data will appear in the destination is more than just deciding on the layout. It helps to iron out all the concerns around the current and future use of the data in the cloud and their access controls as well as planning of the resources even before a single data transfer commences. Creating a portable backup of this collection is essential to get consensus from all stakeholders so that there is enough transparency to the data transfers.

 

Generating SAS keys to enable import is equally important. SAS keys are allowed to expire and they give sufficient permissions to complete the data transfers. This can be included with the collection so that it is easy to lookup the SAS key for the data set to import. When the data is only read from the source without any modifications, there is no need to take a backup of the source. If there are transformation at the source prior to the migration, it is better to take a backup.

 

Any previous dry-run artifacts must be eliminated before a new one commences. Starting from a clean slate is preferable with any data migration efforts because it is harder to sift through the artifacts from the previous run to say whether it is relevant or not.

 

Renaming imported data and containers to prevent conflicts at the destination is another important preparation. Reserving namespaces and allocating containers are essential for a smooth migration.

 

Billing and pay-per-use must represent the proper costs. This can be done by setting up billing for the organization.

 

Reconnect to destination for all consumers must be error free and this can be tried out prior to onboarding them. A guide for the new namespace, organization and access control could additionally be provided.