If you are an organization currently running Microsoft Dynamics CRM/365 on-premises or in a private cloud, and you have less than 200 active users, you’re going to find little help from Microsoft to help you get online. For larger organizations with 200+ active users, there is a program in place with Microsoft called FastTrack. If you qualify for this program, you will potentially have access to Microsoft resources for doing a lift-and-shift migration to the cloud. This is the equivalent of redeploying to on-premises in order to back up your database, run it through some upgrade steps (assuming that you are running 2011, 2013 or 2015), then upgrade your Dynamics instance, and deploy it to the cloud. Unfortunately, this service isn’t available to Partners to migrate smaller organizations; therefore, a large majority of organizations using Dynamics don’t have a clear-cut path to getting online.
Let’s explore what is required to get online in the absence of a lift-and-shift tool, who should potentially do the work, and how you can get the best bang for your buck.
If you are a Microsoft partner reading this and you have multiple clients that are looking to go online, chances are it would be best for your team to “do” the migration. You’ll learn from your mistakes, get better at migrating over time, and potentially reduce costs for your client. However, there are partners out there that specialize in migrations, and it could be in your best interest to partner with one of these companies to reduce the overhead of coming up to speed on the migration process and tools.
If you are a Dynamics customer reading this, then the first step would be to contact your partner, or if you don’t have a partner, reach out to one that has done migrations in the past to assist. It probably doesn’t make sense to try to migrate on your own since it should be a one-time effort that could cause you and your organization a lot of pain during the process.
Depending on several factors, including your current on-premises version of Dynamics 365 / CRM, the size of your database, and the number of customizations you’ve made to Dynamics, the level of effort for migrating to online will vary. Let’s take these scenarios one by one.
If you are running Dynamics 2011 or above, the good news is that a lot of your customizations can be upgraded and potentially deployed with little or no effort. If you are running Dynamics CRM 2011, you’ll have to at least upgrade your customizations to Dynamics 2013 (v 6.0) before you can deploy your solutions online. As you can see from the charts below, solutions exported from Dynamics 2013 will import into all higher versions including 9.0. That’s the good news.
The bad news, however, is that all of your current customizations may not necessarily be supported in the online environment. Specifically, these include custom code, custom reports, and any integrations that require access to the database.
If you have custom plugins or workflow actions that have been coded in your system that don’t conform to certain security standards, this code won’t be allowed to run in an online environment. A quick way to check this is to open Dynamics and go to Customizations -> Customize the System -> Plug-in Assemblies. If you look at the Isolation Mode column of your plugin, it should read ‘Sandbox’. If it reads ‘None,’ you may have to rewrite some of your plugin logic to be compatible online.
Custom Reports and Integrations
Direct SQL access is not available in Dynamics 365 online if you have SQL reports in your on-premises instance that use the Filtered Views in SQL for generating reports. These will need to be rewritten as FetchXML reports, which can be a significant undertaking depending on the complexity of the reports. The same is true for any integration that relies on access to the Dynamics database. The only supported method for accessing data in Dynamics online is through the supported APIs.
If you’re lucky, migrating your customizations won’t be too bad. As long as best practices were followed in your on-premises instance, you should be able to migrate customizations by packaging them up in a solution and deploying them directly to your online organization. Migrating data, on the other hand, may not be so straightforward. The majority of the time spent on a migration project from on-premises to online is spent migrating data. There are several factors at play here, including the size / amount of data being migrated, the complexity of the data and relationships, and the integrity / cleanliness of the data in the on-premises instance. If you want to read more about why migrating data to online is so difficult, I wrote another post on the topic.
How to get started
The first step in any data migration is to decide what kind of migration you are performing. Specifically, are you migrating one to one from your source organization to Dynamics online, or are you going to need to transform data to move it to different places in the target Dynamics online instance? If you choose the former, you’ll likely save yourself a lot of time and headache. While many tools provide transformation capabilities, it can require a lot of setup and trial and error to get it right. However, if you do need to transform or move data from one place to another as part of your migration, I recommend a hybrid approach to trying to transform data on the fly by making the changes in the source org, moving / transforming it where / how you want it, and then migrating one to one.
Get to Know your Data
There are a few things you’ll want to know about your data before you start your migration and before you choose a tool for migrating the data.
- How much data do you have? Before choosing a tool or talking to a partner about migrating, find out the size of your database and the number of records in each table in your database. This will help you determine which tool is best, as well as the overall time it will take to migrate and also help you in the next step.
- How much data do you need to migrate? In a lot of cases, there may be legacy data from as far back as a decade or more that was pulled in when you first implemented Dynamics that is no longer necessary. Not only does migrating a lot of legacy data make the migration take longer, there are storage limits in Dynamics online that will require you to pay for extra storage. In some cases, this data can also affect the performance of you Dynamics instance. It’s important to think about leaving some data behind if at all possible.
- What data is critical to operation? Even if you decide that you need every last scrap of information from your on-premises instance to be online, you may want to think about a phased approach to migrating. If you have a lot of data that is accessed infrequently, you may want to move that data last, or even after going live with your new online org to hasten the time to go live in online.
Choose Your Weapon
Once you’ve decided the type of migration you are doing and have an idea of the scope of data you need to migrate, it’s time to pick a tool. There are several out there, including Cobalt’s Migration Dynamics, KingswaySoft and Scribe. Each has its own strengths and weaknesses, and each requires some amount of setup and monitoring prior to and during the migration.
One of the things we focused on for Migration Dynamics was making the setup / configuration process as simple as possible. Other tools require services to configure and map data from the source organization to the online organization and make several passes at the data in order to ensure all relationships are maintained. Migration Dynamics can be installed, configured, and the migration can be up and running in several hours at the most.
If you want to see us use Migration Dynamics to migrate from on-premises 2011 to Dynamics 365 online in about 20 minutes, you can watch our webinar. The reason it can do this is that it automatically handles a lot of the complicated rules and relationships in Dynamics for you and it is intended to a straight one to one migration from on-premises to online. For this reason, Migration Dynamics is not the best fit if you have to perform complex data transformations on the fly when pushing your data online. In those cases, you’ll want to use a traditional integration tool like KingswaySoft or Scribe that allow for more complex data manipulation as part of your migration.
The final step in your journey is to perform the migration. This will take a while, and it will require you to monitor the progress and make adjustments as you go. Regardless of the tool you select, the preparation you make, or any guarantees that are made, there will be hiccups. Here are a few that you could expect
- Dirty Data: I mentioned earlier about getting to know how clean your data is. If there is data in your on-premises organization that violates validation rules in Dynamics online, that data will get kicked out by any migration tool you choose. An example of this is a field that has text outside the bounds of the allowed length as well as numbers outside the bounds of the minimum or maximum allowed.
- Online Performance: One thing we’ve run up against frequently is SQL errors being thrown by Dynamics online sandbox instances. This doesn’t appear to be a problem in production instances, but if you are testing your migration in a sandbox, you may run into this. This is likely due to either throttling in the sandbox instances or potentially fewer resources.
- Throughput: I mentioned this earlier as a limiting factor for any tool. Generally speaking, the overall size of your database is less important to the time it will take to migrate than the number of records in your database. The reason is that bandwidth is plentiful, in most cases, but only so many transactions can be executed through the Dynamics API over a certain period of time, and that throughput becomes the limiting factor. As a rule of thumb, you can calculate how long your migration will take by knowing the number of records you have to migrate and what the average records per second speed for the migration is.
- Hiccups: In long running processes like a migration that could take days to complete, there can be random failures. These can be caused by noisy neighbors (i.e. other Dynamics tenants on your Dynamics server that are making a lot of API requests), server outages, or other unforeseen data integrity issues.
Help is Available
Whether you choose to migrate yourself or you want to employ a partner to assist with your migration, Cobalt can assist. Migration Dynamics includes free support to ensure you have a successful migration if you choose to migrate your data yourself. If you are a partner and would like to partner with Cobalt to assist you with the Migration of your customers, we are available to do so for all phases of the Migration. If you are a customer that doesn’t have a current Dynamics partner, feel free to reach out to us. Cobalt has Migrated over 150 Dynamics organizations to Dynamics 365 in the past two years and we’d be happy to assist you with your project.