How To Estimate & Speed-up The Migration From SharePoint On-Premise To SharePoint Online

After several migrations executed from SharePoint On-Premise to SharePoint Online using multiple combinations of versions of both source and destination, I found Sharegate as one of the faster and more efficient options. Let’s have an overview of the functioning of the migration tool.

An obvious question comes up while analyzing a SharePoint migration project – how long a tool would take to transport the actual content from the source to the destination.

It may be hard to predict the time because of factors such as system load, bandwidth, average item size and bottlenecks in source or destination. I’ve often seen people saying 5GB per hour or 10GB per hour but that’s not a practical approach because it depends primarily on two aspects – the infrastructure set-up and complexity of content & site objects.

Infrastructure Set-up

How to identify the number of migration servers

Every migration has content inventory analysis phase, this inventory analysis will help the team to understand the content size, content depth (version, metadata and etc), site objects, any custom functionality involved and etc.

Based on the size, depth of the content and client tentative deadlines, we should be able to decide the servers required to run the migration. A rough estimate can be made based on a few test migrations.

To perform the migration on each server, it should be equipped with the below software and suitably configured.

• Sharegate
• Sharegate PowerShell
• SharePoint CSOM PowerShell Management
• PowerShell ISE
• Migration Accounts – The number of migration accounts will depend on the number of migration servers involved
• Associate each migration server with respective migration accounts.
• Connect the source and destination with associated migration accounts. Please do not user same migration account in another migration server to connect the destination to avoid throttling issues.
• Dedicate one server as a test server to resolve any issues that might occur during migration
• Source migration accounts should have farm admin privileges
• Destination migration accounts should have site collection admin privileges

Here is the table that shows how to associate the migration accounts with each server for connecting the source and destination:

Migration Server	Source Migration Account	Destination Migration Account	Comments
Migration Server 1	SourceMigAccount1	DestMigAccout1	Perform migration on this server
Migration Server 2	SourceMigAccount2	DestMigAccout2	Perform migration on this server
—	—	—	Perform migration on this server
—	—	—	Perform migration on this server
Migration Server N	SourceMigAccountN	DestMigAccoutN	Perform migration on this server
Migration Server NR	SourceMigAccount NR	DestMigAccoutNR	This server used to resolve or remediate any issues occurring during migration in above servers

It goes without saying that the network quality matters. Check the following for your network.

Network Latency – The network latency depends on the quality of connection between the data center and the two ends – source and destination. The fastest internet backbone available globally is the preferred choice. Our connections are made from networks with low latencies.

Network Speed – Good internet speed is a must to connect the source and the destination.

The Actual Migration

It is important to understand what type of migration you will be performing because it will have an impact on the time it takes to complete the migration. The content migration strategy is planned, and often tailored, based on the character of the content.

Content Only Migration – This is a simple transfer of contents such as

• Document library to document library
• File share to document library
• Tenant OneDrive to different OneDrive (no version, no meta-data)

Site Objects and Content Migration – This has a big scope to migrate from source to destination.

• Migrating from SharePoint 2007, 2010, or 2013 to a newer version of SharePoint
• Migrating to On-Premises or to Office 365 – Migrating to the SharePoint online versus on-premises SharePoint

Content Only Migration

Once the infrastructure is set up, it’s time to find out the throughput of content migration. This helps to estimate the time it would take to complete the migration for a given content size and commit deadlines with the client accordingly.

Throughput is not fixed for migration from SharePoint On-Premise to SharePoint online. Every migration is different – the way every client is different from another client. There can never be an identical set of steps to fix the throughput. These are the areas where migrations are different from one another:

In this type of migration, it more likely to concentrate on only content and its permissions. Below are a few examples suitable for this type of migration

• File share to On-Premise or Cloud (O365)
• Document library to Document library
• From other applications such as dropbox, google drive to On-Premise or O365 libraries

In the visual below, the red boxed area will fall under content only migration.

• List item size is very low and associates with large metadata, workflow, events etc.
• Library item size quite big (files are in MBs) and are associated with versioning, large metadata, workflow, events etc.
• File share involves no metadata and no versioning
• Videos are big-size files.

Site Objects and Content Migration:

Even with using these assessment tools to get a snapshot into potential roadblocks with your migration, there is always a “gotcha” in there. The reasons most Office 365 and SharePoint migrations are so costly and riddled with change requests (usually demanding additional time) – is during a migration some issues are uncovered that weren’t accounted for even with using some of the greatest migration planning tools. Even if you’ve estimated at this point how long your Office 365 migration will take, completing a test migration will either solidify your estimation or send you back to the drawing board – either way, it is necessary to complete one.

Selecting the site for test migration to cover all test scenarios is also one kind of challenging process in this phase. To get to the desired confidence level on test migration, you should understand the inventory report completely where it has the analyzed data on each area from the source environment in terms of content complexity, versioning, custom objects and etc. After a thorough understanding of the inventory reports, you should be in a position to pick a few 5-10 representative sites which should cover all the complex scenarios.

So, pick a site from each key area illustrated below which really have a great impact on migration estimation and strategy. Below are some guidelines that may be helpful.

– Test a site with some sort of complexity and size greater than 3GB – Do not choose a site with a small amount of content or a site that is pretty easy (no customizations, if you have customizations in your environment). Here are few areas to consider

Site having

• More than 3GB data
• Page Layouts
• Site Pages
• Unique permission
• Site columns
• Site content types
• Alerts

– Test a site having workflows and InfoPath forms – If you have SharePoint Designer or Nintex workflows in your environment, this is essential. You may find that you need to have a separate migration plan in place to move this content. This is also true for assessing the migration of your forms. Here are few areas to consider.

Workflow

• More than 3GB data
• OOTB
• Designer
• Custom
• Nintex

Forms

• InfoPath Forms

– Test migrating between SharePoint versions – If you are planning on migrating complete sites either as they are or into a new design, it is important to see how your existing content copies from the old environment into the new environment. Therefore, this helps to understand the post-migration remediation scope for fixing issues on-the-fly or complete re-development.

– Test a site having more file versions, metadata – Choose a site where libraries have huge number of versions for documents, more columns with metadata. This helps to understand the version’s impact on the client. I would always suggest to the client to migrate up to last 5 versions.

Example: Assume that a library has a file with 1MB = 1 file X 1MB = 1 MB size

If the same file has 10 versions = 10 versions X 1 MB = 10 MB size

Since we are copying a file along with versions it will multiply the size with the number of versions which makes a big impact on migration timelines.

Here are few areas to consider

• List/Libraries having
  • More than 30 versions
  • More than 10-15 fields Metadata

– Test a site having more lookup, people & groups – when a site has more lookup and people & groups and custom columns, it increases the migration workload due to the iteration of work referring the related data for each item/files from other objects or sites. Here are few areas to consider

• List/Libraries having
  • Lookup Fields
  • People and groups fields
  • Unique permission

– Test a site having more list/libraries items – Threshold is always one of the challenges in SharePoint migrations from On-Premise to cloud or vice-versa. I have seen many issues arising during test migration because of an improperly chosen threshold. I would suggest to interact with Microsoft support team for your migration and resolve the threshold issue with them. Here are few areas to consider

• List/Libraries having
  • More than threshold items (>5K)

– Test a site having custom site objects or custom columns – I can blindly say these will not be migrated; however, we need to test these functionalities as well to check whether these customizations are affecting any OOTB components in current migration site or scope. Here are few areas to consider

• Custom objects
  • Custom Site columns
  • Custom Site Content types
  • Custom Page layouts

– Test a site having custom solutions/functionalities – In all likelihood, these will not be migrated; however, we need to test these functionalities as well to check whether these customizations are affecting any OOTB components in current migration site or scope Here are few areas to consider

• Custom objects
  • ASP.Net Forms
  • Web Parts

The complexity of the migration increases based on the areas mentioned above. In order to ensure the migration is accelerated and also runs smoothly we need to define an appropriate migration strategy that fits to your client environment.

When there are custom objects present in the content, migration is not possible in one go. We need to plan it in a linear manner so that nothing is missed out and repeating the process can be avoided. An effective linear migration calls for a detailed understanding of all the custom and non-custom objects in the content.

I can recommend a strategy below based on my current migration complexity level having custom site columns, custom site content types, the huge volume of content in the site, unique permission, lookup columns, and other complex areas.

• Disable all alerts on Source
• Execute script for -DenyAddAndCustomizePages $false on target site collections
• Create site collection or subsites at destination
• Copy structure
• Copy page layouts (21) to site collections
• Create site columns in each site collection
• Add site columns to site content type in each site collection
• Add site columns to list content type in each site
• Copy content for lookup lists
• Yes, to all choice type columns
• Copy content
• Enable alerts on target

Please note that the strategy presented above is generic, however, you can pick and refine the strategy as per your client requirement or migration complexities at hand.
I would like to talk briefly about elements used in migration estimation template.

• Size In GB: It indicates the total size of the site
• No of items: it indicates the total no of the items in a site
• Prep Time: It indicates the preparation time for the setting up scripts the migration. This is usually a single time process
• Migration in Mins: It indicates no of minutes taken to complete the execution step
• Total Hours: It indicates the total no of hours take to complete all execution steps

Formulation

Total Time took for test migration = sum (no of minutes taken for each site)
How much GB migrated in ONE hour, the formula is = Total Size In GB/ Total hours

Test Migration	Size in GB		Minutes
No of hours for completing test site 1	0.4		200
No of hours for completing test site 2	1.05		78
No of hours for completing test site 3	0.4		57
No of hours for completing test site 4	0.4		74
No of hours for completing test site 5	1.15		45
Total Size (GB)	3.4	Total Minutes	454
		Total Hours (Mins/60)	7.57

From the above sheet we can say that for total size of 3.4 GB, it took 7.57 hours to complete the test migration.

Total size migrated within ONE hour (total size in GB/total hours)0.449339

&nsp;	GB	Total Hours	Days
Assuming if we need to migrate a Total Size (GB) of	300	667.65	27.82
Formula for splitting the migration on multiple servers	Total size/size per hour	hours/24
Migration on 1 machine	300	667.65	27.82
Migration on 2 machine	300	333.82	13.91
Migration on 3 machine	300	222.55	9.27

Here is the snapshot that I used for my migration using the above formulas

How elegantly you can use to the Migration tool

In most cases, we have used Sharegate as our migration tool. It helps to migrate content either from the user interface or using the Sharegate’s native PowerShell commands.

It is strongly recommended to use the PowerShell command to execute the migration. It speeds up migration for reasons such as

• Once you create a script for migration, you can run the process for a bulk site or large content
• It needs no manual intervention as compared to using the UI
• It makes the migration process automated

At Netwoven, we have created many Sharegate PowerShell scripts over the past years depending on different migration environments and scenarios. A summary of our experience is as below.

Observations

• Versioning plays a major role in estimation and impacts migration throughput. If the library has a huge version history, then it would be hugely time-consuming.
• It’s also observed that if a library is not enabled with version and has a huge number of files, it can still migrate faster than versioned library.
• Lookup files also impact migration as it has to take the reference point from another list/libraries.
• Unique permission also impacts migration throughput as it has to break the permissions for each list or library or document.
• Custom objects need to be implemented and handled individually before migration.

Conclusion

It is always exciting to move to a new environment with access to new features and a new user interface. But the migration is not an easy process. It is always difficult to commit a delivery schedule to the client but in business, one needs to do that no matter what. However, if an appropriate sample can be chosen during test migration, the data and experience from there may be sensibly extrapolated towards creating the migration plan. Hopefully, the small tips provided in this article will help someone to look at a few things with focus and help speeding up the process of migrating to SharePoint Online.