Data migration is a major part of moving from one business app suite to another. The process is dotted with hurdles that can cause unforeseen problems for businesses with little or no experience in data migration. Many businesses today are unsure where to start with such a large task. Migrating data without an informed approach and a clear path can get expensive and time-consuming, not to mention stressful.
However, with the right tools and support, migrating data between business apps is a painless process for IT staff, managers and end-users.
This whitepaper will identify the most common pitfalls during data migration and address the ways in which they can be avoided. The planning checklists included will help any business save time and money through each stage of migration.
Migrating business users from their usual email system and business tools to a new system can be a daunting task for all involved. How can migration occur without business interruption? What premigration planning is required? How can you be sure that all data has been migrated? And even if the migration is a success, how do you get your users on board with embracing the new products?
This white paper aims to provide a technical overview of considerations such as migration time, approaches, speed, and security, as well as an overview of what can and cannot be migrated.
Cloudiway is a cloud-based migration platform that contains a broad suite of tools and solutions. to aid migrations from G Suite to Office 365. Many of these fast, secure solutions are tailored specifically for migrations from G Suite to Office 365, with nothing to download or install.
The platform provides a flexible approach to data migration, making it suitable for migrations of all sizes and complexities. Components are modular, allowing you to purchase only the tools you need to ensure all data migrations remain cost-effective.
As well as data migration, logging, and a migration dashboard, Cloudiway’s solutions include the following, which will also be discussed in this whitepaper:
|File migration||Choice of destinations for files including OneDrive, SharePoint or a mix of both. This provides a solution to the challenging problem of file locations and sharing which are dealt with differently in each product (discussed later). Cloudiway will automatically convert popular file formats. Other options include metadata migration and preserving all existing files at the target (by renaming any source files with matching file names when they are migrated to the target).|
|Site migration||Pre-migration utility to create a comprehensive list of sites to be migrated, plus a tool to audit site content to help identify any content that cannot be migrated.|
|Group migration||Pre-migration utility to create a comprehensive list of groups to be migrated, and automatic creation at the Office 365 target of groups or shared mailboxes as well as their members and permissions.|
|Enterprise coexistence||For businesses who expect migration to take longer than a weekend, co-existence offers a means for users on different remote systems to share calendar free/busy time, synchronize global address lists, and make use of mail routing, if required. Components are independent.|
We take your privacy and security seriously at Cloudiway, and we have invested significant effort into making our platform and your data secure. Cloudiway provides a cloud-based application hosted in Windows Azure. It means that the software and data are centrally hosted and accessed by clients using a web browser and internet connection. In addition, Cloudiway’s SaaS benefits from Windows Azure’s certifications, ensuring security of the infrastructure, network and physical security layers of the Cloudiway cloud.
For total assurance, Cloudiway provides auditing tools, secure, authenticated data connections and a logging system. More specifically:
*For the delta pass mechanism, an object’s unique ID is used (eg, messageID or file ID). This ensures that no data is duplicated, and for efficiency, only the changes are propagated.
In addition, because the Cloudiway platform needs credentials to connect to the source and the target, you define connectors to connect to them and enter credentials that will be used for the connection. These credentials are stored encrypted using AES 256.
For complete peace of mind, we recommend that you create a temporary migration account for each remote system during your migration which you can delete at the completion of your project.
We automatically delete inactive projects and/or accounts after 90 days, or upon request.
The Cloudiway platform uses all available resources to provide the fastest migration possible and can support both small and large migrations. The on-demand migration engine allocates the capacity that you need to migrate the volume of data of your choice in the time slot you have allocated.
However, there are limitations. Many mail systems can heavily throttle users. When you perform too many API calls, the remote server will begin throttling and decrease the number of calls that can be performed each minute, thus reducing the migration throughput. Cloudiway continuously attempts to migrate email at the maximum capacity allowed to achieve the highest throughput. It’s also possible to mitigate the effects of throttling, when using the Cloudiway platform to migrate.
Office 365 limitations
Office 365 uses throttling policies to limit the resources consumed by a single account. To maximize throughput and limit throttling, Cloudiway follows Microsoft best practice and uses impersonation.
An account that has impersonation privileges can impersonate 100 users concurrently. The platform uses EWS (Exchange Web services) protocol; Microsoft theoretically allows throughput of around 300 MB per user per hour. The Cloudiway platform typically sees throughput between 200 Mb and 300 MB per user per hour. This gives an average throughput of around 500 GB per day with a constant migration of 100 concurrent mailboxes.
To further improve throughput, you can create additional connectors. For example, if you create two target Office 365 connectors, you can migrate 200 users concurrently and reach a throughput of around 1 TB per day. You can also create connectors for each type of migration you’re performing, allowing you to migrate mail, files, sites, groups, and vaults in parallel.
During mail migration, mailbox item count is also a factor. Office 365 throttling policies limit migration to 1500 to 1800 mails per user per hour. Therefore, a mailbox with 1,000,000 small emails will be slower to migrate than a mailbox with 1,000 large mails containing attachments.
Google limits migration to 2.5 GB per user per day. Usually, some extra data migration is possible before throttling begins. When it does begin, the Cloudiway platform will attempt to migrate 10 GB of data per user, then sleep for 6 hours and automatically restart the migration where it left off.
6.1. Mail – what’s migrated
The following components/features of a Gmail inbox can be migrated to Office 365 mail.
In addition, Cloudiway provides some additional tools to enhance and simplify your mail migration.
Vault archive migration to inbox, archive or mix of both
Google Vault archives can be migrated directly into an Office 365 inbox, or to the In-Place Archive folder, or an entirely separate mailbox, if required. You have full flexibility over Vault migrations. Mail items and their attachments can be migrated.
Email address conversion
This option rewrites email addresses found in the header of mail being migrated and replaces source email addresses with their corresponding target email addresses. For example, if Bob sends an email to his colleague, Chloe, from his source address firstname.lastname@example.org to email@example.com and a week later, after migration, firstname.lastname@example.org replies to Bob, the Cloudiway platform has already updated SMTP header in Bob’s original email in her inbox, so her reply will be sent to email@example.com.
Relinking migrated calendar events to users
This tool checks all mailboxes in the user list and tries to link appointments for owners and attendees (including resources such as rooms). When completed, all parties can send and receive appointment updates.
Automatic Outlook profile creation
This tool automatically creates Outlook profiles for PCs via a command line, saving the hassle of manual creation and the time involved. It works with Outlook 2013 or 2016 (32-bit or 64-bit).
Inbox migration to inbox, archive or mix of both
Cloudiway can create a partial archive from a Gmail inbox, which can preserve bandwidth immediately after migration. End-users who access mail via Outlook have their mailbox locally cached (in .ost file format). After a mail migration, Outlook will download all migrated mailboxes the first time users access their mailboxes. Therefore, if many users are likely to access Outlook at around the same time after migration, your bandwidth might slow down due to a glut of downloads.
This can be avoided by partially migrating data to the online archive. The data would remain online and accessible from each user’s inbox as an In-Place Archive folder. After migration, only the most recent emails would be downloaded when each user first logs in, reducing overall bandwidth usage due to smaller mailbox sizes. You have full flexibility over how a partial archive is created.
6.2. Mail – what isn’t migrated
G Suite uses labels rather than folders to organize received emails, which means users can apply more than one label to a single email. Office 365 mail doesn’t offer labels, so storage for each email is limited to one folder. The Cloudiway mail migration platform uses the first label applied to an email and creates a folder with the same name, where the email will be stored. Any additional labels are ignored during migration.
Currently, inbound rules (including out of office rules) are not migrated from G Suite to Office 365.
6.3. Files – what’s migrated
The following components/features of a Google Drive can be migrated to Office 365 (OneDrive or SharePoint targets, or a combination of both).
In addition to converting the file format of migrated files, Cloudiway provides some additional tools to enhance and simplify your file migration.
Mix your targets
Shared files and folders are treated differently in Office 365, which means your shared project folder in Google Drive ends up scattered between your own OneDrive folders (for any files you’ve contributed) and the ‘Shared With Me’ folder (for any others other team members have contributed). To avoid this, heavily-shared Google Drive folders are best published to a SharePoint library — a better destination for collaboration.
On the Cloudiway platform, you can mix files and folders between OneDrive or SharePoint. Migration is fully flexible, allowing you to pinpoint a single file or to mass-migrate an entire Google Drive. A file can only be migrated once, so this requires careful planning (as is discussed elsewhere in this whitepaper).
Cloudiway’s audit tool builds a list of all Google Drive IDs and their respective owners, as well as the file location. It also detects Google Drive folders that are heavily shared and that are de facto good candidates for being migrated to SharePoint Online.
You can use the audit results to decide whether you wish to migrate any files or folders to SharePoint Online. and if so, you can specify the site collection and document library for each folder to be migrated. Within document libraries, folder structures are entirely recreated.
These folders with specific destinations on SharePoint Online would need to be migrated prior to the general migration because Cloudiway only migrates a file once. Therefore, any folders with alternative targets will take priority.
The preprocessing task verifies that the mapping table matches the accounts declared in Google. It also checks the G Suite credentials to ensure migration can begin. When OneDrive is a target, the preprocessing tasks will check target credentials as well as provision any OneDrives that don’t already exist and assign permission to write to the OneDrive targets.
Opt to retain duplicates at the target
If your target already contains files and one has the same name as a file in the source, you have the option to overwrite it or preserve it. If you have opted to preserve all target files, the file from the source will be migrated with ‘_OLD’ appended to the file name (eg: ‘Timesheet_OLD.xls’)
6.4. Files – what isn’t migrated
Some native Google Drive file formats cannot be converted to Office or third-party equivalents, so they are not migrated. The following items are not migrated.
6.5. Sites – what’s migrated
The following components/features of a Google Sites can be migrated to SharePoint.
Cloudiway provides some additional tools to enhance and simplify your file migration.
Get Sites tool
The Get Sites tool returns a list of all sites from any domains you’ve identified. This is a useful tool which provides a complete picture of sites that Cloudiway has access to migrate and avoids typos or spelling mistakes in long URLs.
The auditing tool helps you identify potential errors prior to migration, such as unfound sites or broken items. It also discovers Google Gadgets and helps to identify if the platform can migrate them to an equivalent web part or not).
We recommend that you run this audit as many times as required prior to migrating to ensure your migration list is fully prepared and ready.
6.6. Sites – what isn’t migrated
Google Sites and SharePoint site collections are set up differently, which makes it difficult for some elements to be migrated successfully.
Google Site pages are organized in a tree hierarchy where pages can contain sub-pages (such as http://www.mysite.com/mainpage/subpage. SharePoint stores each site’s pages in a flat library; to avoid page naming conflicts, Cloudiway migration platform renames Google Sites pages as ‘mainpage_subpage’.
Google Site menu depth is unlimited, whereas by default, SharePoint is limited to two nodes. By default, Google Site menus with a depth of more than two nodes cannot be migrated. Only the first two levels will be migrated.
The Google menu control can contain text. SharePoint menus cannot, so text content in the Google Site menu control is lost.
The site logo is not currently migrated, but a solution is being developed so that it can be in future. Please get in touch if you would like this functionality.
Google gadgets that do not have web part equivalents are not migrated.
Announcements are migrated to SharePoint discussion boards. However, discussion boards do not support attachments. To work around this, announcement attachments are migrated in a SharePoint library with the post title.
Automatically generated menus (created with the ‘Automatically organize my navigation’ option within the ‘Configure navigation’ pop-up of any menu) are not migrated. However, if these are constructed manually, they can be migrated.
Google Drive files can be embedded in Google sites, but file owner information is not stored within the links, making it impossible to assign an owner and give permissions during migration. It’s also impossible for the Cloudiway site migration platform to determine where files are stored. To resolve these problems, consider using Cloudiway’s file migration tool, which can locate files and assign correct user access permissions. If you choose not to use the tool, none of the Google Drive files can be migrated (even if they’re public).
We strongly recommend you use Cloudiway’s file migration tool to in conjunction with the site migration tool to achieve the best migration results possible.
6.7. Groups – what’s migrated
The following components/features of a Google Groups can be migrated to Office 365 (groups or shared mailboxes).
Cloudiway provides some additional tools to enhance and simplify your file migration.
Get Groups tool
The Get Groups tool returns a list of all groups from any domains you’ve identified. This is a useful tool which provides a complete picture of sites that Cloudiway has access to migrate and avoids typos or spelling mistakes in long URLs.
6.8. Groups – what isn’t migrated
Some content from Google Groups cannot be migrated.
It’s important to distinguish between attachments and embedded files. When a file is attached to Google Groups content, it is migrated. However, embedded files and folders are not migrated during Google Groups migration. As a result, their links will continue to work after the migration is complete. If you are also performing a Google Drive migration, bear in mind that any embedded files in a migrated group will continue to be accessed from the source rather than the target.
The Cloudiway platform doesn’t use any Google APIs to access Google Groups. To perform the migration, you will need to create a new Google user to use during migration and manually give it manager permission to each source Google group that you wish to migrate.
If any of your users have created a group, they will also need to add the new Google user to the group with manager permission. It’s therefore important that you notify your users that any groups they have created cannot be migrated unless this new user is added.
Delta passes can be used with group migration to ensure all batch migrations are completed. The Cloudiway platform uses the URL of the source group as the unique ID during migration. This ensures a group is only migrated once. However, this also means that a new reply on a group that has already been migrated will never be migrated.
6.9. Enterprise coexistence – what it does
Enterprise coexistence is a suite of communication tools that run independently of migrations. Particularly useful during company mergers of different systems, coexistence may also be useful during longer-term migrations. It consists of three independent modules:
Calendar free/busy manages cross-platform communication with no impact on the end user. It provides a seamless connection between G Suite and Office 365 users who wish to look up free/busy time of users on either or both remote systems.
Mail routing can be used during long-term mail migrations. Remote systems must be able to carry on as normal, adding or deleting users, or perhaps adding or depreciating DNS domains. The Cloudiway email routing tool provides the redirection of incoming emails to the correct mailbox regardless of its migration status via the inbound mail routing service. Mail routing also performs proxy server services.
GALSync automatically checks and updates user details, contact lists and group information stored in your global address lists on G Suite and Office 365. GALSync can be configured to check for address list changes as often as required, so each global address book remains up to date without any manual input.
Cloudiway’s migration platform helps businesses perform elaborate technical migrations through a simple SaaS interface. As a result, migrations require no additional software installation or overhead, and migrations can be performed securely and quickly.
The Cloudiway platform is flexible enough to support all types of migration paths. Your migration strategy will depend on your business setup, type and size. Whichever migration path you choose, Cloudiway provides all the essential features outlined elsewhere in this whitepaper.
Two of the most common migration strategies are cutover and staged migrations. Cutover strategies involve migrating all data over a weekend, ready for your users on Monday morning. Staged strategies provide more flexible migration options, as discussed below.
7.1. Cutover migration
You migrate everybody over a weekend and perform a single migration pass. This strategy is the simplest to implement. After you have switched your MX records to point to the new system, you start migration.
Cutover migration is therefore a strategy where the entire company is switched at the same time.
7.1.1. Cutover migration benefits
7.1.2. Cutover migration considerations
You can combine your cutover migration with pre-staging, if required. In this case, during the days or weeks leading up to your cutover, you would migrate all mails, files, sites and groups older than, say, a week or a month, along with calendars and contacts, then on the day or weekend of your cutover, you would run a quick delta pass to migrate the remaining items.
7.2. Staged migration
A staged migration allows you to migrate batches of data over the course of a few weeks or months. This strategy is useful for migrations with large volumes of data (very full mailboxes or many Google Drives) and you estimate that you won’t be able to do your migration over a single weekend.
Cloudiway offers you additional flexibility in your approach to a staged migration. For example, for your email migration, you could migrate the last six months of emails over a weekend and leave older emails and email archives to be migrated after cutover, explaining to users that their older emails will appear soon.
Prestaging is also an option on the Cloudiway platform. For example, you could perform a multi-pass migration where you migrate most items before performing the final cutover. During the days or weeks leading up to your cut-over, you would migrate all items older than a week or month from mailboxes, Google Drives, groups, sites and vaults, then on the day or weekend of your cutover, you would run a delta pass to migrate the remaining items.
Cloudiway provides a number of options to help you find the best strategy for a staged data migration. We provide coexistence services, plus mail routing, and batch migration of users, which you can define in any way you like. Basically, you can choose who, when and what gets migrated during each pass. Create batches by department, position, alphabetically, by certain types of data (mail, files, groups etc.), or any other way to suit your business needs.
7.2.1. Staged migration benefits
7.2.2. Staged migration considerations
Staged migrations tend to be more complicated than single cutover migrations. Therefore, it’s important that you have planned your approach thoroughly prior to starting any migration. This is particularly importing if you’re performing file and site migrations to a mixture of targets.
Google Drive and OneDrive both provide a shared file area. Google Drive users can move shared files to other private folders within their drive without changing any existing sharing rights. In comparison, if a OneDrive user moves a shared file, any existing sharing rights will be broken and the file will take on the sharing rights of its new parent folder.
During migration, access rights are prioritized over location, so any shared files which have been organized into other folders within a Google Drive will be bundled together into OneDrive’s ‘Shared with me’ area. This isn’t ideal for Google Drive users who have reorganized files shared with them into folders. It’s also not suitable for a shared folder with files owned by a variety of people; after migration to OneDrive, those files will no longer be grouped together.
For example, before a migration, a Google Drive folder called Project X is shared between three users. Each user can see the folder and its contents in their own Google Drive (Google uses links to the original folder but the user is blissfully unaware, and can move the folder to anywhere else in their Google Drive). Each user has placed a document in the folder, which is automatically visible to the other users who have been given access to the folder.
After migration, the documents are split so that the document of each user created is in their private OneDrive area. The shared project documents they don’t own are placed in the special ‘Shared with me ‘ folder within their OneDrive, along with any other unrelated files that have been shared with them. This change in file organization is rarely the desired outcome.
One alternative is to move everything to SharePoint, where shared files will remain within their parent folders regardless of owner. Using the example from the diagram above, the project files would be migrated together to a ‘Project X’ folder on SharePoint. However, private folders and files are also migrated to the SharePoint site. This is rarely an ideal solution either.
The Cloudiway migration platform can migrate Google Drive files to different targets. A user’s shared files and folders can be migrated to a SharePoint site while private files and folders are migrated to their OneDrive.
This is possible through the use of several tools offered by Cloudiway (the file migration tool and the site migration tool) and carefully planning your migration path. You can use Cloudiway’s file audit tool to identify folders that are heavily shared, then direct their migration to either SharePoint or OneDrive. You have complete control over the destination, and you can change target destinations for individual files and folders. The Cloudiway migration process is therefore both easy to automate and flexible to suit your needs.
In addition, Cloudiway can perform file migrations where the destination SharePoint site has already been used to migrate a Google Sites website. No data is overwritten, and no data is duplicated.
A typical migration to overcome the problem outlined above would follow the order:
9.1. Plan, plan, plan
Without fully planning a mail migration from start to end, steps can be missed out or misunderstood and performed incorrectly. At worst, the wrong migration path can lead to an unsatisfactory migration that may need to be restarted from scratch, costing time and money.
Make sure you take time to analyze your migration goals and the details of what, how and when all data should be migrated. Take note of the source and target system details, the size and type of data to be migrated, and the timeframe for migration. Decisions of what to include in a migration are needed for:
9.2. Use workarounds to throttling
As discussed earlier, data migration can slow down due to throttling at both the source system and the target system. Throttling is often unavoidable on both G Suite and Office 365, but being prepared for it and mitigating it through a planned approach can often avoid a lengthy migration disaster.
To improve throughput, create additional connectors. For example, if you create two target Office 365 connectors, you can migrate 200 users concurrently and reach a throughput of around 1 TB per day. You can also create connectors for each type of migration you’re performing, allowing you to migrate mail, files, sites, groups and vaults in parallel.
To benefit from using multiple connectors, every connector must have a unique migration account that isn’t used for any other connectors. You will need to create additional migration accounts in your remote system first, then use each one in a single connector.
9.3. Keep end users happy
The most important aspect of a mail migration is a happy end user. Nobody likes change – particularly the uninformed, untrained user. They’ll be even unhappier if they start their computer on Monday morning and it grinds to a halt because of a poorly performed migration.
An informed user is a happy user! Although the actual data migration process should be as transparent as possible to the end user, keeping them informed prior to migration is beneficial. Consult with staff so they are aware of the full migration process and when it’s due to take place. Provide users with details of what will be migrated so that they too can plan for a successful migration. For example, if you’re not migrating the trash folder from inboxes, let your users know: it might seem like a small detail, but informing will avoid any post-migration complaints about the trash not being migrated. Ask end users to get in touch if they have any concerns about the migration process so that they can be addressed prior to migration.
As the Office 365 interface is different from G Suite, provide training and consider a short tip sheet for users to help them remember the basics. If end-user PCs need to have their settings tweaked after migration (for example, Outlook mail server access), ensure they have support available to either perform the tweaks or to help them through the process. If you’ve decided to archive older emails to avoid any network slow-down after migration, make sure their training or tip sheet has detailed how they can access their archives.
9.4. Check the source system setup
A data migration will fail if it doesn’t have the right information from the source system. One common problem is using email aliases instead of SMTP addresses, making the originating mailboxes impossible for the mail migration tool to identify. Another is that the admin password expires during the course of migration. Some mail migration tools will use a single account with impersonation rights to all user accounts, and these rights must also be granted at the source (or self-service tools offered to end-users to start their own migrations).
Taking time to audit the source system is a requirement for a successful migration. Follow this checklist to minimize any hiccups:
9.5. Check the target system setup
It’s easy to overlook what’s required on a target system prior to migration, particularly if it’s new to the business. Mailboxes can only be migrated to a Office 365 if it’s ready to receive the data. Office 365 must have enough licenses purchased to accommodate all users prior to migration. In addition, all users must be provisioned, otherwise the migrated items have no inboxes or OneDrives to be copied to.
A general checklist for the target system includes:
9.6. Leave time for post-migration tasks
After mailbox migration, your target system might need an updated global address book, or user terminals might need tweaks if mail is accessed through a client. A migration might also involve mail routing and/or a domain name move, and these processes will need to be addressed before, during and after migration. These are just some of the tasks that must be addressed to ensure a successful migration process.
Post-migration tasks will differ, depending on the target system. The checklist below is provided as a general overview to the most common tasks:
Not every migration tool provides a full, flexible suite of migration tools. Cloudiway are the only migration solution to offer a coexistence solution between G Suite and Office 365 for mail routing, free/busy calendar queries and automatic global address list updates.
In addition, Cloudiway’s SaaS-based migration platform is installation-free, saving you time, effort and end-user interruption. It’s also a cost effective solution for all business types and sizes, with two free support tickets at any time during the migration process, plus consulting services are available, if required.
Cloudiway’s mail migration tool also works alongside its other migration tools, including file migration, mail archive migration, site migration, group migration and, of course, enterprise coexistence.
In addition, Cloudiway embraces the security and speed of the Azure framework, with delta pass technology a standard feature of any migration. Your data is always in your control.
Cloudiway’s migration platform is both reliable and secure, with free support to help you during migration, plus fully scalable and flexible migration tools to meet all your migration scenarios. In addition, a variety of licensing options makes Cloudiway’s migration solution the most cost effective migration tool available. And for clients who need more help, our migration experts can provide consulting services to ensure your migration goes according to plan.
A general data migration checklist is included at the back of this document to help you plan for a successful migration from G Suite to Office 365.
If you’d like to test Cloudiway’s data migration platform, you can set up a free trial account today.
Visit https://apps.cloudiway.com to create a no-obligation migration account today.
For more information, get in touch with firstname.lastname@example.org and we will happily answer any questions you may have.
Below is a general checklist to help you plan for migration. Every migration is different, so you might want to add further items to the bottom if you have other considerations.
Migration goals made (eg, to get merged businesses on same mail/business app platform)
List of items to be migrated made (eg, mail, trash, contacts, calendars, delegations, tasks, metadata, source files with the same name as any existing files at the target)
Any existing Vault archives identified and destination chosen (eg, inbox or separate archive)
Migration strategy chosen (eg, cutover or staged)
If staged migration strategy, migration order/batches defined
Decision made about archiving older mail, and destination defined
Number of mailboxes to be migrated established
Expected total size of migration established
Migration timeframe established (eg, start date, end date)
End users notified of planned migration (and training performed/tip sheets provided)
Source server preparation complete (user list ready, special migration account(s) set up etc.)
Target server preparation complete (licenses purchased, resources provisioned etc.)
Post-migration tasks identified and included in migration plan
Any tweaks to end user local email clients identified and included in plan
Any MX record updates identified and included in plan
Any global address list updates to be made identified and included in plan
Outlook profiles recreated at destination, if required
If mail routing was used, services to be deactivated
Identify and find manual workarounds for any items unable to be migrated (eg, custom site gadgets, native Google file formats)
Special mail migration accounts deleted