Microsoft Core Services Engineering and Operations (CSEO) manages one of the largest enterprise infrastructures in the world. CSEO planned and executed the staged migration of more than 150,000 mailboxes from Exchange on-premises servers to Office 365 and Exchange Online in 2015—a multistep process that spanned several months.
By planning ahead and then applying lessons learned during early deployments, we accomplished our goal of moving nearly all user mailboxes and decommissioning the vast majority of on-premises Exchange servers. This paper describes the steps we took, the challenges we faced, and the lessons we learned that IT pros can benefit from as they prepare for their own migration.
This paper is based on the experience of CSEO. It’s not intended to serve as a general procedural guide. Because each enterprise environment is unique, each organization should adapt the plans and best practices presented here to meet its specific needs.
Note that this paper is an edited version of the original and has been updated for length and to reflect product and organizational name changes.
Benefits of using Exchange Online
Before releasing Office 365 to the public, Microsoft recognized the advantage—to both users and the company—of migrating to Exchange Online. Those benefits include:
- Continually updated Outlook Web App and Outlook client
- Access from anywhere and from any device
- A much larger mailbox—up to 100 GB—and unlimited archive quotas (called auto-expanding archiving)
- More storage (50 GB) for on-premises non-login-enabled accounts, such as system accounts not tied to a user
- Guaranteed email service uptime of 99.9 percent
As IT managers, we also benefited from:
- Reduced server hardware and storage, which reduced costs
- Less time needed to support email infrastructure, which freed up resources for other projects focused on features and experiences
- Simplified litigation hold capabilities
- Guaranteed business continuity and disaster recovery
Planning and preparation were key
We took care to prioritize the migration methodically and thoughtfully. We determined which internal business applications would be affected by mailbox migration and decided how to test them. We also developed tools that both CSEO and users could use to fix problems that arose.
But before any implementation could begin, we needed to tell everyone at Microsoft about changes coming to their email, coordinate with stakeholders, and prepare support teams. We confirmed the migration process, so we knew it would work, and we identified business-critical internal applications to make sure they would run.
Once everything was in place, we decided who would migrate first—and then we set the date.
The communication plan
Timely and thorough communication was a priority. We identified stakeholders and asked for their approval. We prepared support materials and trained Helpdesk staff to support the new service.
We also created an array of informative communications and self-service materials for users. Throughout the migration, employees could find information about the service they were migrating to on:
- CSEWeb. CSEWeb is a site on the Microsoft intranet that helps employees find support channels or self-service information about technologies used at Microsoft. We published a matrix to CSEWeb that listed email features before and after migration. This helped set user expectations early by explaining the features they would gain or lose.
- MySetup. When a user goes to the MySetup page, the request to the page identifies the user. The page displays personalized information, including the name and status of their current email service and support and setup preferences. On this page, users could see if their mailbox was still on Exchange on-premises or on Exchange Online.
We communicated with people whose mailboxes were being migrated in two ways. First, we published information to CSEWeb. Then, we sent email. We also shared information with the IT managers at each office site.
We wanted to make sure that users could get the Exchange service reference materials they needed after migration, so we published this information in SharePoint. We used standard templates to display service information, a setup page, troubleshooting tips, known issues, and support and other information.
Both the troubleshooting and known-issues sections saved support teams and CSEO time by reducing the number of user-assistance requests.
The communication cadence
Our communication plan included a series of emails to inform people about the upcoming migration. After trialing different timing for our emails and revieFwing user satisfaction scores, we set up a schedule for user notification. The initial, All-in email went out two weeks before migration, a Scheduled email was sent two days before migration, and a Welcome email was sent immediately upon successful migration. Although some people complained that this schedule didn’t give them enough warning, most agreed that it left enough time to prepare for the migration.
We also set up contingency email templates to use if a migration would be delayed or if there were problems during migration.
Executive support: All-in email
The All-in email notifying people of the migration to Exchange Online came from a high-level executive. We decided on a top-down approach so that users would be more at ease with their executive leadership team paving the way forward. It notified people that their email would soon be moving and that they would receive a follow-up email with more information.
Preparation: Scheduled email
We sent the Scheduled email two business days before the migration. That email informed people of the date their email would migrate and any changes they might experience. We added, “Please read this email and take appropriate action or you may lose email access.” It also gave instructions to:
- Use their username (firstname.lastname@example.org) to log in to their mailbox instead of the domain\alias format.
- Use Outlook Web Access (OWA) and Exchange ActiveSync (EAS) and the new web addresses of each.
- Update to the latest version of Outlook before migration.
- Contact support, along with links to CSEWeb and MySetup.
We also recommended that users print the email in case they could not access their mailbox after migration, and urged users to complete any necessary steps before the migration date.
Completion: Welcome email
We sent the Welcome email immediately after migrating a group of users. We welcomed them to the Exchange Online service, thanked them for their patience, and pointed out differences in the new experience. It was similar to the Scheduled email, but it contained more detail.
Users would obviously not see the Welcome message if their email went down during the migration. Using the Scheduled message, they could find support information on the CSEWeb pages as well as finding their current service on the MySetup page at any time in the process. Finding their current service status told users whether their migration had succeeded.
After migration, users’ mobile devices stopped synchronizing email until users reconfigured them by correcting the EAS address in configuration settings. For users, this was one obvious way to know that they had successfully migrated.
Follow up: Delayed email
We sent a Delayed email if we had to postpone a migration. We sent it two business days after the scheduled migration date if it was not completed. We informed users that we were still working on their migration, thanked them for their patience, and reminded them that they’d receive another email after migration was complete.
An Incident email was sent when something went wrong. It described the incident and the resolution, and it reminded users that they could check the status of their service on the MySetup page.
During the planning, execution, and follow-up phases of migrating to Exchange Online, we engaged Microsoft employees in other ways.
Marketing, surveying, and special requests
To attract volunteers for the first migration, we placed notices on the corporate intranet home page and on digital displays around the corporate campus. A week after migration, we sent out a survey that asked people about their migration experience. A month later, we sent a survey that asked about overall performance and satisfaction with the service. We also provided forms for people to make special requests:
Sign up. If users wanted to migrate early, they could fill out and submit a sign-up form.
Opt out. To request a migration delay, users could submit an opt-out form. Once a particular reason for an opt-out was addressed, targeted communications were sent to that audience informing them that they were eligible for migration and would be migrated in the near future.
Request fulfillment. This page offered forms through which users could request changes, such as adding accepted domains and adding an IP address to the allow list. We monitored these requests and collected all necessary information before routing the request to the appropriate team.
Establishing governance for a smooth transition
Other Microsoft teams were involved in the mailbox migration effort. The purpose of the governance step was to make sure that these teams understood how migration would affect their workloads and responsibilities, and to give them the opportunity to ask questions, raise objections, and suggest changes.
We created a PowerPoint deck that described the project charter for the migration. We sent this deck to the required governance and review channels, also called CABs (change advisory boards) or stakeholders. These partner teams included Networking, Identity (Active Directory/Accounts/Federation Services), Legal, and the Support and Helpdesk teams. We needed to make sure that all teams were prepared to support migration.
As the last step in the governance process, we got signoff from stakeholders on the final version of the project charter. At this stage, we engaged our Legal and Human Resources teams to ensure that we were in compliance with local requirements at regional datacenters around the world.
Preparing support teams is vital
To help user support teams prepare for the calls they would receive during migration, we gave them training materials, email setup steps, descriptions of all known issues and bugs, information about the escalation process, and a SharePoint location to direct users for more information, or to leave feedback on a specific issue or bug.
Tier 1 support is the internal Helpdesk. At this level, users might be using an incorrect password. If Tier 1 cannot solve the problem, they escalate the issue to Tier 2. At this level, a user might need to repair their Outlook profile. When an issue couldn’t be resolved at Tier 2, it escalated to the CSEO team that worked on the migration, or Tier 3. Issues at this level might be problems such as bugs in the product or issues that need to be fixed at the identity level.
Defining escalation paths
To prepare for migration, we had to define and communicate escalation paths. The escalation paths changed according to the phase of the migration.
- During the Pilot phase, we escalated directly to the Exchange product group. When we encountered a non-trivial issue, we emailed a specific engineer in Exchange. This was the most efficient tactic because we expected things to break in this phase.
- After the Pilot phase, we followed a more typical process of going through Helpdesk and then, if necessary, escalating to the Exchange product support team in Microsoft Customer Service and Support.
- We created escalation paths for other stakeholders, including the Identity and Networking teams.
- We prepared our own team members by holding daily sync meetings and making sure that everyone knew where to find the training materials that were distributed to the Helpdesk and product support teams.
Creating service-quality metrics
We set and closely followed many service-quality metrics, such as the number of mailboxes that had been successfully migrated. Watching the metrics during migration helped us make prompt decisions. The primary metric we watched was Helpdesk ticket count. If we noticed more than a 10 percent increase in tickets, we stopped all migration. This increase indicated a serious problem.
Validating migration and testing business-critical applications
Before we could move forward, we needed to validate and test the migration process and locate business-critical internal applications that depended on email.
Validating migration. We created test accounts and migrated them to test migration scenarios using Outlook, using OWA, and on our mobile devices. We developed a validation procedure that guided our testers and focused on different parts of the new service. In one scenario, for example, we tested the Outlook free/busy schedules of users in different Exchange services.
Testing business-critical applications. The many businesses in Microsoft depend on internal business applications that use mailboxes. These are often high-impact applications, which means that if the application (or its email) breaks, it could disable some part of the company. We needed to identify internal apps with mailboxes and prepare them for migration. And someone had to test the applications to find out if we could migrate them to the new service at all.
We located business owners and asked them to test their internal business applications. Once testing was complete and the app was verified on the new service, we could migrate it a later, appropriate date. Any internal business applications that broke during migration had to be reengineered to run on the new service.
We did not know what or where these important applications were when we started this project. Tracking them down required significant amounts of research because some of these applications had no documentation. For others, ownership had changed, making it difficult to find current owners.
Selecting users and scheduling migration
We had to carefully prioritize which users to migrate, and when. The nature of a person’s work and the technologies they worked with were considered carefully. Some examples of scheduling choices include:
Engineering first. We migrated engineers and IT people in the pilot phase because they work primarily with their own internal teams. If their email went down, only they would be affected. Plus, these people have strong technical skills, were highly motivated for the migration to succeed, and could give us valuable information about any problems they encountered. We wanted to find problems early in the process so that we could learn from them and apply these lessons throughout the migration.
Customer-facing groups later. We chose to migrate customer-facing employees—those who often work with partners or customers—later in the migration process, after we had expertise in supporting the migration.
Avoid sensitive weeks. We didn’t migrate certain teams at certain times because of the nature or timing of their work. For example, because Sales and Finance employees have special responsibilities near the start and end of months and quarters, we migrated them only during the middle two weeks of a month.
Avoid freeze periods. Freeze periods are the last two weeks of the calendar year (December) and the fiscal year (June). These are busy times for some teams—especially Finance—so we did not want to cause issues and escalations during those times. We also avoided migrating members of any product group that was releasing a new product, which was the case with Xbox One.
Creating tools and reports
We created tools to help us keep track of the overall health of the migration. For example, we knew that during migration we would need to find out how many mailboxes were in each of the services (on-premises Exchange and Exchange Online), so we tracked those numbers by setting up PowerShell scripts to copy data to SQL databases. Other reports included:
Delegation families. A delegation family consists of users who have delegated permissions to someone else (described in more detail in the Pilot phase section).
Mailbox count. The number of migrated mailboxes across all our services and the number of migrated mailboxes classified by office, site, and region.
We also developed:
Outlook troubleshooting tool. We built this tool when we discovered that, after migration, up to 10 percent of users needed to repair their profiles. This tool allowed users to change a value deep in the Outlook settings with a single click. We later added functionality to make registry key changes for users when needed.
Other custom tools. Before migration, we developed tools that let users fix problems themselves by confirming and setting mailbox permissions, for example. We updated these tools to work with the new Exchange Online service.
Tracking issues and bugs
We ensured we had a process to track issues and bugs on SharePoint, and that we had a follow-up and closure process for each issue or bug.
The migration journey
The migration was divided into four phases:
- Pilot phase
- Initial volume phase
- Continuous volume migration phase
- Decommissioning phase
Although any corporate IT migration is likely to encounter many of the same challenges, these aspects of our deployment were unique:
- During the early phases of migration, we moved mailboxes to a beta version of Office 365 Exchange Online because Microsoft had not yet released Office 365. Some migrations that failed in the early years would succeed now.
- We had to migrate mailboxes from multiple Active Directory forests.
- In addition to making sure that the email service worked, we supported ongoing development of new products and features for the Microsoft Exchange product group.
Phase 1: Pilot
During the Pilot phase, volunteers and internal users that would have the least impact on Microsoft business were moved first, giving us time to learn from and refine the migration. We tested our communications plan, refined our user support plan, and fine-tuned preliminary steps so that the process would run smoothly when we automated and expanded the Exchange Online deployment across the company.
Much of the Pilot phase consisted of migrating the Microsoft Office product group and our own IT group. Using lessons learned during this phase, we updated validation testing for new vulnerabilities and specific client scenarios. These scenarios included not only using Outlook and Outlook Web App on desktops and laptops, but using email on mobile devices and other client applications. As a part of this testing, users validated whether all features of the migrated mailbox worked or whether some functionality was broken. We also carefully monitored each batch of migrating accounts and documented Helpdesk issues and escalations.
The most significant lesson learned was that we needed a process to allow migration exceptions. We learned that some people had to temporarily delay migration because something about their environment, such as the applications or technologies they used, was unworkable. We even had to turn down certain volunteers because we found that, for technical reasons, they couldn’t successfully migrate.
We initially followed these steps to migrate mailboxes:
- We ran tools to ensure that mailboxes could be moved by filtering out exceptions and for users opting out.
- We sent out the Scheduled migration email.
- We ran a script that removed any non-accepted domains in each user’s email address list.
- We disabled Exchange Unified Messaging (EUM) for users.
- We added the service address (service.microsoft.com) to each user’s email address list. (Note that adding this address can cause a delay due to directory synchronization.)
- Migration was automatically suspended when mailbox content was 95 percent moved, until the scheduled cutover.
- The script retried if migration failed and sent an error log if failures continued, so it could be escalated to service engineering to investigate.
- The auto-suspended migration finished and cut over on the scheduled date.
- We added the service license.
- We re-enabled EUM.
- We sent the Welcome migration email.
- We sent the migration survey to users seven days after the completed migration.
- We sent the performance survey to users 30 days after the completed migration.
As we continued to migrate users and discover issues, we added more steps to the process, including:
- We confirmed that objects were synced across our services and that there were no mismatched globally unique identifiers (GUIDs) or persistent unique identifiers (PUIDs). (Other organizations with less complex deployments than Microsoft should not encounter this issue.)
- Before migrating a mailbox, we copied all the mailbox attributes to ensure that permissions and rules would be intact after the migration.
- We made sure that the migration didn’t create duplicate mailboxes by performing an Active Directory cleanup after migration was complete.
- We verified that the new mailboxes were created, and that mail would be delivered to them.
- We archived and deleted move requests within seven days after the migration. (Typically, move requests were stored on the server for 30 days after migration.)
We encountered a number of challenges during the Pilot phase:
- We were confronted with with the complexity of ensuring a positive client experience across multiple protocols, applications, and networks.
- We had to edit our pre-migration communications to ensure that a general audience could understand all related messages.
- There were initial onboarding issues due to limited egress proxy capacity in the internal network.
- There was an impact on Helpdesk and the user experience, due to limited proactive monitoring of the email service with a tool such as System Center Operations Manager (Operations Manager). This meant that if a service outage occurred, we weren’t notified with a custom alert, so we couldn’t reach out to users quickly enough to prevent calls to Helpdesk.
- We were tasked with mitigating negative assumptions about the service, which caused some users to opt out of migration.
In the hybrid (cross-premises) deployment, Azure Active Directory didn’t always synchronize in a timely manner. We had to manually replicate cross-premises items including dynamic distribution groups and Send As mailbox permissions.
Creating the exception list
User mailboxes were grouped in exception categories that applied to their circumstances. We used Active Directory and PowerShell queries to define the exception-list categories.
Standard: No exceptions applied to these mailboxes. We could migrate them with no known negative effect.
Sales/Finance organization: These people were in the Standard category but were temporarily exempt during sensitive times and freeze periods.
Location: Migration was limited to locations within the United States. Some US office locations were on hold until network signoff due to a network egress issue.
Feature requirement: The mailbox was temporarily exempt from migration until particular features, such as S/MIME encryption, was enabled or legacy telephony requirements were resolved. S/MIME encryption wasn’t supported at the beginning of the migration process, so people who had an S/MIME certificate had to be migrated after the Exchange team added support for this feature. We also had to create dial plans for legacy telephony users, which took extra time and delayed the migration of these users.
Executives and delegated mailboxes: Many executive administrators have delegation authority over their manager’s mailbox so that they can send mail on the manager’s behalf. They might also have delegation authority over conference rooms that have mailboxes, which allows them to accept invitations for the room. This group of executive + admin + conference room is called a delegation family. We worked carefully with delegation families, engaging executive admins to confirm migration schedules. We verified each individual for whom the executive admins needed to maintain delegation, because some of them reported to more than one executive. We migrated people with delegation relationships separately, but we had to migrate each family in the same batch. People in delegation families got special versions of the Scheduled and Welcome emails, which gave them specific resources to use if they encountered problems with delegation.
Additionally, many conference rooms at Microsoft use displays by Crestron. When we started the migration process, Crestron displays weren’t compatible with Office 365. Once a compatible version of Crestron was deployed, we could migrate the mailboxes of all affected conference rooms.
System accounts: Also called service accounts, these were created for business purposes (such as gathering team email), but they aren’t employee mailboxes. We migrated them in a separate batch from their owners but at the same time as their owners. It was important to move them at the same time because cross-premises delegation was not supported. As with delegated mailboxes, we had to engage with owners and then manually move system accounts.
Pilot phase: What we learned
We discovered during the Pilot phase that we had to examine how different parts of Microsoft used various email features so we could enable the corresponding features (if any) in Exchange Online for those users. For example, certain businesses within the company were still using old SMTP services simply because they had been using them for many years.
Before migration, we alerted users to the following tasks:
- To minimize connectivity issues, all Outlook users must update to the latest version. Users can do this easily through Windows Update. (We asked people to do this in the Scheduled email message.) This is necessary because server changes are often applied to Office 365 and all clients must have the corresponding changes.
- Any user who has rules that they want to keep after migration must save them to the server, not to the client. We take this safety measure so that they can repair or rebuild their Outlook profile after migration, if necessary.
We sometimes needed to move people back to on-premises Exchange, at least temporarily. In some cases, they had to return to the original service to keep using a feature that was, at the time, unsupported in the online service. In other cases, people were migrated back and had to remain in on-premises Exchange until we found a solution.
Phase 2: Initial volume
During this phase, we automated the migration process to make it more manageable. This allowed us to increase our migration rate to 4,500 mailboxes per week. Early on, we worked with executives to send the All-in email to people who would be migrated. We continually updated user communications (surveys, the Knowledge Base site, and the email cadence) and we started sending a monthly newsletter to update users on migration progress. For this phase, we targeted certain classes of mailboxes to move first:
- Mailboxes in the Standard category on the exceptions list, namely mailboxes that had no special feature requirements such as S/MIME encryption.
- Mailboxes that were hosted in legacy services that would be decommissioned.
We limited the number of users migrated to 1,500 per day on Tuesdays, Wednesdays, and Thursdays after 6:00 PM Pacific Time to lessen the impact on Helpdesk. We chose these days because on Mondays people are catching up on their work after the weekends, and on Fridays, we wanted to avoid Helpdesk-issue escalations.
We also started tracking service-quality metrics. An example of a service-quality metric is the Helpdesk ticket rate after migration. We would stop migration if the Helpdesk ticket rate exceeded 10 percent for a given migration batch. Similarly, we continually refined the process for analyzing user feedback and taking action. We set up a weekly check-in with partners to review any impacts they experienced. Finally, we expanded our validation process to include other applications, protocols, and networks.
We encountered new challenges during this phase of migration. Our goal was to stay ahead of issues that affected or may affect the quality of service. We worked to counter negative assumptions about the online service that came from users who migrated during the Pilot phase, which caused some users in this phase to opt out. Despite our best efforts, some migrations weren’t completed on schedule.
It was challenging to migrate large mailboxes (over 10 GB), so we updated our processes to start the migration process a week before they were scheduled. We also mitigated possible delays by setting expectations with users—we targeted a specific date, but in some cases it was delayed two or three days, depending on the mailbox complexity.
Updating our processes
During this phase, we made several changes to our processes. The biggest change was automating email communications and the migration process itself. This started with determining mailbox eligibility by using PowerShell to obtain data from Exchange, Active Directory, and HeadTrax (human resources) systems.
To support the migration process and improve the user experience, we developed troubleshooting tools that ran batch files, and we developed client synthetics to establish baseline performance metrics that helped us get a better view of the migration process from the client side. We created and used synthetic (or derived) metrics, which we gathered by running Microsoft Azure and on-premises virtual machines. A client synthetic is an artificial account that emulates a user, which we put through the migration process. By monitoring simulated performance numbers, we could put ourselves in the clients’ shoes and make changes to improve their experience. A synthetic also generates performance alerts, automatically notifying us if something stops running.
At first, we deployed client synthetics to mimic both on-premises (over the WAN) and Azure-based (over the internet) virtual machines to test the performance of client scenarios such as mail flow, load time of free/busy calendar information, and time required for autodiscovery in Outlook. We connected these client synthetics to Exchange at each datacenter location. We developed a C# application to produce synthetic transactions and analyze the Exchange environment. We also developed a PowerShell app that connected to the Exchange environments to check for known issues and bugs.
All synthetic results and data were stored in SQL Server databases in Azure. We received reports for many worldwide locales, which included performance numbers on Autodiscovery performance, MailFlow performance, Create Rule, Create EAS Partnership, Load Free/Busy, and Load GAL (Global Address List). For each of those actions, we got metrics on Min Performance, Median Performance, Max Performance, Average Performance, and Availability.
Phase 3: Continuous volume migration
In this phase, migration became an automated run-state process. Although automation succeeded in general we found more challenges, such as users we could not easily group into a category.
During this phase, we continued to use the established automated process that, based on updated eligibility criteria, continually scheduled the migration of 1,500 users daily from Tuesday through Thursday. We also began to migrate some mailboxes on Mondays and Fridays when necessary.
We avoided migrating on weekends because that was when network maintenance and synchronization were most likely to occur. And after learning about the needs, for example, of executive accounts, we established separate projects to migrate more complex mailboxes.
We set up a service team that was responsible for communications, onboarding, service management, engineering, tools, and reporting. We decided to stop synchronizing with our partner teams (the Networking and the Identity teams) to get signoff for migration, and used an automatic escalation process if issues occurred.
We further developed and updated our troubleshooting tools (including collecting metrics), and we continued to add client synthetic metrics to the process. We used synthetics to test performance so that we would know when to stop migrating, if necessary. We also set up a process to collect additional mailbox metrics, including volume usage and client connections.
We continually updated our user communications and feedback processes, and developed new communications to target specific user personas such as admins, which included information about delegation and calendar management.
Some users wanted to migrate early but weren’t eligible to do so. Highly complex mailboxes, such as those belonging to executives and system accounts, had to wait until we could account for cross-premises delegation. (In this case, cross-premises means spanning the on-premises Exchange service and Exchange Online.)
We recommend that larger organizations account for these complex mailboxes in the migration schedule. Smaller organizations without such complex mailboxes may not encounter this specific problem, but you’ll want to closely examine your own edge cases for potential problems.
A pilot migration can help mitigate the risk here. To account for these scenarios, first define eligibility for a pilot migration as precisely as possible, blocking users who don’t meet the requirements. Migrate those users only when the defined requirements are met, or when the dependencies that led to those requirements have been resolved.
We established a separate process with executives and admins to migrate executives, by first examining the migration schedule and confirming the delegation family to ensure that they migrated as a group. For higher-level executives, we created a “war room” and required key IT staff on-hand from various groups to ensure a smooth transition, or to immediately respond to any issue that might have have surfaced. Fortunately, by that time, migrations were seamless and smooth.
System account migration
Also called service accounts, system accounts are special non-user accounts created for specific uses. We created system accounts to test migration, for example. We advised mailbox users on how to configure their post-migration mailboxes in certain ways, such as how to auto-discover the EWS endpoint.
We established a separate process to engage system account owners and to ensure that their mailbox and programmatic uses could migrate. We also developed service guides that described how to code with autodiscovery, and gave them extra time to confirm that they could successfully migrate.
New tool for account management
The Accounts team created and used an account-management tool that provisions and manages accounts and mailboxes for new hires. Originally, the tool provisioned new hires in on-premises Exchange. We asked them to update their tool so that it would provision new hires in Exchange Online. We then accounted for the time it would take this team (outside of our IT team) to develop and test this tool.
Phase 4: Managing edge cases
Once users in on-premises Exchange servers were moved to the cloud, servers could be decommissioned. We started with more than 150,000 mailboxes and migrated more than 99.9 percent to Microsoft Exchange Online. We needed only a fraction of the original on-premises Exchange servers. We decommissioned the rest, removing over 220 servers.
A couple of issues caused us to modify our processes during this phase.
Addressing dial plans
Some users who had Exchange Unified Messaging (EUM), which supplies voicemail, used dial plans. With a dial plan, they could call a toll-free number to retrieve their voicemail. These dial plans were routed through on-premises Exchange servers. We discovered that we needed to reroute these dial plans early in the process to make them point to the new service.
Working with litigation hold and email data retention
When a legal department places an email account on litigation hold, it means that the contents of the mailbox must be safeguarded for a prescribed retention period. A litigation hold and a 30-day retention period applied to some email accounts during our migration.
In other words, after we moved all the mailboxes from a particular server, we were required to wait 30 days before deleting anything from the server, even though the email was in the new service. After 30 days, we contacted Legal and the HR representative to verify that the retention period had passed. Once they gave their assent, we decommissioned the on-premises Exchange servers.
During this process, we learned that when an account is on litigation hold, the user’s Deleted Items folder doesn’t empty. This causes the mailbox to become very large—in some cases, over 100 GB—and difficult to move. And almost all executive email accounts were on litigation hold.
We worked with Legal teams to determine what email we could remove from a user’s mailbox to reduce its size and make migration more manageable. We also moved content to an archive if the mailbox was too big, in order to start the migration.
Managing other exceptions
We had to delay or cancel the migration of certain mailboxes due to special circumstances, often because of the nature of a user’s job or because of applications and features that caused technical issues. In many cases, we found that migration disabled the mailboxes that we moved. We then had to migrate them back to on-premises Exchange to investigate and solve the problem.
Office 365 email sending limits
In order to accommodate sending and receiving limits in Exchange Online, some high-volume senders remained on our on-premises Exchange infrastructure while we identified alternative solutions to meet the needs of these users sending scenarios. System accounts sending line-of-business alerts and notices only to mailboxes within the tenant can use Direct Send. Those who require advanced analytics, or must send to recipients outside the tenant, must either provision their own SMTP service or engage a third-party SMTP provider.
When another company was sued for overtime pay because its employees checked work email in their off-hours, Microsoft took steps to avoid such a lawsuit. Our original solution in on-premises Exchange was to block access to Outlook when people were not on the corporate network and at specific times of day. However, we didn’t have a post-migration solution in Exchange Online to block access in this way.
Eventually, the decision was made to let these people check mail after hours and be paid for it. When we got official signoff from HR and Legal, we migrated these people, who now have the same access to email as other employees.
At the time of migration, Office 365 was limited to datacenters in the region where the account was opened. For example, users on the Microsoft Redmond campus were subscribed to a tenant in North America, which meant that we could only use datacenters hosted in North America.
With Office 365’s current Multi-Geo capabilities, this is no longer the case. Note, however, that using regional datacenters can head off any compliance or latency issues, and that cross-regional setup may require extra resources to ensure compliance and performance.
Loss of password reset through OWA
In our on-premises Exchange installation, people could use OWA to reset their passwords rather than connecting to the corporate network through Windows. However, Office 365 doesn’t let users reset their passwords in this way, so migrated users lost that capability. The solution was for the Identity team to implement a self-service password reset option. (The Identity team owns Active Directory and the accounts process at Microsoft.) This solution was intended primarily for partners, such as a Samsung employee who works in a Microsoft office and uses a Microsoft mailbox. We had to wait for the Identity team to port a solution for them before we moved them to Exchange Online.
In the process of migrating hundreds of thousands of on-premises Exchange mailboxes to Exchange Online, we learned several valuable lessons. Applying these lessons as best practices can help smooth the transition for those who plan to make the same migration.
These lessons, compiled by our Service Exchange Manager, mostly fall under one of three headings: technical preparation to smooth out the path before the migration; ensuring best practices are adhered to during the migration; and communicating clearly and effectively with end users.
Before migration: Eliminating technical hurdles
Learn about dependent services and service configurations. Familiarize yourself with other dependent services, such as your network infrastructure, DirSync (directory synchronization of Active Directory), corporate account creation, and how your service and client protocols interact with them end-to-end. Get acquainted, too, with current service configurations and settings, and any other legacy artifacts.
In addition, learning to use PowerShell can ensure that you have fast access to the information you need, when you need it. And an IdFix tool can notify you of any identity issues that should be resolved beforehand.
Profile and snapshot user and system mailboxes. Profile mailboxes to determine beforehand the average mail volume through Exchange Client Access Server (CAS) and transport logs, and to determine migration order and mailbox volume. Snapshot mailboxes prior to migration to ensure that you can carry over attributes, permissions, and rules once migration is complete.
Plan for additional network traffic and large mailboxes. Network issues that prevent the synchronization of large mailboxes can bring migration to a halt if not accounted for. Ensure that your network is capable of handling the additional traffic.
Learn how to copy mailbox items to an archive and to remove email data that was delivered during particular times. This should minimize disruption when migrating mailboxes on litigation hold, for example.
Adjust rule quotas as necessary. Exceeding Outlook Rules quotas may result in those rules being disabled after migration. For users with significant amounts of rules (30 or more), consider adjusting the quota. Make sure that all users have updated Outlook to the latest version, and that they’ve saved their Outlook rules to the server (or otherwise backed them up). Document instructions for technicians and ambitious users on how to repair or rebuild Outlook profiles.
Ensure access to your own tools and adjust access in your on-premises environment. Make sure you can access the necessary internal tools via the internet, and that those tools don’t require a VPN to access. Do the same for Microsoft Azure, SharePoint Online, or Microsoft SQL Server, if applicable. Verify that there are no other restrictions to that access that might block migration.
You’ll also want to adjust user permissions at the earliest possible time. For example, after a majority of users were migrated from our on-premises environment, we removed permissions from support teams who no longer needed access to provide user support. As security standards and modern authentication matured, we added two-factor authentication and altered the role-based access control to limit access to designated administrator service accounts.
Make use of the Microsoft Support and Recovery Assistant for Office 365 when needed. The recovery assistant can help troubleshoot, and resolve, network and access issues, for example. Note that in some cases, this functionality can be accessed directly from the Office 365 menu rather than as a separate download.
During migration: Creating a positive user experience
Optimize your network. Pay attention to Network Optimization for online (anytime and anywhere) services outside the corporate network (WAN) and for the internet.
Use federated authentication. Federated authentication (Active Directory Federation Services) will enable users to access corporate services anytime and anywhere over the internet.
Involve teams early and often—especially support teams. Involve teams responsible for the directory service to ensure consistent synchronization across the services.
Make sure that your internal support team is prepared to ensure a good user experience when things go wrong. If the taxonomy is accurate and adhered to, systemic issues can be detected and addressed much more efficiently.
Strive for clear, timely communication at every step
Communication is critical to the migration. Poor communication can hinder migration progress just as easily as technical hurdles can.
Early in the process, vocalize and document a shift from a server-side to a client-side focus. Ensure that your users will be heard through surveys and feedback channels, and take care to keep those lines of communication open and responses prompt.
Set expectations upfront. Make sure users know what to prepare for, and how to prepare for it. For example, if users lose delegated permissions, make sure they know where to turn for support.
Ask for the features you need. If a required feature is unavailable, that feature can be requested through the Office 365 UserVoice forum. Alternatively, you can discuss other options through your Technical Account Manager (TAM).
Managing your own migration journey
By planning ahead and then applying our lessons learned, you can take your company through the migration of Exchange Online with minimal disruptions. After the migration, your users will enjoy the benefits of working anywhere and from any device, while your organization benefits from reduced server hardware and costs, less email support, and reliable business continuity and disaster recovery.
© 2020 Microsoft Corporation. This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.