Microsoft CSEO is meeting the needs of an evolving, increasingly modern workplace by upgrading employees from Skype for Business to Microsoft Teams, the new hub for calls, chat, meetings, and file sharing in Office 365. After introducing users to Teams by configuring it to run alongside Skype for Business, we began upgrading them to Teams-only mode. Using a phased approach, we emphasized readiness and productivity to accelerate adoption and help ensure a smooth transition.

EXPLORE RELATED CONTENT

The concept of the modern workplace is evolving at lightning speed. It is dissolving physical boundaries, moving critical workloads to the cloud, and creating a largely mobile workforce. Employees want physical and digital workspaces where they can easily share, connect, and work together. As part of our ongoing commitment to empower employee productivity and collaboration, Microsoft Core Services Engineering and Operations (CSEO) has been upgrading users across the company to Microsoft Teams, the hub for communication, collaboration, and teamwork in Office 365. Teams offers persistent chat, file-sharing, and a comprehensive meeting and calling experience, with built-in, fully integrated voice and video.

This technical case study is part of a series on how CSEO deployed Teams across Microsoft. To learn about adoption and change management, read the case study “Microsoft Teams adoption strategy prepares employees for a new culture of work.” To learn about the Teams meeting experience, read the business article "Enabling simplified, modern meetings with an integrated Microsoft Teams solution" and the technical case study “With Microsoft Teams Rooms, comes a globally scalable modern meeting experience.”

We have long relied on Skype for Business for messaging and calling at Microsoft. We considered several different coexistence modes as we thought about how to upgrade our users to Teams. Rather than moving straight to Teams-only mode, or running in a coexistence mode that didn’t include capability overlap, we initially configured Teams and Skype for Business to operate in Islands mode (aka side-by-side mode) with feature overlap. In Islands mode, each client application operates as a separate island: Skype for Business communicates with Skype for Business, and Teams communicates with Teams. Users run both clients and can communicate natively in the client from which the communication was initiated. Using Islands mode, we provided employees the opportunity to start using Teams while we were planning our upgrade to Teams-only mode.

In Teams-only mode (see Figure 1), Teams becomes the default client for meetings, calls, and chat. To prepare for the transition, we:

  • Developed a phased-upgrade approach for a smooth transition from running Skype for Business and Teams to Teams-only mode.
  • Established a change management framework and data-driven listening strategy to measure and monitor upgrade success.
  • Implemented an adoption and readiness strategy that included creating a community of Champions to drive awareness and support the shift in the way we work together.

In September 2018, we began upgrading employees to Teams-only mode for conversations, collaboration, calls, and meetings. It’s since become the primary hub for teamwork for more than 200,000 Microsoft employees and vendors. Our goals for the upgrade from Skype for Business to Teams include:

  • Building and solidifying foundations that enable team productivity and teamwork. Beyond meeting the evolving expectations that Microsoft employees and customers have for how they work, we wanted to provide a digital workspace like Teams that feels inclusive and open, where users can work together seamlessly.
  • Simplifying the meeting and calling experience. Upgrading from Skype for Business to Teams gives us a single client application for meetings, calls, and integration with other collaboration tools.
  • Providing better quality and reliability. Teams was built for the cloud. Its modern infrastructure supports meetings, voice, and video experiences that typically result in better quality meetings and calls. This includes faster meeting-join times and improvements on how calls and meeting are handled in low-bandwidth locations. The modern infrastructure also provides administrators the ability to maintain and upgrade the service without interruptions.
Screenshot of Microsoft Teams in Teams-only mode as the default client for chat,  file sharing,  meetings,  and calling.

Figure 1. In Teams-only mode, Microsoft Teams is the default client for chat, file sharing, meetings, and calling

Capitalizing on our early adopters

At Microsoft, employees are usually the first customers for products and services. During the Microsoft Teams pilot, we ran a comprehensive early-adoption program in which we collected user feedback and shared our deployment and integration experiences with the product group. The purpose: to drive improvements before the product was ready for release.

After the general release of Teams, we broadly deployed Teams to run alongside Skype for Business in Islands mode while we onboarded users and promoted Teams as the hub for teamwork. For more than a year we partnered with the Microsoft Teams Engineering Group to define and achieve what we considered a minimum viable product (MVP). Achieving MVP status meant that Teams was ready to provide functional parity with Skype for Business for calls, video, meetings, and messaging (chat).

Before we could contemplate a broad migration to Teams-only mode, we had to vet our approach and ensure that the product was ready to be the default communication and collaboration hub for the enterprise. As we began migrating 20,000 early adopters to Teams-only in waves, using a phased approach, we operated under three guiding principles:

  • Ensure technical readiness. We partnered with the product group and validated user scenarios to make sure that the product was ready to meet the needs of all our users.
  • Encourage user adoption. We created a strategy to drive adoption by winning the hearts and minds of employees.
  • Enable and ease into the change through executive sponsorship and personalization. We streamlined the transition for employees with personalized training, White Glove service, and worldwide support programs. Our leadership also played a significant role.

Ensuring technical readiness prior to the upgrade

During the Microsoft Teams pilot, we worked closely with the product group to ensure that we had feature parity (and beyond) in Teams, for the features and workloads utilized by our users, before we could begin upgrading users to Teams-only mode. We mapped our user scenarios to criteria that defined our MVP. Our early-adoption community provided user validation of the MVP criteria for the following features and functionality: 

  • Conference rooms. We upgraded 14,000 meeting rooms to Teams Rooms (formerly Skype for Business Room Systems v2) to improve the user experience. We wanted to include the convenience of one-click meeting joins for Teams meetings, in addition to Skype for Business meetings.
  • Calling. Users needed the ability to initiate and receive their calls in Teams. We had to ensure call health for both Public Switched Telephone Network (PSTN) calling that enables users to make, receive, and transfer calls to and from landlines and mobile phones and Voiceover IP (VoIP) calling from one Teams client to another.
  • Meetings. Meeting functionality includes audio, video, and presenter capabilities for up to 250 users. Users needed to be able to schedule new Teams meetings in Outlook and Teams, join Teams meetings from their desktop, mobile device, through the web client, or via conference dial-in. Users also needed the ability to join any Skype for Business meetings scheduled by their colleagues or customers.
  • Accessibility. It was crucial to ensure that the productivity of the accessible-user community wasn’t slowed by the change. We needed to show that all accessibility features were available and that all accessible user scenarios were intuitive.
  • Messaging. Users had to be able to receive chat messages from Skype for Business users.

It was important to us that we were doing more than simply achieving feature parity with Skype for Business functionality before we started migrating users to Teams-only mode. We wanted employees to really see that the value of the integrated teamwork hub can be realized only through Teams.

Encouraging user adoption

As we worked to ensure that Microsoft Teams was ready for the Teams-only upgrade, we also worked to ensure that employees were ready for the change. Because Teams is an important part of Office 365, we wanted to create excitement around the launch and set up our employees with the resources they required for a successful transition. We developed a comprehensive data-driven listening strategy and managed our ongoing success through feedback mechanisms.

Our adoption strategy included marketing campaigns and persona-based training to promote the launch of Teams. To build awareness and help employees envision the value of Teams, we created user-validated scenarios to develop Level 100 and Level 200 user training. We developed Art of Teamwork training, “snackable” three- to five-minute activity-based training videos, and other accessible learning engagements that were tailored to roles and organizations. We also established a volunteer community of 1,400 Teamwork Champions. These employees, in 158 buildings in 57 countries/regions, helped spread the word about Teams and provided information, support, and training to their peers.

See “Microsoft Teams adoption strategy prepares employees for a new culture of work” to learn more about how we managed change and drove adoption of Teams at Microsoft.

Sponsorship and personalization enable and ease change

We made sure that we were in alignment with the organization as we created our upgrade deployment timeline, drove adoption, and implemented our plans to train and support employees before, during, and after the migration. We developed a global toolkit tailored to organization and business-area needs. To elevate our engagement, we developed a White Glove training engagement and worldwide support teams that helped prepare leadership and critical organizations, and user groups, for the transition within Microsoft. We also used local adoption teams to guide organizations through change.

The impacts of Teamwork Champions engagement and executive engagement really became evident as we began to look at how they influenced Microsoft Teams feature adoption and usage metrics across business groups (and locations) before and after a Teamwork Champion was introduced, or a leadership communication was sent.

Planning for a successful upgrade

Introducing any new tool can be challenging and requires a thoughtful approach. We couldn’t just replace Skype for Business with Microsoft Teams and mandate that all employees start using it at once. We didn’t want to impact daily productivity without preparing and easing users into the change. With Teams, new ways to collaborate and communicate were integrated in a single hub, and we wanted to give them time to see the possibilities that a hub for teamwork presented.

We wanted to do more than simply ensure that Teams, as a product, was ready for the enterprise, and that we had migrated all eligible users—we wanted to win the hearts and minds of employees and make sure they were ready to embrace the changes that come with a new way of working.

Creating a set of guiding principles for the migration

To ensure a successful Microsoft Teams upgrade, we started with six guiding principles to engineer our approach. They included:

  • Using telemetry to ensure service health
  • Ensuring employees can work with customers and partners during and after the upgrade
  • Keeping end-user productivity as the priority
  • Providing the manageability features required by IT admins
  • Planning support (globally and locally) from IT managers
  • Establishing user confidence

Using telemetry to ensure service health

We required the means to measure improved service reliability and quality in Teams. Service health for Teams is displayed on the Office 365 Admin portal main page. Since Teams is integrated with other Office 365 services, the status of Exchange, SharePoint, and OneDrive for Business are also important indicators.

We also used telemetry to help us monitor top feedback trends. The data-driven insights and user surveys provided visibility into what features were being used and how they were performing.

Ensuring employees can work with customers during and after the upgrade

We accommodated the business needs of specific organizations and user groups within Microsoft when scheduling the specific timing of their upgrade. We couldn’t introduce disruptions to field sellers and the finance organization during Q4 or at month’s end.

We also honored existing Skype for Business meetings and provided the ability to migrate existing recurring meetings.

Keeping end-user productivity as the priority

We ensured that all MVP features were fully functional and available before we started the broad migration to Teams-only mode. One area that we really focused on was making sure that all the meeting-room upgrades were completed first. We also implemented a 10,000-users-per-migration-wave policy to ensure that user disruptions during migrations were minimized, and we placed a high emphasis on providing flexible training options.

We partnered with several different teams, including legal, the product group, and the accessibility-user communities to ensure we were creating valuable training that provided users with the most productive ways to execute key functions of the Teams application. Some of the training modules we created included:

  • Getting Started in Microsoft Teams. This module provided an overview of the user interface, its settings, and the Chat feature.
  • Meetings in Microsoft Teams. This module covered how to create or join a meeting, how to share content, and how to record a meeting.
  • Navigating Teams and Channels. This module included user guidance for finding or creating a team, replying in a channel, and marking favorites.
  • Collaborating in Teams. This module described how to collaborate with others effectively using Teams.  

Our training content, such as the Art of Teamwork, is designed to be accessible to all users. While the courseware itself was accessible, we identified a content gap for users who rely on assistive technology while using Teams. To bridge that gap, we used our existing framework for training and updated the content to add the specific steps a user would need to take when using a screen reader.

For the accessibility-user communities, we gave them as much time as they needed to prepare for the upgrade, gain familiarity with the accessibility features in Teams, and arrange for specialized training as required.

Providing the manageability features required by IT admins

IT admins required the ability to control Teams features at the tenant level, including the ability to manage features like guest access and meeting-migration policies. In Teams there are four admin roles available: Teams service administrator, Teams communications administrator, Teams communications support specialist, and Teams communications support engineer. To learn more about what each role can do and which tools the admin can use in the Teams admin center and PowerShell, read "Use Microsoft Teams administrator roles to manage Teams."

Planning support (globally and locally) from IT managers

We had to ensure that we had a comprehensive change management and support plan in place for greater migration success—in terms of adoption and overall employee satisfaction with the change. We began planning our strategy from the beginning. Before the first rings of early adopters were transitioned to Teams-only mode, we made sure that support teams were ready to help them.

For every new feature that was rolled out, we held a brown-bag session with support teams to help them get ready to answer questions and troubleshoot issues. We also held sessions before each wave of the upgrade to provide details about which user groups were being upgraded and on what timeline.

During each wave of user upgrades, we set up a war room to work directly with support teams and monitored their queues for trends that could indicate possible issues. We had product-team engineers and support-escalation teams on standby to address any issues.

Establishing user confidence

During the broader migration to Teams we worked to provide a smooth transition for users by giving them an intuitive and guided experience before, during, and after their upgrade to Teams. We had a dedicated CSEO website that served as the one source of truth for information. The website included:

  • Announcements and updates
  • Instructions for installing the desktop and mobile apps
  • Learning guides, videos, and training
  • Frequently asked questions
  • Known issues
  • Information about feedback and support channels
  • Links to related blogs

We wanted users to have confidence in the product and in the upgrade process. Throughout the early-adoption program, we remained transparent and authentic about the end-user experience. We provided clear communication about known issues and provided regular status updates about the work being done to resolve them.

Determining upgrade eligibility criteria

As we started planning the upgrade to Teams-only mode, we determined that most employees were eligible for the upgrade. Yet, there were a number of roles in the business that have telephony needs that we can't yet accommodate in Microsoft Teams, and there are also some local considerations in a few countries/regions that make their users ineligible for a Teams-only upgrade. Employees on the ineligible list are exempted from the Teams-only policy change and migration. At this time, they include:

  • Users whose accounts are still in Skype for Business on-premises.
  • Members of response groups or services, including technical support and helpdesk. We are working on implementing call queue in Office 365 and are working with the product group to ensure that there’s feature parity with call queues and Response Groups. Once it’s configured in our environment, before the end of 2019, we will migrate those users.
  • Users in India. Due to laws and regulations in India, users must use location-based physical infrastructure for telephony services. Our employees in India are running Teams in Islands mode with Skype for Business. We are in the process of implementing location-based call routing for users in India and have begun working on the plan for their upgrade.
  • Users who have requested to temporarily defer their upgrade. There are some business reasons why users may request to defer their upgrade, including developers and product engineers that need to support Skype for Business. Another scenario in which a deferment may be requested is when a group of users relies on a custom business application that uses Skype for Business for communications. That group can request a temporary deferment while they update their application to work with Teams.

Allowing users to opt in

We created a form that allowed users who had not yet been upgraded to onboard earlier to the Teams-only policy. During the first two weeks of the migration there was a lot of excitement; as more than 6,000 users opted in early. As long as those users were eligible, the policy was automatically changed within 15 minutes of their request. The Elite early-adoption program, the Teamwork Champions community, and the ability for users to onboard early were key in the accelerated adoption of Teams across Microsoft.

Requesting a temporary deferment

We gave users who weren’t ready to upgrade the ability to temporarily opt out of the Teams-only policy. The request automatically changed the policy within 15 minutes of the request. We used Power BI to gain some insight about which users were opting out and why. During the migration period, fewer than one percent of users temporarily deferred their upgrade. We reached out to users who were requesting the temporary deferments to see if we could address their concerns or issues. In any instance in which a user group had a blocking issue, we engaged the product group. After the migration, we followed up with all deferred users and upgraded them in the final eligible wave deployment.

Evaluating the network

Before we introduced Microsoft Teams in the environment, we had to validate our existing network health to support our Teams deployment and evaluate how Teams would impact our network. We assessed the recommended bandwidth requirements for each Teams activity and calculated the impact. Because we were already using Skype for Business Online and Skype for Business Server for meetings, calling, and chat, the existing traffic patterns don’t differ much from Teams.

Teams creates network traffic in several different ways:

  • Data traffic between the Office 365 online environment and the Teams client for signaling, presence, chat, file upload and download, and OneNote synchronization.
  • Data between integrated internal and third-party apps and Teams clients, such as connectors, bots, and tabs.
  • Peer-to-peer, real-time communication traffic, such as audio, video, and desktop sharing.
  • Conferencing real-time communication traffic, such as audio, video, and desktop sharing.
  • Software updates from the cloud to each client device.

For optimal traffic flow, we had to ensure that traffic on specific ports and IP ranges can pass through firewalls and gateways between our network sites and Office 365 in the cloud. Our network evaluation was straightforward, and we determined that our network was robust enough to handle Teams communication traffic.

For more information about assessing your network and preparing it for Teams, read "Prepare your organization’s network for Microsoft Teams."

Identifying dependent services

Teams combines multiple Office 365 services and is dependent on the correct implementation and operation of those services. These services, all in use in our environment at Microsoft, include—but are not limited to—SharePoint Online, Exchange Online, and OneDrive for Business.

See "Prerequisites and environmental dependencies for Teams" to learn more about prerequisites and about how Teams interacts with other technologies.

Planning for major workloads

We identified our major workloads and considerations for Microsoft Teams implementation, and we established requirements for each to help us plan and prepare for deployment:

  • Collaboration. Adding Teams to our collaborative toolset was straightforward. We were already using SharePoint and Office 365 groups, and Teams’ persistent conversations and its ability to aggregate web, file, and app experiences integrated well without needing much work. Teams offered the hub that brought familiar tools, resources, and people together.
  • Messaging. Messaging, or chat, represented a significant portion of our Skype for Business use at Microsoft. Our support scenarios considered desktop-control capabilities in Skype for Business that we were using. For common user tasks, like accessing chat history, we had to consider user-interface differences between Skype for Business and Teams. Presence-sharing with everyone is always enabled for users in Teams-only mode. If a user had a contact list in Skype for Business, it is visible in Teams. Teams has the capability to provide external access (federation) and guest access. Guest access allows an employee to add an external user as a member of their team. Federation allows an employee to chat with and call users in other organizations who also use Skype for Business or Teams. For a detailed comparison, see "Manage external access in Microsoft Teams.”
  • Calling. We looked at our existing telephony environment alongside Teams’ calling capabilities. For example, if the existing calling services were PSTN-based, we had to blend that technology with Teams. This included setting up direct routing as our solution for PSTN calling.
  • Meetings. Meetings were a significant consideration for us. We used Skype for Business extensively for meetings, and how we transitioned to Teams was important. We had to make sure that both our tools and user behaviors were ready for the transition. We also considered the best mode for running Teams and Skype for Business side-by-side, and how that would affect the implementation and migration. We decided how much flexibility we wanted to give people and balanced that with the amount of support and services we were able or willing to provide. We had the option to migrate all existing meetings to Teams using the Meeting Migration Service. Since it was an all-or-nothing option, and honoring existing meetings was one of our guiding principles, we chose to let the existing meetings stand. We gave users the option to migrate their own existing recurring meeting instances according to their own timelines. This was important for accessibility users and other employees that required more training or time.
  • Integration with other services. If groups had invested in custom business apps, we had to plan for connecting those experiences with Teams. Office 365 connectors, Teams apps, and third-party apps together offered many solutions. Popular integrations used at Microsoft include Planner for task management, Visual Studio for Kanban boards and queries, and website tabs to bring SharePoint sites and web apps into the Teams environment.
  • Storage. Teams simplified our storage options for users. All peer-to-peer chats are stored in Exchange, files shared in peer-to-peer chat are stored in OneDrive for Business, and all team collateral and files are stored in SharePoint Online. That means that users don’t need to worry about where their documents are being saved in the cloud. We didn’t expect storage volume to change because we’ve already adopted OneDrive for Business and SharePoint. While Teams recognizes the compliance and retention policies in OneDrive for Business and SharePoint, there are some regions with specific regulatory requirements that required that we plan for multi-geographic storage when we implemented Teams.

Microsoft Teams desktop-client-installation strategy

The Microsoft Teams desktop client provides the best integration with hardware and operating system capabilities for meetings, calling, and notifications. The client app includes desktop logic-enabled Office 365 client capabilities like:

  • Teams meeting functions
  • Presence information
  • Contact-card settings to initiate Teams chats and calls

The standard desktop-client installation launches Teams at sign-in. When the Teams desktop app starts and a Teams-only user signs in, Teams registers itself as the primary chat and calling client for Office 365. This allows the user to have presence and chat capability in other Office applications, such as Outlook.

Integration with Office 365 ProPlus now includes the Teams client, which allows us to use our established processes for software deployment and PC imaging. Since software updates are initiated directly by the client, no additional client management is necessary for Teams.

As illustrated in Figure 2, approximately half of our employees installed the Teams client on their own. The number of installations increased as additional mechanisms were employed to make the Teams client available.

A graph showing the steady growth of Microsoft Teams desktop-client installations from November 2016 to now.

Figure 2. The distribution of Microsoft Teams desktop-client installations

 

The progress of our Teams desktop-client deployment was tracked through four categories:

  • Viral adoption. Installations that were influenced by early excitement and the network and viral effect.
  • Installation recommended. Organization-driven messaging and in-product prompts promoted installation for desktop and mobile clients.
  • Automated client push. Before Teams-only mode, Software Center Configuration Management installations ensured that integration with Office succeeded. (Automation does not reach 100 percent because some workstations are not managed with System Center Configuration Manager or do not install Office.)
  • Office 365 ProPlus integration. Delivery of Teams has now been established through Office 365 ProPlus deployment channels and new workstation images.

In addition to the Teams desktop-client application, there are also Teams client apps for iOS and Android devices. While we have no formal client-installation strategy for the Teams mobile client apps, based on traffic volumes we have seen an increased rate of adoption and use over the volumes we saw with the Skype for Business mobile apps.

End-user experience in Teams-only mode

Prior to the upgrade, employees ran Skype for Business and Microsoft Teams in Islands mode with feature overlap. This gave them time for our communication campaigns to gain traction and for users to get familiar with the features and capabilities in Teams. This model (seen in Figure 3) is one of the upgrade paths for other organizations that are preparing to phase out Skype for Business and move to the Teams-only model.

An illustration showing the progression from Skype for Business-only mode to Islands mode to Teams-only mode.

Figure 3. Features available to users before, during, and after upgrade

 

Once a user is upgraded to Teams-only mode they can no longer use the Skype for Business application for chat, calling, or scheduling meetings from the ribbon in Outlook. The Skype for Business plug-in for Outlook is set to inactive, but the Skype for Business client is not uninstalled—it stays in reduced-functionality mode, which allows users to sign in, as illustrated in Figure 4, and join existing Skype for Business meetings (or meetings scheduled by their customers). Their existing recurring Skype for Business meetings can be updated or recreated in Teams.

Screenshot shows the Skype for Business log-in dialog box informing user that the organization is now using Microsoft Teams. It includes options for the user to start Skype for Business,  or go to Teams.

Figure 4. Signing into Skype for Business after the Teams-only upgrade

Measuring readiness and success

We are extremely rigorous about measuring our success. We developed a listening system to capture user feedback across Microsoft during the transition from Skype for Business to Microsoft Teams. We also deployed Teams transition surveys (before, during, and after the upgrade) to help us collect feedback about the user experience.

Data-driven insights and user-sentiment analytics help us gain a deeper understanding of the practical challenges that employees face when using Teams in their roles. Teams includes a new analytics and reporting experience in the Teams admin center. It provides different reports that drive insight into how users are using the different features. We combined that information with telemetry, usage, and health data, available through the internal Office 365 tenant in Power BI to create custom views that helped us better understand usage patterns before, during, and after each migration wave. That helped validate our approach, and the impacts of our training and communication efforts.

Our metrics have been trending higher than the call- and meeting-quality numbers we were experiencing with Skype for Business. Meeting and call health are important measurements that help validate upgrade success and monitor overall service health. For calls and meetings, usage and health information is sent from the devices to the team. That telemetry is used to improve the meeting experience. When we measure call or meeting health, we look at audio, video, and content-streaming quality. We flag as unhealthy any meeting or call that has even one unhealthy stream, based on several measurements that mark healthy data transfer and quality. This approach has helped us identify areas for improvement in our environment.

Measuring user-upgrade readiness

We used a combination of measuring user-parity function and listening to user sentiment to help us determine when employees were ready for the Teams-only upgrade. One example of how we used the comparative values of a parity function to evaluate if users are ready to be in Teams-only mode is illustrated in Figure 5. When we first began to migrate users to Teams-only mode we generated a bar chart for the parity function called “sent new private chat message, joined Teams meeting, or made/took call on Teams.” When the delta, or difference, between the users targeted for upgrade and the users who performed all parity functions within the last 30 days nears zero, we consider that user group ready to upgrade to Teams-only mode.

A bar chart showing comparative values of a parity function.

Figure 5. Parity-function usage helps us determine upgrade readiness

Even without specific numbers, in looking at this chart (Figure 5), it is easy to determine that parity-function usage, at that point in time, was well above the number of users that are already in Teams-only mode. We could also see that while a high number of users had performed at least one action in the parity feature we were measuring within the last year, there was still a substantial dropoff in terms of how frequently they continued to use that feature.  

As the migration matured, we began to see how our activities impacted ongoing usage. In the example in Figure 6, we look at how Skype for Business and Microsoft Teams meetings and peer-to-peer private chat trended as we prioritized and refined our scenario-based readiness for those functions.

Side-by-side bar charts. The bar chart on the left compares meetings-joined data for Skype for Business vs. Microsoft Teams,  month by month. The bar chart on the right shows chat usage for Skype for Business vs. Microsoft Teams,  month by month

Figure 6. Measuring the impact of scenario-based readiness

The chart (Figure 6) illustrates how the chat workload shifted from Skype for Business to Teams for a group of users as they have been upgraded to Teams-only mode.

  • Purple: Represents the monthly trend of Teams use for that function per active adoption users.
  • Blue: Represents the monthly trend of Skype for Business use for that function per active adoption users.

In the “Meetings joined” chart, it is easy to visualize how the introduction of the Teams Meeting add-in for Outlook impacted usage. The trend continued to climb as more users within that organization were upgraded to Teams-only mode. In the Chat chart, we were impressed by how much overall usage of chat rose over the prior volume we had seen in Skype for Business.

Measuring the effectiveness and impact of user training

Using data from the Office 365 tenant admin center, we created a Power BI dashboard that helped provide insight into the effectiveness of training and Champions programs, as well as the adoption rate of key features related to user readiness for the Teams-only upgrade.

The dashboard in Figure 7 provides sample metrics taken prior to the Teams-only upgrade and introduction of the Teams Meeting add-in for Outlook. It illustrates how we compared the parity-function usage for attendees and “users under the same direct manager” to determine that, on average, for every single user trained, six other users had increased usage in the key function areas. Analyzing Teams telemetry and metrics in the dashboard helped us assess our month-to-month progress and determine whether the efforts were effective.

A dashboard that illustrates how chat,  meeting,  and call usage is impacted by being trained,  working with people that have taken training,  or working in the same building as a Teamwork Champion.

Figure 7. An example of metrics in the dashboard that we use to measure the impact of training

Defining our upgrade-success criteria

To validate the success of each upgrade phase and to help ensure that our approach was on the right track, it was critical that we defined what success looked like to us and the business. We gathered information from our user scenarios and mapped it to measurable data.

Some of the metrics we used to validate our upgrade-success criteria included:

  • Support tickets. Any increase in the number of support tickets should be less than 5 percent of the total number of upgraded users.
  • Temporary opt-outs. We monitored the number of users requesting to temporarily opt out of the upgrade and verified that such moves didn’t make up more than a certain percentage of the upgrade user base.
  • Calling quality. The percentage of calls that experienced no unexpected issues or drops, and the poor call ratings that we received through in-product surveys.
  • Meeting quality. The percentage of meeting-quality ratings and the percentage of poor call ratings that we received through in-product surveys.
  • User sentiment. Gathered through surveys and sentiment analytics services like the monthly CSEO Sentiment Pulse Survey and the biannual Global Employee Satisfaction Survey, and by monitoring Yammer conversations and roundtables.

Identifying our rollback criteria

Measuring our progress and success was important, and equally so was ensuring that we had defined criteria to help us identify the conditions in which we would roll back a wave of upgraded users.

After every upgrade-deployment wave, we:

  • Validated that the Teams-only policy change went out to every target user within the wave.
  • Re-ran the script to reapply the Teams-only policy for all users if required.
  • Performed a manual spot-check for upgraded users to ensure accuracy and rollback if any issues persist.
  • Set up a war room for each wave of the deployment and monitored the support queues for trends that could indicate possible issues. Some of the criteria we monitored included:
    • Support tickets. If more than five percent of the upgraded user base in a wave requires support, we’ll review and assess their issues for 24 hours. Rollback would be indicated if the number of support tickets increased or if a Severity Level 1 (the most severe) issue occurred.
    • “Opt-out” requests. If more than 5 percent of the upgraded user base per wave requests to temporarily opt out, we review and assess their requests for 48 hours. During that time, we contact users and meet with the organization’s leadership to determine why. If the opt-out requests continue, a rollback for that wave would be indicated.
    • Call and meeting quality. If call- or meeting-quality ratings drop below our acceptable threshold, we review and assess feedback, continue to monitor daily, and escalate to the product engineering group. A rollback would be indicated if the call- or meeting-quality ratings continue to drop and users are encountering unexpected setup or drop issues.

It’s noteworthy to point out that at this late stage of our upgrade to Teams-only mode, we haven’t yet triggered any condition that would call for a rollback.

Lessons learned

  • Don’t assume that your needs or your role in IT or program management accurately represent the needs of the users in your environment. Engage with user groups across your organization to identify their needs and requirements before implementing functional changes that may impact their workflows.
  • Look beyond usage metrics. When employees aren’t using a feature, it may have nothing to do with the quality of the feature—it may not be intuitive or discoverable, or they might not know it’s there.
  • Understand the impact of Microsoft Teams on your existing collaboration and teamwork tools. Be wary of using limited Teams capabilities in your target audience; for example, we had to be cautious about introducing calling capabilities for some people while our on-premises PSTN infrastructure was changing.
  • Ensure that all users targeted for upgrade to Teams-only mode have the desktop client installed. Because early versions of Teams had been made available for employees to try as far back as 2016, many employees had installed—and subsequently uninstalled—the product from their systems. We had to make a special point of ensuring that the Teams desktop client was installed on each employee’s laptop through Configuration Manager before we migrated them to Teams-only mode. Running the desktop client helps ensure that users are getting the richest Teams experience available.

Best practices

  • Provide user training to encourage adoption. As we looked at metrics, we found that even if only a small number of users (or even one) within an organization attended training, adoption and usage metrics went up across the organization.
  • Engage listening systems. Listening systems bring active feedback from our people. We monitored multiple listening systems to increase our awareness of what people are saying and how they’re responding to Microsoft Teams. For example, by identifying themes on our Yammer forum and the types of issues fielded by the helpdesk, we were able to fix environment-configuration issues and develop more prescriptive guidance that helped people use Teams.
  • Manage user expectations during and after the transition. When we were running in Islands mode, some people were primarily using Teams while others were still using Skype for Business. With the feature overlap, some employees were not sure how messages would be handled between the two experiences. It’s important to communicate to your users that they need to be running both Teams and Skype for Business while in Islands mode to receive all their real-time communications. If someone, using the client they aren’t running, tries to contact them, they will receive a missed call or chat notification in Outlook. We used target training to reduce some of the confusion. Teams-only mode is now the default experience for most users at Microsoft, but there are still some who use Skype for Business.
  • Take advantage of your existing security and compliance settings and policies. We didn’t need to create specific security and compliance settings for message retention, Office 365 group expiration, or our SharePoint information classifications. Teams honors the policies we have in place through Office 365 and SharePoint.
  • Establish data governance. Conversation data from Teams is stored in individual Exchange mailboxes for chat and group mailboxes for channel conversations. Therefore, the capabilities in the Office Compliance Center became part of our regular processes for content discovery and legal hold. Similarly, chat files are stored in OneDrive for Business and channel files in SharePoint, enabling capabilities such as DLP and protection policies from SharePoint and OneDrive. You have to ensure that your file-management solutions extend to files added through Teams.
  • Enable self-service creation. A core principle for the Teams implementation was to ensure that any full-time employee could create a Teams channel for a new team or project. We were able to make self-service creation available in Teams and SharePoint through Office 365 groups creation, classification, and lifecycle management, while ensuring IT manageability and protections were in place.
  • Align new capabilities and features with your organization’s strategy. Ensure that the product’s features fit the needs and rhythm of your business. Each business is different, and we found that our Teams adoption needed to be targeted and molded to our vision of communication and collaboration. Align new capabilities of a product to your business strategies and make sure that business needs drive technology adoption—not the other way around.
  • Replace existing collaboration tools with Teams. If your organization is using other collaboration tools and is just starting to implement Office 365, your users might need additional support and training. Include scenario-based training early on to help them make the most of Teams as they shift their communication and collaboration to the new centralized teamwork hub.
  • Offer guest access. Guest access allows us to work with internal teams while extending collaboration to partners and customers outside the company. Because of the high demand for guest access, we needed to quickly ramp up our support staff and update user guidance. We had to help people understand how the guest experience works within Teams and where to get help resolving issues in which guests tried to log in using different email addresses.
  • Review and reuse Office 365 groups. Because people had already started to adopt Office 365 groups for Exchange, Planner, and Power BI, many groups already had a set of members working together. To keep member management simple, encourage owners to review their existing groups when provisioning Teams. A team is provisioned based on existing group files, dashboards, notebooks, and other data. Using existing groups reduces duplication and minimizes redundant entries in the directory.

Accelerating the upgrade without disrupting productivity

We were able to comfortably accelerate both the adoption of Microsoft Teams and the upgrade to Teams-only mode without impacting daily productivity. We placed an emphasis on improving user satisfaction, even in the face of change, and monitored user sentiment with the same rigor that we monitored the health and quality of the service. We not only measured the number of users who were migrated, and their ongoing parity-feature usage, we also looked at how different user-training options and the Champions program were effective catalysts—driving employees to embrace Teams, the hub for communication, collaboration, and teamwork in Office 365.

Are you ready to learn more? Whether you’re just getting started with Microsoft Teams, have begun using Teams alongside Skype for Business, or are trying to determine whether you are ready to upgrade, visit "Getting started with your Microsoft Teams upgrade” to navigate your own successful journey.

For more information

Microsoft IT Showcase

microsoft.com/itshowcase

Microsoft Teams adoption strategy prepares employees for a new culture of work

IT expert roundtable: Transitioning from Skype for Business to Microsoft Teams

Enabling simplified, modern meetings with an integrated Microsoft Teams solution

With Microsoft Teams Rooms, comes a globally scalable modern meeting experience

Improving employee engagement with live events in Microsoft 365

Microsoft Teams

Microsoft Teams

Getting started with your Microsoft Teams upgrade

Microsoft O365 Accessibility

 

© 2019 Microsoft Corporation. This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.


You might also be interested in

Implementing a Zero Trust security model at Microsoft
August 06, 2019

Implementing a Zero Trust security model at Microsoft

Read Article
Microsoft Teams era takes flight inside Microsoft
July 18, 2019

Microsoft Teams era takes flight inside Microsoft

Read blog
Microsoft Teams adoption strategy prepares employees for a new culture of work
July 11, 2019

Microsoft Teams adoption strategy prepares employees for a new culture of work

Read case study
Speaking of security: inside enterprise security at Microsoft
July 11, 2019

Speaking of security: inside enterprise security at Microsoft

Learn more