Microsoft IT is responsible for keeping roughly 250,000 Windows 10 devices that connect to the corporate network up to date. As Windows moves from a product to a service, upgrades are released at a faster cadence, and it’s become important that we can quickly identify any potential deployment blockers and issues. We need that information—as well as visibility into the applications in use on our client devices, and their hardware and drivers—to effectively plan upgrade deployment.
Microsoft IT is using the Windows Upgrade Analytics Service to help us gain a better understanding of the Microsoft enterprise environment. Having that kind of information available during our deployment process helps us make faster decisions, based on actual data.Upgrade Analytics gives us valuable information about the individual components that can be the common source of a deployment failure. Using it, we can create targeted deployment collections that will help us quickly and successfully deploy Windows upgrades to the broadest number of devices.
Upgrade deployment challenges
With every deployment and upgrade, there’s some potential for issues or productivity loss. We need to ensure that our operating system deployment activities aren’t disruptive to our employees.
Deployment failures can be caused by application compatibility issues, drivers that can fail or stop working, and other hardware errors. Understanding the environment is crucial in identifying the potential failure points within our computing environment. We have testing practices in place that help us determine issues, or scenarios, that can cause an upgrade or deployment to fail, but we didn’t have complete visibility into the presence of those variables on our client devices.
We needed to gather more data from our Windows devices to help us understand and analyze the enterprise environment. We also wanted the ability to use that information within the context of our familiar management tools to better plan our upgrade deployments.
Upgrade Analytics gathers information and provides insight
Upgrade Analytics is offered as a solution in Microsoft Operations Management Suite (OMS), a collection of cloud based services we use to manage our on-premises and cloud environments. Upgrade Analytics provides:
- Computer, application, and device driver inventory.
- Guided workflows we can use from pilot to deployment.
- Tools to help us make data-driven decisions about our application portfolio.
- Guidance and insights into our organization’s upgrade readiness.
Upgrade Analytics architecture
As illustrated in the following graphic, the Upgrade Analytics service architecture includes both cloud and local resources.
Enabling and configuring Upgrade Analytics
To begin using Upgrade Analytics, we needed to add Upgrade Analytics to our OMS workspace and enable it to collect the system, application, and driver data that it uses to help us assess our upgrade readiness. To add Upgrade Analytics to our workspace, we simply selected it from the OMS Solutions Gallery.
NOTE: Organizations that aren’t running OMS can visit the Upgrade Analytics page and select Sign up now to begin the onboarding process.
Enabling telemetry and connecting data sources
On the client side, most of our devices were already running Windows 10 and we had telemetry enabled. To send that telemetry to Upgrade Analytics, we needed to map the collected information about employee computers to our OMS workspace. We generated a unique Microsoft commercial ID in Windows Telemetry and deployed it to computers in an Upgrade Analytics deployment script. To install the Upgrade Analytics appraiser on computers running Windows 7 and Windows 8, we needed to install a compatibility update.
After telemetry was enabled, computers then started sending encrypted computer, application, and driver information about themselves to a secure Microsoft datacenter through the Microsoft Data Management Service. The information is only identifiable by association to its commercial ID, and it’s only shared in aggregated, anonymized form.
Collecting data through telemetry
To ensure that employee computers are able to send the most up-to-date telemetry data to the service, we:
- Enabled the telemetry service by whitelisting endpoints at the appropriate firewall level.
- Enabled automatic updates to ensure that all updates and related Knowledge Base articles are applied to the devices in our environment.
- Scheduled regular employee computer scans to view monthly active computer and usage information.
Using analytics to plan an upgrade deployment
Upgrade Analytics analyzes system, application, and driver telemetry data, and makes it available through our OMS workspace. where we can use it to plan and manage our Windows upgrades. Upgrade Analytics identifies the application and driver compatibility issues that may block Windows upgrades. We’ve started using that information to make data-driven decisions about our upgrade readiness.
The Upgrade Analytics workflow helps us navigate through the discovery and rationalization process until we have a list of computers that are ready to be upgraded. We’re using the Upgrade Analytics workflow to prioritize and work through application and driver issues, assign and track issue resolution status, and identify computers that are ready to upgrade. Information is refreshed daily so we can monitor upgrade progress. Any changes that we make, such as assigning application importance and marking applications as ready to upgrade, are typically reflected within 24 hours.
The Upgrade Analytics workflow gives us compatibility and usage information about computers, applications, and drivers, and it walks us through these high-level tasks:
- Preparing the environment
- Resolving application and driver issues
- Identifying computers that are upgrade ready
- Review site discovery (optional)
Preparing the environment
To prepare the environment, we use Upgrade Analytics to report our computer and application inventory. The first thing we look at is the Upgrade Overview pane.
The Upgrade Overview pane displays the total count of computers in our environment that are sharing data with Microsoft, and the count of computers that have been upgraded. User changes are processed once every 24 hours and read Up to date in green when there aren’t any pending changes. From this pane, more detailed information is available, click Total computer or Total applications to view the following:
- Total computers provides a list of the more than 400,000 devices in our environment and information about them. Details include:
- Computer ID and computer name.
- Computer manufacturer.
- Computer model.
- Operating system version and build.
- Count of system requirement, application, and driver issues per computer.
- Upgrade assessment based on analysis of computer telemetry data.
- Upgrade decision status.
- Total applications provides both a comprehensive view and granular details about all applications in our environment—it includes all of the managed, line-of-business, and third-party applications installed. Detailed information about more than 140,000 applications discovered helps us make informed decisions about application criticality and includes the:
- Application vendor.
- Application version, including earlier versions.
- Count of computers the application is installed on.
- Count of computers that opened the application at least once in the past 30 days.
- Percentage of computers in our total computer inventory that opened the application in the past 30 days.
- Issues detected, if any.
- Upgrade assessment based on analysis of application data.
- Roll up level.
We can determine the priority level of applications to indicate which applications we should focus on to get them upgrade ready. In Upgrade Analytics, applications are listed, grouped by importance level. We use a combination of usage data collected in Upgrade Analytics, and the criticality levels used by the application compatibility test team to assign priority levels to the applications that are discovered by Upgrade Analytics.
An additional benefit is that we can now see both the presence and usage information for all the applications installed on computers in our environment. Moving forward, we’ll be able to use that information to make data-driven decisions about the applications in our line-of-business application portfolio.
All our early adoption or pilot deployments are self-service, so we don’t create pilot collections for deployment planning. Approximately 1,500 computers were in our pilot group. With Upgrade Analytics, we can compare our list of upgraded computers to the list of computers we attempted to upgrade, see the successes and failures during the pilot, and can use that information during our broad deployment planning. Without Upgrade Analytics, we would have needed to query the devices or the System Center Configuration Manager database to get that information.
Resolving application and driver issues
We use the resolving application and driver issues task in the Upgrade Analytics workflow to report application and driver inventory and determine:
- Which applications in our environment have known issues.
- Which applications in our environment have no known issues.
- Which drivers in our environment have issues.
Upgrade Analytics identifies applications and drivers that need attention and will also suggest fixes when one is already determined. We use this information to help drive our upgrade decisions.
Identifying computers that are upgrade ready
The task of identifying computers that are upgrade ready is as easy as exporting a list. During the previous task, we reviewed and resolved application and driver issues that would block or break during our deployment. As we were resolving application and driver issues and making deployment decisions, Upgrade Analytics was building a list of computers that are upgrade ready.
The computers, grouped by deployment decision, are listed. For more details—including computer name, manufacturer and model, and Windows edition currently running on the computer—we can select Export computers. When viewing the inventory items in table view, the maximum number of rows that can be viewed and exported is limited to 5,000. We can query the data further to refine the scope and export fewer items at a time.
After sorting or further querying the data, we can then select Export to generate and save a comma-separated value list of upgrade-ready computers. We can use exported lists to create collections in System Center Configuration Manager.
Creating computer groups or collections
We were already using collections, or computer groups, through System Center Configuration Manager to help us target computers in specific areas of the business, user groups, or regions during deployments. We now have a wealth of detailed information about the computers in our environment, making the process of creating collections more impactful to our deployment planning activities. We can now create groups of computers based on any type of system or usage information that’s collected, including OEM, computer model, hardware configuration, and installed applications.
Review site discovery
There’s another optional task that is available for use in the Upgrade Analytics workflow. Review site discovery provides an inventory of websites that are being used by client computers that run Internet Explorer on Windows 8.1 and Windows 7 in your environment. We haven’t yet enabled site discovery data in Upgrade Analytics. This information is provided as optional data related to upgrading to Windows 10 and Internet Explorer 11, and is meant to help prioritize compatibility testing for web applications. Using site discovery, you might be able to make more informed decisions about testing based on your usage data.
Using the power of analytics to troubleshoot failures during an upgrade deployment
Our first experience using Upgrade Analytics was during the broad deployment of Windows 10 version 1607 (also known as the Anniversary Update). While we were reviewing the upgrade failures, we noticed that there was a common action that was causing the upgrade to fail. Something was causing an unexpected reboot. We reviewed the logs with some subject matter experts and discovered that most of the computers that the upgrade was failing on had the same application installed. We took a closer look at the application and determined that it had its own mechanism that was triggering the reboot. Using the information available in Upgrade Analytics, we were able to quickly identify all of the machines that were running the application and create a collection.
After we had all of the computers running that application in a collection, we decided how to best address the blocking issue, and applied it to all of the computers in that collection. We have several choices in a scenario where we discover an application is the cause of upgrade failures. We can create a script that will apply a fix for the issue before performing the Windows upgrade, or we can uninstall the application, run the upgrade, then reinstall the application. If the application is critical and there isn’t a fix in sight, we can also choose to not upgrade that collection.
Increased insight improves our existing deployment processes
Upgrade Analytics provides upgrade readiness insights and recommendations about the computers, applications, and drivers in our organization. The insights and recommendations are helping guide us through upgrade projects, using a workflow based on Microsoft recommended practices. The up-to-date inventory data that it collects and makes available in OMS is helping us balance cost and risk in our upgrade projects.
Upgrade Analytics reduces the amount of time and effort that would have been required to manually collect device data and integrate it into our existing management solutions, and it’s helping us gain efficiencies in the decision-making process prior to deployment. It’s also helping improve the user experience and increase satisfaction by making operating system installations more seamless and less prone to errors.
Looking forward, we’re excited to begin using a new feature that will propose action plans for testing applications and drivers. The order in which we perform that testing can have a huge impact on how quickly we’re able to get the computers in our environment ready for an upgrade deployment. The Proposed Action Plan feature has its own algorithm that looks for commonalities or groups of applications and drivers that are typically installed together on different machines. The Proposed Action Plan will provide a recommended logical order of testing our applications and drivers that will help us deploy a Windows upgrade to the largest group of machines in the shortest amount of time.
For more information
© 2019 Microsoft Corporation. All rights reserved. Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners. This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.