To protect our most critical corporate assets, Microsoft IT creates secure, isolated environments for business groups that manage highly confidential, regulated, or restricted data. We’re taking advantage of services in Windows Server 2016—including shielded virtual machines and Host Guardian Services—to isolate host, storage, computing, and network services, and to separate component administration within each environment.

EXPLORE RELATED CONTENT

Microsoft IT protects our high-value corporate assets—beyond just the network. We use shielded virtual machines (shielded VMs) and Host Guardian Services (HGS) in Windows Server 2016 to isolate our data. This ensures that control and administration of infrastructure and environment remain completely isolated from control and administration of data and applications.

Critical data and high risk environments

At Microsoft IT, we classify approximately one percent of the services and data that we host as High Value Assets (HVAs). An HVA is a single isolated environment that provides a secure space for company workloads. Access to HVA data by unauthorized users could negatively affect Microsoft business in a significant way.

In our organization, we host several HVAs for different business groups that need a highly secure environment to prevent unauthorized access or data leaks. Most data in an HVA is classified as highly confidential. HVAs also host data that’s regulated by government policy or other legal restrictions, or that’s physically isolated from other datacenter assets and from our corporate network. A typical HVA can be broken down into several components:

  • HVA fabric is the hosting environment for all HVAs. The HVA fabric encompasses the secure hosts, storage, computing, and network services used by all our internal HVA customers.
  • HVA stamp is a single instance of an HVA that’s hosted on the HVA fabric. HVA stamps are also called HVA instances.
  • Tier 0 holds highly privileged Active Directory resources (such as computer and user accounts) that can give an attacker significant access to the network. These resources include domain controllers, which host the Active Directory database, and highly privileged accounts or groups, such as domain admins or enterprise admins.
  • Tier 1 hosts privileged services and systems used by HVAs. These resources provide control over enterprise servers and applications. Tier 1 contains a significant amount of business value hosted in the assets.
  • Tier 2 is also called the customer tier. It hosts services that are provided and managed by the internal customer who uses the HVA. Each HVA hosts a unique set of customers and services. Tier 2 services may be privileged in comparison to other systems, but within the scope of the specific HVA, they are the least-privileged services.

HVA topology

A standard HVA host includes the three-tier administrative model and uses the HVA fabric for storage, network, and related services. The components of an HVA are distributed and managed in highly secured datacenters. Each access tier gives a layer of protection against credential theft.

The HVA system is multi-tenant. Each HVA stamp is an isolated environment that’s built for a specific customer or isolated workload. We use isolation techniques to help create clear boundaries between HVA stamps. HVA stamps can be of mixed size (with a different number of virtual machines, different sizes of virtual machines, and so on) and can host a variety of environments. One HVA stamp might host a single Tier 2 service, and others might host full end-to-end environments that have hundreds of servers.

Figure 1 shows a high-level view of an HVA environment with several HVA stamps.

The graphic depicts 3 HVA stamps,  each consisting of the same 3 tiers: Tier 0 Services (AD,  PKI),  Tier 1 Services,  and Tier 2 (Customer) Services

Figure 1. HVA topology

Using shielded VMs for HVA

To create the private cloud environment that hosts our HVA resources, we use Windows Server 2016, System Center Virtual Machine Manager, and Windows Azure Pack. Windows Server 2016 introduces the shielded VM feature in Hyper-V. It protects virtual machines from threats outside and inside the fabric. It does this by encrypting disk and virtual machine states so that only virtual machine admins or tenant admins can access them.

By using System Center Virtual Machine Manager and Hyper-V host clusters in our private cloud environment, we can quickly and efficiently provision HVAs. We don’t have to worry about provisioning specific hardware to host HVA resources. The Windows Azure Pack offers a familiar, browser-based interface that our internal customers can use to provision resources. When needed, we provision shielded VMs and provide the computing resources to host an HVA workload.

Guarded fabric health attestation and key release

Shielded VMs are part of the guarded fabric system in Windows Server 2016 Hyper-V. The guarded fabric consists of several layered components:

  • Code and boot integrity uses virtualization-based security to allow only approved code to run on the Hyper-V host from the moment it starts.
  • Virtualization-based security uses hardware security technology in Windows Server 2016 to create an area that’s isolated from the Windows kernel and other applications to prevent external attacks.
  • Trusted Platform Module (TPM) 2.0 is used to securely measure a Hyper-V host's boot process and code integrity policy. These are then sent to the HGS as part of the health attestation process.
  • Host Guardian Service (HGS) acts as an arbitration point for the guarded fabric that contains shielded VMs. HGS provides health attestation for the Hyper-V hosts and key protection for the material that’s required to run the shielded VMs.

Guarded host attestation

As illustrated in Figure 2, HGS handles the attestation process for the guarded Hyper-V hosts on which the shielded VMs reside, including key requests and health information. This process ensures the health of the host, the protection of the shielded VM, and the appropriate access for users.

The graphic depicts the process of attestation. On the left side of the diagram is a box labeled host guardian service,  and on the right side is a box labeled Guarded Hyper-V host. The Gaurded Hyper-V host box contains the following components: Shielded VM,  virtual TPM,  Host operating system,  virtualization based security,  hypervisor,  and Hardware trusted platform module v2. There are six attestation steps between the host guardian service and the shielded VMs residing in the Guarded Hyper-V host

Figure 2. Guarded host attestation process with HGS

The attestation process includes the following steps:

  1. The guarded Hyper-V host sends a key request to the HGS.
  2. The HGS replies that it can’t verify that the Hyper-V host is a legitimate host.
  3. The Hyper-V host sends its endorsement key to HGS from its TPM module to establish identity, along with health baseline and code-integrity policy.
  4. If HGS recognizes the identity of the Hyper-V host and considers the baseline and code-integrity policy healthy, it supplies a certificate of health to the Hyper-V host.
  5. The Hyper-V host re-sends the key request and health certificate to the HGS.
  6. The HGS sends an encrypted response back to the Hyper-V host’s virtualization-based security, and the response can be decrypted only by the host hardware security module, to start the shielded VM.

Implementing HVA fabric using shielded VMs

The implementation of HVAs using shielded VMs starts at the datacenter. All HVA servers should be in physically isolated and secure environments. Physical access to the datacenter requires two-person access, and it’s limited to the HVA fabric team and the administrative team.

Physical access implementation

Best practices for implementing physical security components for the HVA include:

  • Physical access to the hosting fabric hardware and datacenter floor should require two-person biometric access controls and smart card access to all server cages and racks.
  • Physical access to the hosting fabric hardware and datacenter floor by an HVA team admin should require datacenter access tool tickets and a fabric admin escort.
  • Datacenter floor access should be granted to only permanent employees.
  • Cameras should be used to record all physical access to the datacenter floor and racks.
  • The datacenter should have around-the-clock security guards on site—they monitor the facility, datacenter floor, and all access paths.

Hardware implementation

We use only specifically configured hardware in our HVA fabric. Our host hardware runs Windows Server 2016 and Hyper-V. Table 1 lists the components and management responsibilities.

Table 1. Hardware components and management responsibilities

Component

Managed by

Description

Fabric host servers with TPM 2.0

Fabric admin team

Host servers are grouped into isolated racks, or pods, and they’re managed by System Center Virtual Machine Manager. They belong to a separate fabric Active Directory Domain Services domain.

Storage systems

Fabric admin team

These are grouped into the same pods as the server infrastructure. HVA fabric storage is provided by System Center Virtual Machine Manager.

Network hardware

Network infrastructure services team in conjunction with the fabric admin team

Firewalls are configured between each layer of the HVA fabric.

Hardware security modules

Network infrastructure services team in conjunction with the fabric admin team

These modules control access to each grouping of Hyper-V host servers that we call a pod. The hardware security modules host secured private keys that participate in the certificate services implementation in HGS. Any administrative function on a hardware security module requires a two-out-of-three security officer quorum.

The graphics depicts the HVA hosting fabric,  which consists of two identical copies of the infrastructure described: there are four layers,  moving left to right. The first layer,  labeled HSM contains HSM modules. This layer is separated by a firewall from the second layer,  laebeld HGS,  which contains HGS servers. The HGS layer is separated by a firewall from the third layer,  labeled Guarded Hyper-V hosts,  which contains hardware pods and VMM. The HGS layer is separated by a firewall from the fourth and final layer: Fabric DCs.

Figure 3. The HVA hosting fabric

The architecture groups together pods of Hyper-V servers as pods, managed by System Center Virtual Machine Manager and fabric domain controllers. The pods are controlled by a group of HGS servers, with access controlled by hardware security modules. We can use this layer separation to separate the administrators of the underlying virtualization fabric from the administrators of the applications and the administrators of the HGS.

Moving forward with HVA using shielded VMs and HGS

We’re experiencing several significant achievements in our HVA environment by using shielded VMs and HGS:

  • Dedicated hosting environment (HVA fabric) is a separate host environment for HVAs. It includes specialized configuration and staff who manage the day-to-day operations of all fabric systems. This hosting environment uses the latest and greatest security configurations and systems to help protect HVA customers and workloads.
  • HVA stamps isolation is a dedicated environment that holds all necessary services to support a designated workload. This isolated HVA stamp is built to protect itself from all outside threats.
  • Role separation keeps every tier of access throughout the fabric, HVA stamps, and related systems isolated. It provides role separation between admins of different functions or types. Fabric admins, network admins, and HVA stamp admins all have isolated credentials and access rights. Most systems also require at least two-person access to change the HVA hosting fabric.
  • Privilege elevation mitigation requires different credentials and remote access systems for each tier. IPsec, Group Policy, and silos prevent tier access cross-pollination.
  • Network isolation keeps each HVA stamp isolated. All inter-network connections are terminated through a physical firewall. Intra-network connections are protected by IPsec, Windows Firewall, and Group Policy configuration.

For more information

Microsoft IT

microsoft.com/ITShowcase

Step by Step – Configuring the Host Guardian Service in Windows Server 2016

 

© 2017 Microsoft Corporation. This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.


You might also be interested in

Microsoft 365 helps create a secure modern workplace
November 30, 2018

Microsoft 365 helps create a secure modern workplace

Read case study
Automating cloud infrastructure management with Azure Resource Manager
September 19, 2017

Automating cloud infrastructure management with Azure Resource Manager

Read case study
Standardizing performance with Storage QoS in Windows Server 2016
October 04, 2016

Standardizing performance with Storage QoS in Windows Server 2016

Read Article
Configuration as code: Automating Windows Server 2016 configuration with PowerShell and DSC
September 23, 2016

Configuration as code: Automating Windows Server 2016 configuration with PowerShell and DSC

Read Article