supercomputing. simplified.

New features in Windows HPC Server 2008 R2 Suite

HPC in the cloud with Windows Azure
Windows Azure is a public flexible cloud-computing platform that offers on-demand, pay-as-you-go, access to highly scalable compute and storage resources with 99.99% uptime. It relieves customers from the burden of having to maintain their own infrastructure in their own data centers. While Windows Azure is well suited for running a wide variety of applications, it is particularly well suited to running HPC applications for two reasons: (i) HPC applications need more computing resources than most other applications; Windows Azure has computing power to match and (ii) HPC applications exhibit a “spiky” behavior i.e. their compute needs can change significantly over small intervals of time. Windows Azure’s pay-for-only-what-you-use model allows you to maintain an HPC infrastructure that meets your typical demands and rely on Windows Azure to address periods of peak computational needs.
Windows HPC Server 2008 R2 Service Pack 2 (SP2) enables service oriented, HPC, jobs to be executed as a service using Windows Azure i.e. you can use resources running within a Windows Azure data center as temporary “compute nodes” to extend the capabilities of your on-premise Windows HPC cluster. The key points to note about this new capability are:
You have the choice of running your jobs (a) entirely on your on-premise clusters, (b) entirely in Windows Azure or (c) partially in Windows Azure and partially on on-premise clusters at the same time. The third choice enables the “burst” scenario where in you run your jobs mostly on your on-premise cluster, but expand to Windows Azure only during peak loads. Note that in all cases, the head-node has to run on-premise. Applications run unchanged on Windows Azure. Your on-premise applications do not have to be changed in any way to run completely or partially in Windows Azure. HPC applications are developed the same way, irrespective of whether run in Windows Azure or on-premise. Note that in this release service oriented, serial and parametric sweep jobs are supported.
Transparent job submission. To a user, submitting a Windows HPC job that runs partly on-premises and partly in the cloud looks just as it always does—it’s no different from submitting one that runs entirely in an on-premise cluster.
Transparent scheduling. To schedule jobs on Windows Azure, all that Administrators have to do is specify (using a graphical wizard) how many Windows Azure nodes they need and at what times. Windows HPC Server takes care of the rest: starting the Windows Azure instances, installing the required software and scheduling the jobs on the Windows Azure nodes. Administrators also have the option of manually starting and stopping the Windows Azure worker nodes.
Integrated administration. For the most part, HPC administrators can treat the Windows Azure worker nodes as regular compute nodes of an on-premise cluster. The same administration tool (the Cluster Manager) works. For example the Cluster Manager Heatmap can be used to monitor the status of all the nodes, including the Windows Azure worker nodes. Additional capabilities are included for Windows Azure specific administration such as determining the Windows Azure instance size (small, medium, large or extra large).
Future releases may provide for additional capabilities such as running the head-node in Windows Azure and the support for MPI and Excel-based.
Windows HPC Server 2008 R2 SP2 brings the scale, power, and economic benefits of public clouds to High Performance Computing with a simple, easy-to-use solution that extends your existing investments.
Download Windows HPC Server and Windows Azure whitepaper
Download Windows HPC with Burst to Windows Azure: Application Models and Data Considerations whitepaper
HPC in the cloud with Windows Azure
Improved Excel 2010 Performance
Microsoft Excel is an important business application used across a broad range of industries. Due to extensive usage for ever increasing number of application scenarios Excel calculations are becoming more and more complex and taking longer and longer to complete. Windows HPC Server 2008 R2 Suite was made to accelerate Excel calculations by one or more orders of magnitude by executing them on HPC clusters in a manner transparent to end users. With Windows HPC Server 2008 R2 Suite three different options are available for speeding up Excel calculations:
Using Excel as Cluster SOA client: Using Visual Studio Tools for Office, developers can write custom code to run Excel calculations on a HPC cluster utilizing Service Oriented Architecture (SOA) calls.
Running Excel User Defined Functions (UDFs) on a HPC cluster: Integration with Excel 2010 enables multiple UDFs to be executed in parallel on compute nodes to increase performance.
Running Excel Workbooks on cluster: Multiple long-running and iterative Excel 2010 Workbooks, in their entirety, can be run in parallel on the compute node of a cluster to achieve better performance. Unattended Excel model can be executed without user interaction.
Download these “Accelerating Excel 2010 with Windows HPC Server R2” Whitepapers
Accelerating Excel 2010 with Windows HPC Server 2008 R2: Technical Overview whitepaper
Building VBA applications and workbooks for a Windows HPC Cluster whitepaper
Building Custom Clients and Services whitepaper
Building .NET Applications that Run Excel Calculations on a Windows HPC Cluster whitepaper
Improved Excel 2010 Performance
Support for Workstations Nodes
Many customers have hundreds or even thousands of very powerful Windows 7 workstations or desktops that are idle for several hours a day, especially at night when their users have gone home. With Windows HPC Server 2008 R2 Suite those workstations can be utilized to run HPC jobs thereby extending the compute capacity of existing clusters without buying additional hardware.
Administrators can control when (time of day and days of the week) the workstations become available for running compute jobs. The Windows HPC Server 2008 R2 ensures jobs are scheduled on workstations only when they are available to do so.
Workstation security is maintained at all times. The workstation owners determine what privileges are available to the HPC jobs running on their workstations.
Workstations can be managed as regular compute nodes using the HPC cluster manager and monitored using the HPC Cluster Manager Heat map.
Download Adding Workstations to HPC Server Clusters whitepaper »
Support for Workstations Nodes
“Business Critical” SOA Applications
Windows HPC Server 2008 R2 enables the development of business critical SOA applications that less susceptible to hardware, software, and network failures:
Durable sessions: Using a Fire and Recollect Programming model that decouples requests and responses developers can implement re-attachable and durable SOA sessions. Requests and Responses, persisted in durable storage, survive broker node failures.
High Availability Broker Nodes: Automatic fail-over across broker nodes in the event of hardware failures increases the reliability of applications. Active sessions on failed broker nodes are automatically migrated to healthy ones. Failed broker nodes are auto-restarted.
Finalization hooks: Enables developers to add logic to perform cleanup before a service exits.
New Service-balanced scheduling policy: Ensure that as many jobs are started as soon as possible, so that important jobs are not starved behind a few large resource-intensive jobs.
Download the SOA Applications, Infrastructure and Management with Windows HPC Server 2008 R2 Suite whitepaper »
“Business Critical” SOA Applications
Improved Performance and Scale
With across-the-board improvements (Job Scheduling, Networking, MPI etc), benchmarks and customer evidence indicate that Windows HPC Server 2008 R2 has performance that rivals Linux for real-world applications.
Out-of-the-box, with little tuning or configuration, Windows HPC Server R2 scales beyond 1000 nodes and scales to much larger numbers with additional tuning and optimization. The Heat map shows the status of 1000 nodes without scrolling or paging.
In the Microsoft Performance Labs hundreds of ISV applications and open source codes have been tuned for optimal performance on Windows HPC Server 2008 R2 Suite.
Includes support for RDMA over Ethernet (iWARP) from Intel and RDMA over infiniband quad data rate (40Gbps) hardware.
Optimization of shared memory implementations for new Intel “Nehalem”-based processors.
Download the Windows HPC Server 2008 R2 Suite Technical Overview whitepaper »
Improved Performance and Scale
Enhanced Systems Management, Diagnostics and Reporting
A customizable console displays Deployment, Job, and Task progress at-a-glance.
Flexible, customizable, color-code Heat map can display the status of a 1000 node cluster at-a-glance, without scrolling. Node status can be viewed in either “overlay” view (one performance counter) or “stacked” view (up to three counters). Nodes can be displayed grouped by their location, specifying up to three levels of location detail for each node: primary, secondary, and tertiary.
Extensible diagnostics and troubleshooting framework, open to partners, allows creation of customizable diagnostics tests in any programming or scripting language. Includes a new set of diagnostic tests to quickly find common problems that can affect node deployment.
iSCSI network boot provides support for diskless compute nodes enables centralization of storage for improved consistency. Support for NetApp filers and Windows Storage Server.
Dual-Boot Clusters: Partnership with Adaptive Computing and Platform Computing to enable dynamic dual-booting of compute nodes so that both Windows and Linux HPC applications can run at the same time on the same cluster.
Extensible Reporting infrastructure lets you capture and report on custom error messages, status messages and usage data. The data is stored in a SQL Server database and be viewed with SQL Server Reporting services or Excel.
Download the Windows HPC Server 2008 R2 Suite Technical Overview whitepaper »
Enhanced Systems Management, Diagnostics and Reporting

Features and Capabilities

Windows Azure Integration
Windows Azure Integration
Windows Azure is a public flexible cloud-computing platform that offers on-demand, pay-as-you-go, access to highly scalable compute and storage resources with 99.99% uptime. It relieves customers from the burden of having to maintain their own infrastructure in their own data centers. While Windows Azure is well suited for running a wide variety of applications, it is particularly well suited to running HPC applications for two reasons: (i) HPC applications need more computing resources than most other applications; Windows Azure has computing power to match and (ii) HPC applications exhibit a “spiky” behavior i.e. their compute needs can change significantly over small intervals of time. Windows Azure’s pay-for-only-what-you-use model allows you to maintain an HPC infrastructure that meets your typical demands and rely on Windows Azure to address periods of peak computational needs.
Windows HPC Server 2008 R2 Service Pack 2 (SP2) enables service oriented, HPC, jobs to be executed as a service using Windows Azure i.e. you can use resources running within a Windows Azure data center as temporary “compute nodes” to extend the capabilities of your on-premise Windows HPC cluster. The key points to note about this new capability are:
You have the choice of running your jobs (a) entirely on your on-premise clusters, (b) entirely in Windows Azure or (c) partially in Windows Azure and partially on on-premise clusters at the same time. The third choice enables the “burst” scenario where in you run your jobs mostly on your on-premise cluster, but expand to Windows Azure only during peak loads. Note that in all cases, the head-node has to run on-premise. Applications run unchanged on Azure. Your on-premise applications do not have to be changed in any way to run completely or partially in Azure. HPC applications are developed the same way, irrespective of whether run in Windows Azure or on-premise. Note that in this release service oriented, serial and parametric sweep jobs are supported.
Transparent job submission. To a user, submitting a Windows HPC job that runs partly on-premises and partly in the cloud looks just as it always does—it’s no different from submitting one that runs entirely in an on-premise cluster.
Transparent scheduling. To schedule jobs on Azure, all that Administrators have to do is specify (using a graphical wizard) how many Windows Azure nodes they need and at what times. Windows HPC Server takes care of the rest: starting the Windows Azure instances, installing the required software and scheduling the jobs on the Windows Azure nodes. Administrators also have the option of manually starting and stopping the Windows Azure worker nodes.
Future releases may provide for additional capabilities such as running the head-node in Windows Azure and the support for MPI and Excel-based.
Windows HPC Server 2008 R2 SP2 brings the scale, power, and economic benefits of public clouds to High Performance Computing with a simple, easy-to-use solution that extends your existing investments.
Download Windows HPC Server and Windows Azure whitepaper » Download Windows HPC with Burst to Windows Azure: Application Models and Data Considerations whitepaper »
Accelerating Excel 2010 Performance
Accelerating Excel 2010 Performance
Microsoft Office Excel is a critical business application across a broad range of industries. Due to extensive usage for ever increasing numbers of application scenarios, Excel calculations are becoming more and more complex and taking longer and longer to complete. Windows HPC Server can accelerate Excel calculations by one or more orders of magnitude by executing them on HPC clusters in a manner transparent to end users. Three different options are available for speeding up Excel calculations:
Using Excel as Cluster SOA client: By using Visual Studio Tools for Office, developers can write custom code to run Excel calculations on a HPC cluster utilizing Service Oriented Architecture (SOA) calls.
Running Excel User Defined Functions (UDFs) on a HPC cluster: Integration with Excel 2010 enable multiple UDFs to be executed in parallel on compute nodes to increase performance.
Running Excel Workbooks on cluster: Multiple long-running and iterative workbooks can be run in parallel on the compute node of a cluster to achieve better performance.
View Video: HPC Services for Excel 2010 » Download Accelerating Excel 2010 with Windows HPC Server 2008 R2: Technical Overview whitepaper »
Support for Workstation Nodes
Support for Workstation Nodes
Most customers have hundreds or even thousands of very powerful Windows 7 workstations or desktops that are idle for several hours a day, especially at night when their users have gone home. With Windows HPC Server 2008 R2 those workstations can be utilized to run HPC jobs thereby extending the compute capacity of existing clusters without buying additional hardware.
Administrators can control when (time of day and days of the week) the Windows 7 workstations become available for running compute jobs. Windows HPC Server ensures jobs are scheduled on workstations only when they are available to do so.
Security is maintained at all times. The workstation owners determine what privileges are available to the HPC jobs running on their workstations.
Workstations can be managed as regular compute nodes using the HPC cluster manager and monitored using the HPC Cluster Manager Heat map.
Download Adding Workstations to HPC Server Clusters whitepaper »
HPC Application Development
HPC Application Development
Developing robust and high performing cluster applications is a challenge even for experienced developers. Windows HPC Server, in conjunction with the highly popular Visual Studio, addresses this challenge through a set of tightly integrated tools specifically designed for parallel and cluster development:
Support for developing MPI-based application including MPI cluster debugger.
Support for developing SOA-based applications including cluster SOA debugger and profiler.
Parallel LINQ and Task Parallel Library (TPL), included in .NET Framework 4.0
Parallel Patterns Library (PPL) and Asynchronous Agents Library included with Visual C++ 10.
Support for GPGPU programming in partnership with NVIDIA.
Download the following resources about developing HPC applications:
Hanweck Associates Risk Management: Technical Computing Tools for Financial Services video » Taking Parallelism Mainstream: Parallel Technologies in Microsoft Visual Studio 2010 whitepaper » C# Cluster-SOA Debugger for Windows HPC 2008 R2 Suite » Building VBA applications and workbooks for a Windows HPC Cluster article and workbooks » Profiling HPC Applications » Classic HPC Development Using Visual C++ whitepaper » Classic HPC Development Using FORTRAN whitepaper » HPC Development Using F# whitepaper » GPGPU Computing Horizons: Developing and Deploying for Microsoft Windows whitepaper »
End-to-End Systems Management
End-to-End Systems Management
A complete, integrated, end-to-end systems management tool (the Administration Console) obviates the need to buy any 3rd party or open-source system management software. Everything that an administrator needs to deploy, provision, monitor, troubleshoot, and manage entire clusters-through the lifetime of the cluster-is included in the graphical and command-line tools:
Heat map, scalable upwards of 1000 nodes in single view, offers at-a-glance view of the status of all nodes in the cluster. Heat map is highly customizable, allowing specific metrics of interest to be easily tracked. It supports multiple, customizable tabs for different views of heat maps and other system data.
A customizable console displays Deployment, Job, and Task progress at-a-glance.
Integration with existing IT infrastructure including Active Directory and Systems Center Operations manager to enable single sign-on, remote access, and integrated management.
Getting Started with Windows HPC Server 2008 R2 » Cluster Administration for Windows HPC Server 2008 R2 » Download the Windows HPC Server 2008 R2 Technical Overview whitepaper » Windows HPC Server 2008 R2 Management Pack Guide »
Rapid Deployment
Rapid Deployment
Rapid and flexible deployment is vital to efficiently set up and maintain large clusters. Window HPC Server includes powerful tools to simplify setting up clusters and networks:
A graphical wizard integrated into the Administration Console to build consistent compute node images that can be created, modified and deployed across the cluster. A "to do" list shows the steps necessary to complete the configuration of a complete cluster.
Rapid deployment of compute nodes in parallel by leveraging the Windows Deployment Service transport using the Windows Imaging Format (WIM) file and multiband multicast.
Compute node images can be easily generated by injecting drivers, applications, or patches. You can import and manage pre-deployed compute nodes.
Ability to boot nodes over the network using an iSCSI connection to enable diskless compute nodes.
In partnership with Platform Computing and Adaptive Computing, supports dual-booting of compute nodes to enable dynamically switching of operating systems so that both Windows and Linux applications can run on the same cluster at the same time.
Deploying a Windows HPC Server 2008 R2 Cluster » Deploying iSCSI Boot Nodes in Windows HPC Server 2008 R2 Step-by-Step Guide » Download the Windows HPC Server 2008 R2 Technical Overview whitepaper »
Platform for Multiple Application Types
Platform for Multiple Application Types
Runtimes, APIs, and a range of job scheduling policies enable multiple types of HPC applications to developed, deployed and managed through a single unified product. These include:
MPI-based applications using either Microsoft's own MS-MPI or 3rd party MPI libraries and runtimes.
Long running batch as well as low-latency interactive applications.
Conveniently parallel applications as well as applications that require extensive inter-node communication and synchronization.
Applications based on the Service-Oriented Architecture (SOA).
Parametric tasks sweeps.
Excel Workbooks and Excel User-defined functions (UDFs).
100s of commercial ISV packaged applications and open source codes optimized for Window HPC Server.
Download the Windows HPC Server 2008 R2 Technical Overview whitepaper »
Powerful, Versatile Job Scheduler
Powerful, Versatile Job Scheduler
A multi-core, job scheduler allows for scheduling by core, socket or server, and addresses a variety of application types. Windows HPC Server offers a range of scheduling policies: Preemption, Priority, Grow and Shrink, Backfill, Exclusive, and (new in R2) Service Balanced Scheduling.
Support for enterprise scale clusters—upwards of 1000 nodes―and large number of Jobs.
Just-in-time parametric sweep expansion for creating large parametric task sweeps.
Multiple interfaces including GUI, Command line, Windows PowerShell 2.0, COM, .NET and Open Grid Forum's HPC Basic Profile Web Services Interface.
Ability to partition large clusters into node groups for manageability and better matching of Jobs to server types. Changes to node groups immediately impact the Jobs that are queued.
Command line and API support for prepare and release tasks that run before and after a job to prepare and cleanup nodes.
A new user interface and API for viewing and reporting job progress.
Download the Using Windows HPC Server 2008 R2 Job Scheduler whitepaper » Submitting Jobs to a Windows HPC Server 2008 R2 Cluster » Download the Windows HPC Server 2008 R2 Technical Overview whitepaper »
Diagnostics and Error Reporting
Diagnostics and Error Reporting
Extensible diagnostics and troubleshooting framework, open to developers, allows creation of customizable diagnostics tests in any programming or scripting language. Includes a new set of diagnostic tests that help you to find common problems that can affect node deployment.
Extensible Reporting Infrastructure lets you capture and report on custom error messages, status messages and usage data. The data is stored in a SQL Server database and can be viewed with SQL Server Reporting services or Excel.
Integration with the Event Tracing for Windows (ETW) infrastructure for easier trouble shooting.
A rich collection of pre-built diagnostic reports to help system administrators verify the correctness of their clusters and networks.
Download the Windows HPC Server 2008 R2 Technical Overview whitepaper »
Service Oriented Architecture HPC Applications
Service Oriented Architecture HPC Applications
Service Oriented Architecture is a powerful way to build distributed, loosely coupled, interactive applications using web service standards. Windows HPC Server brings the power, flexibility, and ease of development of SOA-based applications to HPC.
A service-oriented programming model for developing HPC web service based on the Windows Communication Foundation (WCF).
A cluster SOA debugger and cluster SOA profiler for building high performing, robust HPC SOA applications.
Support for business-critical SOA applications through multiple techniques including durable and reattachable sessions, persistent requests and responses, automatic failover of failed SOA sessions fire-and-recollect programming model, and auto-restart of broker nodes.
New Service-balanced scheduling policy so that as many SOA jobs are started as soon as possible― important jobs are not starved behind a few large resource-intensive jobs.
Download SOA Applications, Infrastructure and Management with Windows® HPC 2008 R2 whitepaper »
MPI, Networking and High-Speed Interconnects
MPI, Networking and High-Speed Interconnects
A network configuration wizard and diagnostics tools, and simplified driver management, help to reduce the complexity of configuring multiple networks across a cluster, while use of NetworkDirect helps to achieve better performance and enable efficient CPU utilization.
NetworkDirect, Microsoft’s RDMA networking interface built for low lat and high bandwidth applications. Includes support for infinband quad data rate (40 Gbps) hardware.
OpenFabrics driver for Windows includes support for NetworkDirect, Winsock Direct (Sockets Interface) and IPoIB (TCP/IP over Infiniband protocols)
Integration with the Event Tracing for Windows (ETW) infrastructure including MPI events for easier trouble shooting. In addition, Microsoft Visual Studio 2010 includes a MPI Cluster Debugger. Developers can automatically connect to MPI processes on each node and individually pause and examine program variables on each node.
F5 experience so that MS-MPI applications can be deployed and debugged on the cluster via the usual F5 keystroke in Visual Studio 2010.
Network diagnostic tests to detect networks errors and issues.
Download the Windows HPC Server 2008 R2 Technical Overview whitepaper »
Security
Security
Windows HPC Server integrates with Active Directory to enable role-based security and single sign-on for all cluster access. The scheduler runs each job under the context and credentials of the submitting user.
All credentials are encrypted and stored with the job until the completion of the job.
Integrated, bidirectional Windows Firewall for enterprise-facing networks.
Integrated patch management provides assurance that patches won’t interfere with running jobs.
All job management related-related communication takes place over encrypted and authenticated channels.
Download the Windows HPC Server 2008 R2 Technical Overview whitepaper » Learn more: Security in Windows HPC Server 2008 R2 »