University of Bath researchers test theories and release findings faster than before with Azure HPC

Improving research and teaching with additional compute nodes and cloud monitoring

At the University of Bath, cutting-edge teaching, research, and innovation come together to inspire the next generation of creative minds and thought leaders. The university wanted to make sure that its academic community of 23,500 students, teachers, and staff could freely pursue knowledge and ideas unhindered by technology, so it expanded its cloud footprint with Microsoft Azure high-performance computing (HPC). In addition to replacing its on-premises resource capacity, the university’s Research Computing team created a dynamic, user-friendly interface that staff and students use to manage their Azure HPC expenses and user accounts.

Named University of the Year in the Sunday Times Good University Guide 2023, the University of Bath has invested heavily in technology to support its world-class academic research program. The university is no stranger to the world of HPC, having invested in supercomputing and hardware for its on-premises environment over the past 15 years. In the university’s search for the perfect cloud platform to support its digital growth strategy, the Research Computing team was attracted to Azure HPC for running large-scale parallel computations with InfiniBand networking. “Now using Azure HPC, we’ve maintained an InfiniBand network, connected our Slurm scheduler, and carried over ways of working that researchers are familiar with,” says Dr. Roger Jardine, Director of Research Computing at the University of Bath.

In 2022, the University of Bath was deep in the planning stages of its Azure HPC rollout. In less than a year, its Research Computing team moved the university from an intermediate supercomputing system in Azure, known as Janus, to Nimbus, its current Linux-based production environment. The university decommissioned its main on-premises cluster and now runs hundreds of workloads and applications with Azure HPC, including computation-heavy chemistry, engineering, and physics research, domain modeling, and software written by researchers themselves.

University of Bath researchers and teachers successfully migrated to their new Azure HPC environments with no downtime and minimal training. Researchers can now test theories and release their findings faster than before with access to thousands of compute nodes in the cloud versus hundreds on-premises. Teachers also can tailor Azure resources to their course curricula almost on the fly. “Azure is like having a very big toolbox—if you have lots of tools, you can build anything with them,” says Dr. Stefano Angioni, Research Computing Manager at the University of Bath.

The university estimates that its community uses 200 different types of software, a stark contrast to other industries such as engineering that might only use five or fewer. It’s a testament to the university’s dynamic and diverse research community. “If you name a software program, we have it, and if we don’t, we know somebody will ask for it soon,” says Dr. Angioni. To support this breadth of resources, the University of Bath wanted not only considerable compute capacity but always-on monitoring. The Research Computing team uses Grafana and its Azure Monitor data source plug-in, along with Azure Virtual Machines insights and network insights in Azure Monitor, to visualize and protect its full environment and correlate issues at the infrastructure level. The university plans to provide access to some dashboards for all of its users in the future. “Azure Monitor is one of our most important services,” says Dr. Angioni. “We use it to monitor our Azure virtual machines, networking, and almost everything else in the system.”

Virtual machines and storage take center stage

Computational needs vary by individual researcher, and the University of Bath appreciates the flexibility to use a range of Azure virtual machines (VMs). The choice of different underlying Intel, NVIDIA, and AMD processors and configurations means that researchers can work in their own ways. From powerful performance and massive scale to optimizations for specific workloads, researchers can quickly access the VMs they need to solve problems and drive innovation. “When a new type of VM like the ND H100 v5 or HBv4-series that’s purpose-built for HPC comes out, we can add it to our environment far faster than with an on-premises resource, and researchers can start using it immediately,” says Dr. Angioni.

All of the university’s VMs are deployed using Azure CycleCloud. Researchers can easily load the software they need into predefined Azure VM images using EasyBuild and get to work. For its storage needs, the university primarily relies on Azure Disk Storage and is looking to offer archival storage with Azure Blob Storage. “Storage in Azure is really important for our users and their research workflows because a lot of research councils have quite strict rules on how long files used for research output are kept, and it can be quite a long time,” explains Dr. Angioni.

A day in the life of the university’s top Azure HPC advocates

As the University of Bath progressed toward the launch of its Nimbus production system, it used its existing Janus environment to test new features with help from a select group of faculty members, including physics professor Daniel Wolverson. Having users like Professor Wolverson try out early-stage Nimbus features and provide feedback in real time allowed the team to identify and avoid potential issues and offer the most value-added features possible to researchers and teachers alike when the production system launched. “I’m a very conventional scientific user, and it’s been seamless moving from our on-premises environment to Janus and then Nimbus,” shares Professor Wolverson.

The university hopes to further expand cloud computing by researchers beyond traditional HPC workloads. For example, the Life Sciences department is eager to understand how it might utilize new high-memory VMs in Azure. “We only had 250 compute nodes on-premises, but Azure offers thousands of nodes, so our researchers and instructors don’t have to wait to spin up their H-series, N-series, and F-series VMs,” notes Dr. Angioni.

Professor Wolverson appreciates that these benefits cascade down to students. “I have funding for a project where undergraduates would take part in the research, and that’s a lot easier now with Azure HPC because of the short waiting time,” he says. “Undergraduates work under very tight deadlines, so if they have a one-hour class or a project due the next day, they need the results to come back in that hour or that day. In our Azure HPC model, we can involve them in the research and offer them fast results with much more confidence.”

Working with Microsoft to speed time to solution

With so many VM types, chip processors, and other resources to choose from, University of Bath researchers rely on guidance from the Bath Research Computing team to get answers to their most difficult technical questions and design personalized Azure HPC environments to fit the needs of any project. As Professor Wolverson describes, “I learned that HC-series VMs are what you want for computational chemistry, so that’s what I’ve used consistently throughout my physics research.”

A dedicated Microsoft team also played a key role in supporting and superseding the Research Computing team’s expectations around time to solution for the Nimbus environment. “While building our production system, we had weekly meetings with our Microsoft team where we could summarize our project status and any issues we faced,” says Dr. Angioni. “Microsoft would bring in people with expertise on architecture, design, and implementation, which helped us get to solution quicker.”

Researching the future potential of HPC and promoting industry collaboration

Migrating to the Azure-based Nimbus has spurred new efficiencies across the university. “Our Azure cloud solutions like Azure Monitor come out of the box, and we don’t have to do anything to have them work for us. With them, we’re increasing productivity, like employees now spending less time on monitoring and management and more time creating new opportunities,” says Dr. Angioni.

In addition to IT staff benefits, some academic staff report feeling more empowered to pursue their academic interests through hands-on research projects. “We had great feedback recently from a lecturer saying they could complete research work using Azure HPC in between their teaching commitments, whereas before they might have had to queue tasks and wouldn’t be sure whether they were going to get results,” says Dr. Jardine. “Now that our lecturers and researchers are more confident in an instant response, they know they can still be productive in those small gaps in their day.”

As it masters the gamut of cloud benefits internally, the University of Bath hopes to help advance the greater academic community by sharing its Azure HPC and cloud services knowledge. “As more universities and academic institutions start using the public cloud for HPC, we’re looking forward to sharing how we’ve tackled particular problems and how our peers might use Azure CycleCloud or other Azure products,” says Dr. Jardine. Adds Dr. Angioni, “Rather than focusing on building, maintaining, and watering a hardware solution, we can help researchers use Azure HPC in the best way possible and then focus on what’s next in research computing and how we can help the academic community at large use that to its fullest potential.”

Find out more about the University of Bath on Twitter, Facebook, and LinkedIn.

#AzureHPCAI