We’re on a mission to transform our human resources systems here at Microsoft. To make it happen, we’re upgrading the way we use analytics and AI.
Our digital transformation has been a twofold journey.
First, we upgraded our core processes, providing efficient and effective self-service portals for our employees and powerful tools for our HR team using SAP SuccessFactors. Those processes include the nuts-and-bolts applications associated with human capital management (HCM): the employee portal, rewards, payroll, and other essential HR functions.
With the core processes in place, our Microsoft Digital Employee Experience (MDEE) team had everything they needed to revolutionize the data at the center of HR.
The architecture we chose? Microsoft Azure Data Lake.
[Explore all the ways that AI is driving Microsoft’s digital transformation. Learn how Microsoft is creating the digital workplace.]
Building a modernized HR data estate
When data is scattered across disparate systems, it’s difficult to provide agility, insights, and advanced analytics through AI. In today’s world of big data and predictive intelligence, these capabilities aren’t just a luxury. They drive talent conversations, workforce planning, and an improved employee experience that affects business outcomes.
But when an enterprise’s data is siloed or fragmented, those outcomes are out of reach.
“What happens when you don’t have a modern data architecture?” asks Harsh Raj Singh Thakur, principal software engineering manager on the MDEE HR Data and Insights team. “You have a tedious and drawn-out process before you can retrieve your metrics. It’s a cumbersome task, it’s expensive, it’s not easy to maintain, and there’s a lot of cost to get it all done.”
To make HR insights more accessible and insightful, MDEE first had to assemble a unified and accessible data estate. Our SAP SuccessFactors implementation for core HR processes helped lay the groundwork by streamlining external and operational data to make them more organized and available for processing.
With modern core processes in place, MDEE engineers could turn their attention to data.
The journey to data transformation
Like all large-scale transformations, this one involved a great deal of complexity and multiple touchpoints. Microsoft Azure Data Lake provided the modern analytics platform that would not only enable the team to ingest, store, transform, and analyze the data, but also deliver simpler data discoverability, maintain data security, and ensure compliance.
Unifying the data
Considering the wide array of HR systems at Microsoft, it was important to bring all the data together to give HR an end-to-end view of the employee lifecycle and the moments that matter in an employees’ journey. At the same time, the team took efforts to reduce redundant data copies across the enterprise.
The ease of use from actually having everything collocated in an Azure Data Lake makes it easy to build out connected insights. It’s the foundation of our modernization journey.
—Harsh Raj Singh Thakur, principal software engineering manager, Microsoft Digital Employee Experience HR Data and Insights
“Enabling connected insights which are trusted and secure through a modern data platform in Azure Cloud was a key goal as we set out to drive the digital data transformation in the HR ecosystem,” says Johnson Samuel, principal group engineering manager for MDEE’s HR Data and Insights team.
Multiple systems make up the HR ecosystem: Employee Central for core HR, iCIMS for applicant tracking, listening systems, rewards, CRM, employee learning, and more. While each of these systems serves an important purpose, the potential to unlock insights by unifying all of their data is immense.
“The ease of use from actually having everything collocated in an Azure Data Lake makes it easy to build out connected insights.” Raj Singh Thakur says. “It’s the foundation of our modernization journey.”
Azure Data Lake Storage Gen2 serves as the common storage layer, which ingests data through Azure Data Factory, messaging systems, and other sources. By properly defining storage structures and models, the team had made the first step toward a more modern data platform.
Expanding the data footprint with new metrics and scorecards
Ever-increasing volumes of data illustrated the need for advanced analytics. They were no longer a choice—they were a necessity.
“There are many lines of businesses within HR, like Global Talent Acquisition, Talent and Learning, and HR Services who manage HR operations,” Samuel says. “We’ve enabled new capabilities for each of these different HR functions.”
Key metrics across the ecosystem include the recruiting funnel, workforce, headcount, employee engagement, learning and development, and other functions across HR. The analytics apparatus uses a combination of Azure Synapse Analytics, Azure Analysis Services, and Power BI Shared Datasets, while Microsoft Power BI is responsible for visualization.
This powerful combination of technologies helped build complex analytics and drove consistency across teams. It also unlocked the ability to bring disparate metrics together to help determine correlation and causation between different factors.
Data governance
Next, the team needed to ensure that engineers and end users could access data in the lake safely and securely. Good governance keeps data access compliant because users can only request information that’s relevant to their roles. Driven by the HR Privacy team and enabled by a home-grown security and governance platform, MDEE established column-level security (CLS) on the Data Lake.
“When an HR team requests data, they get access to only the specific data set,” Raj Singh Thakur says. “So if you’re looking for an employee’s name and alias but your role doesn’t require you to know their salary, gender, or other aspects of their identity, you won’t get access.”
This approach makes sure we respect our employees’ privacy and that we comply with local laws that regulate how we use our data. Data governance also includes data discoverability, quality, and lineage functionality, which the team established through Microsoft Purview and in-house solutions to support more complex scenarios.
Modern engineering
MDEE also developed key platform capabilities that ensure high-quality and trustworthy data across the estate and drive engineering efficiency.
Whether the metric is headcount, performance management, employee learning, or any other area, each of them follows the architectural pattern of a Data Lakehouse, a system where all information resides in the Data Lake, without the need to build separate data marts. It allows our engineers to scale storage and compute independently for greater efficiency.
Between telemetry dashboards that help engineers understand system health and continuous optimization across code and infrastructure, this new architecture has helped save significant Azure costs—a reduction of around 50% over 2 years. Meanwhile, enabling agile development and DevOps is helping the team deliver iteratively and realize business value faster.
But the real value lies in the insights that unified, normalized data empowers.
“We’ve normalized the data by leveraging a company-wide taxonomy that we can use across other projects very easily,” says Mithun Manganahalli Goud, principal software engineer on MDEE’s HR Data and Insights team. “So from a data-delivery service standpoint, we can provide information to a wide range of downstream systems and data consumers.”
Building a platform for the future
While the new architecture is actively meeting current reporting needs, MDEE also looked toward the future.
We’ve created a rich content system where we can manage emerging requirements with the current data and metadata, so it’s future-ready. We already have the process in place, so we won’t have to go back and reinvent the wheel.
—Mithun Manganahalli Goud, principal software engineer, Microsoft Digital Employee Experience Data and People Analytics
The platform is capable of enabling deep insights that leverage machine learning. While today’s focus is on descriptive and diagnostic functions, the team is working toward predictive and prescriptive analytics through AI and machine learning.
“We’ve created a rich content system where we can manage emerging requirements with the current data and metadata, so it’s future-ready,” Manganahalli Goud says. “We already have the process in place, so we won’t have to go back and reinvent the wheel.”
When our HR team takes the next step into AI-driven insights, the foundations will already be in place.
Driving human-centered innovation with Microsoft Azure Data Lake
Our modernized data architecture has enhanced the HR teams’ capabilities. Better data immediacy means data pulls that used to take 24 hours now get done in a fraction of the time—around four to six hours. Similarly, the time it takes to enable self-service access for bring-your-own-compute data processing is rapidly falling.
One of the most unique and forward-thinking outcomes is that we’ve been able to combine qualitative with quantitative data. We’re able to create data models with our survey information as well as more quantitative data like attrition and diversity, then combine them in an aggregated, de-identified way to understand broad insights.
—Dawn Klinghoffer, vice president, People Analytics
But the most powerful outcomes are the cross-category, cross-disciplinary insights that unified and accessible data provides for HR leaders.
“One of the most unique and forward-thinking outcomes is that we’ve been able to combine qualitative with quantitative data,” says Dawn Klinghoffer, vice president of People Analytics at Microsoft. “We’re able to create data models with our survey information as well as more quantitative data like attrition and diversity, then combine them in an aggregated, de-identified way to understand broad insights.”
The more people interact with the data, the more it will lead to deeper questions and better insights to drive their business or Microsoft as a whole.
—Patrice Pelland, partner group engineering director, Microsoft Digital Employee Experience
For example, by combining sentiment data with de-identified calendar and email metadata, we’ve been able to quantify the impact of blocking focus time on employees’ perception of work-life balance.
“Making data available to all people in a self-service, consumable way gives them the opportunity to ask the questions they don’t even know they have,” says Patrice Pelland, partner group engineering director for MDEE. “The more people interact with the data, the more it will lead to deeper questions and better insights to drive their business or Microsoft as a whole.”
Those questions and insights have already led to human-centered improvements and innovations. One example is the wide adoption of team agreements that empower employees to collectively self-determine the work modes that serve them best. HR’s work has even informed some of the “nudge” product features for employee experience tools like Microsoft Viva, for instance, recommending focus blocks to improve productivity and overall work-life balance—a metric that’s currently on the rise across Microsoft.
Ultimately, the more people who have access to high-quality, trustworthy data, the more we can provide a world-class experience for all employees.
“There’s a lot of envisioning based on the services that we’ve been building that people didn’t even think could exist,” Pelland says. “We’re building the foundational layers to offer things that will be truly transformational for the HR business. Whatever size your organization is, and whichever HCM you use, with Azure, you can do what we’re doing right now.”
- The gold standard should be unity between transactional tools and data tools.
- Start from an understanding that it’s about people and ground your work in that.
- Think big but think holistically; start with a goal and work toward it iteratively.
- Consider the experiences that will delight your end users.
- Start from how you’re going to use the data, then work backward.
- Collaborate early and often. Otherwise, preconceived notions can creep in.
- Explore all the ways that AI is driving Microsoft’s digital transformation.
- Learn how Microsoft is creating the digital workplace.
Want more information? Email us and include a link to this story and we’ll get back to you.
Tags: AI and Machine Learning, developer tools, digital transformation, Microsoft Azure
