{"id":670932,"date":"2020-07-06T11:15:30","date_gmt":"2020-07-06T18:15:30","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=670932"},"modified":"2024-02-14T15:36:55","modified_gmt":"2024-02-14T23:36:55","slug":"toward-trusted-sensing-for-the-cloud-introducing-project-freta","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/toward-trusted-sensing-for-the-cloud-introducing-project-freta\/","title":{"rendered":"Toward trusted sensing for the cloud: Introducing Project Freta"},"content":{"rendered":"

\"Representative<\/p>\n

Editor\u2019s note, Feb. 14, 2024 \u2013 <\/strong>The Project Freta analysis web portal is no longer publicly accessible. Please contact <\/i>project-freta@microsoft.com<\/i><\/a>.<\/i><\/span><\/p>\n

\n

\u201cSunlight is said to be the best of disinfectants.\u201d<\/em><\/h4>\n

\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u2015Louis D. Brandeis, 1914<\/h4>\n<\/blockquote>\n

We often think about the field of computer security as a field of walls and barriers that keep intruders out. With Project Freta, we invite readers to think not of walls but of sunlight. When attackers build malware that cannot be detected, they gain enormous economic value. Undetected malware can be continuously re-used: it is never part of attack reporting, never summons incident responders, and never alerts the victim to a data theft event. The economics of reuse can justify enormous attacker investment in malware non-discoverability. Conversely, once a malware strain is discovered, its value plummets in tandem with its reusability. In this stealth economy, that which hides in darkness is removed with sunlight. The question for defenders, then, is how can we raise the cost of non-discovery? Is there a point beyond which a class of malware is no longer economically viable?<\/p>\n\n\n\n\n\n\n
Quick Start<\/strong><\/span><\/td>\n<\/tr>\n
Project Freta:<\/strong> free service from Microsoft Research for detecting evidence of OS and sensor sabotage, such as rootkits and advanced malware, in memory snapshots of live Linux systems<\/td>\n<\/tr>\n
Access Project Freta Portal (opens in new tab)<\/span><\/a>
\n(connect with any AAD or Microsoft Account)<\/td>\n<\/tr>\n
Documentation (opens in new tab)<\/span><\/a> | Questions or Feedback?<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Incubated at Microsoft Research, Project Freta is a roadmap toward trusted sensing for the cloud that can allow enterprises to engage in regular, complete discovery sweeps for undetected malware. The project\u2019s namesake, Warsaw\u2019s Freta Street, was the birthplace (opens in new tab)<\/span><\/a> of Marie Curie, a pioneer of battlefield imaging (opens in new tab)<\/span><\/a>. While snapshot-based memory forensics is a field now in its second decade, no commercial cloud has yet provided customers the ability to perform full memory audits of thousands of virtual machines (VMs) without intrusive capture mechanisms and a priori<\/em> forensic readiness. Just as yesteryear\u2019s film cameras and today\u2019s smartphones have similar megapixels but vastly different ease of use and availability, Project Freta intends to automate<\/em> and democratize<\/em> VM forensics to a point where every user and every enterprise can sweep volatile memory for unknown malware with the push of a button\u2014no setup required.<\/p>\n

The goal of this democratization effort is to increase the development cost of undiscoverable cloud malware toward its theoretical maximum. What would happen if a commercial cloud could guarantee<\/em> the capture of malware, no matter how expensive or exotic, in volatile memory? Producers of stealthy malware would then be locked into an expensive cycle of complete re-invention, rendering such a cloud an unsuitable place for cyberattacks. This is the future we wish to realize.<\/p>\n

To this end, we propose four properties of trusted sensing to maximize malware discovery and present our technical work along this roadmap to date. As a technology demonstration, Project Freta is opening public access to an analysis portal capable of automatically fingerprinting and auditing a memory snapshot of most cloud-based Linux VMs; over 4,000 kernel versions are supported automatically. Hyper-V checkpoint files captured from a modern enterprise can be searched for everything from cryptominers to advanced kernel rootkits. This prototype previews an exciting future option for cloud consumers: transitioning from boutique forensic consulting services to automated malware discovery built into the bedrock of a commercial cloud.<\/p>\n

Unbiased data from armored sensors<\/h3>\n

In computer security we strive to be evidence-driven. Unfortunately, using reports of cyberattacks to guide defensive approaches is subject to a well-known yet powerful survivor bias (opens in new tab)<\/span><\/a> that can distort the importance of data and lead us astray.<\/p>\n

Abraham Wald\u2019s work during WWII provided a famously repeated (opens in new tab)<\/span><\/a> example of survivor bias in a dataset: reports of bullet holes in airplanes. Rather than recommend armor where bullet holes were reported, Wald recommended armor only in areas where there were no reports of bullet holes. This was due to a critical insight about the nature of bullet hole reporting: bullet holes were only counted when an airplane arrived home.<\/em> Successful attacks removed the attack from the dataset, hence successful attacks could be described only by their absence from the dataset. No reports of bullet attacks on the engines? Armor the engines.<\/em><\/p>\n

\"outline<\/p>\n

Today\u2019s computer security sensors suffer from this same bias. When attackers obtain a model of our sensors and design a way to evade these sensors, we receive no reports of cyberattacks. It is tempting to look at areas in which we receive few or no attack reports and think: \u201cI\u2019ve received no reports of successful attacks on my attack reporting capability!\u201d<\/em> This statement is not as reassuring as it sounds. While we are flooded with billions of reports of malware a year, it is important to understand that synthesizing or mutating existing malware is a nearly cost-free exercise (opens in new tab)<\/span><\/a>.<\/p>\n

Meanwhile, attackers that manage to strike directly at a security sensor, security agent, or privileged operating system component can disable one step in the attack reporting pipeline and vanish without a trace, leaving only an absence in our attack reporting. The lowest-resourced attackers dominate our datasets; the highly resourced live outside our datasets forever. We must invert this data.<\/em><\/p>\n

Project Freta was designed and built with survivor bias at its core. It is a security project designed from first principles to drive the cost of sensor evasion as high as possible and in many cases render evasion technically infeasible. To achieve this, in July of 2018 we started with a clean slate.<\/p>\n

A Greenfield Mandate<\/h3>\n

The fact that sensor evasion is possible at all is surprising to many outsiders to the field of computer security. Anyone who\u2019s visited a retail store understands that it\u2019s a good idea to put the security cameras out of reach\u2014why haven\u2019t we done something similar in the cloud? The answer is a familiar one: backwards compatibility<\/em>. Our first networked computers didn\u2019t have the silicon real estate to devote to an isolated security sensor. Opportunities for technology companies to break backwards compatibility and \u201cgreenfield\u201d redesign with security in mind have appeared only periodically, seen primarily in the mobile and console industries. These redesigns have allowed for increasing hardware separation between the compute plane and the security plane, a detailed topic for platform security journals beyond the scope of this blog post. In today\u2019s cloud, this separation between compute and security planes usually occurs at the hypervisor; tenant workloads are separated from provider workloads via the hypervisor barrier. Is the hypervisor barrier strong enough to prevent sensor evasion? Maybe not.<\/em><\/p>\n

Recently, microarchitectural flaws and forensics research have called some of the properties of the hypervisor barrier into question. A recent forensics research paper, \u201cWho Watches The Watcher? Detecting Hypervisor Introspection from Unprivileged Guests\u201d (opens in new tab)<\/span><\/a> presented at DFRWS in 2018, articulated a method that attackers resident in the compute plane could employ to detect when they are being observed from the security plane\u2014piercing the hypervisor barrier and allowing for pre-emptive self-destruct in order to avoid discovery. This paper came complete with an open-source implementation of the technique. This meant that even Virtual Machine Introspection (opens in new tab)<\/span><\/a> (VMI) endpoint sensors could be evaded with existing open-source software.<\/p>\n

Implementing and democratizing trusted sensing for the cloud meant first articulating the properties of a system that would be immune to these types of attacks:<\/p>\n\n\n\n\n\n\n\n
Project Freta\u2019s four properties of trusted sensing<\/strong><\/td>\n<\/tr>\n
\n

1. Detect. <\/strong>No program can:<\/p>\n

Detect the presence of a sensor prior to installing itself<\/em><\/td>\n<\/tr>\n

\n

2. Hide. <\/strong>No program can:<\/p>\n

Reside in an area out of view of the sensor<\/em><\/td>\n<\/tr>\n

\n

3. Burn. <\/strong>No program can:<\/p>\n

Detect operation of the sensor and erase or modify itself prior to acquisition<\/em><\/td>\n<\/tr>\n

\n

4. Sabotage. <\/strong>No program can:<\/p>\n

Modify the sensor in a way that can prevent the program\u2019s acquisition<\/em><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

To achieve these properties and end the stealthy-malware arms race, incrementally improving existing endpoint technology is insufficient. When attackers and defenders share a microarchitecture, every detection move a defender makes disturbs the environment in a way that is eventually discoverable by an attacker invested in secrecy. The only way to discover such attackers is to remove their insight into defense. This left the question: how much engineering was required to fully automate memory forensics, operate at efficiencies that enabled cloud-scale processing, and still retain the element of surprise?<\/p>\n

Building Project Freta<\/h3>\n

A brief technology evaluation taught the team that starting from scratch was the only viable approach. To leave no place to hide<\/strong>, we needed to accept the huge data footprint imposed by whole-system memory analysis. To address detection<\/strong> and burn<\/strong>, such as \u201cred pill attacks (opens in new tab)<\/span><\/a>,\u201d we needed two components: 1) an offline analysis system that could operate in batch mode and 2) a sensor that could provide whole-system memory captures without executing a single clarifying instruction on the guest. Finally, to mitigate sabotage<\/strong> we needed to ensure our system was built from the ground up to be memory safe.<\/p>\n

For Project Freta, many of these challenges compounded to make \u201cinstant forensics for everyone\u201d a daunting task:<\/p>\n