Microsoft Security AI RFP

Microsoft Security AI RFP

About

Update, June 5, 2020 – The recipients for the 2020 Microsoft Security AI proposal have been announced.

Microsoft is committed to pushing the boundaries of technology to empower every person and every organization on the planet to achieve more. The cornerstone of how Microsoft does this is by building systems that are secure, and by providing tools that enable customers to manage security, legal, and regulatory standards.

The goal of this Request for proposals (RFP) is to spark new AI research that will expand our understanding of the enterprise, the threat landscape, and how to secure our customer’s assets in the face of increasingly sophisticated attacks.

Security is rapidly gaining importance in an ever-growing digital world with expanding heterogeneous systems, an explosion of data, and increasingly motivated and sophisticated adversaries. The asymmetric nature of security adds to this challenge. The defender must protect all assets, yet the attacker need only find one vulnerability. At the same time, availability of data and cloud capabilities create an opportunity for defenders to flip the balance in their favor. AI will enable us to increase awareness and actionable insights, and make our customers more agile than their adversaries when defending their enterprises.

Microsoft is launching a preliminary academic grants program. We will fund one or more projects (up to $300K in total funding for this RFP) in new collaborative research efforts with university partners so that we can invent the future of security together.

Goal

Research is an integral part of the innovation loop. Most of the exciting research is happening in universities around the world. The goal of the Microsoft Security AI (MSecAI) RFP is to develop new knowledge and capabilities that can provide a robust defense against future attacks. Through our grants program, we hope not only to support academic research, but also to develop long-term collaborations with researchers around the world who share the same goal of protecting private data from unauthorized access.

Proposals are invited on all areas of computing related to security and AI, particularly in the following areas of interest.

Understanding the enterprise

  • An enterprise is a collection of entities (users, computers, files, documents, processes), relationships between entities, and behaviors over time. Assuming data representing an enterprise exists, what are the privacy risks and possible mitigation strategies upon sharing that data with third parties? Assuming data cannot be shared, how could a realistic representation of an enterprise be created to enable subsequent learning tasks?
  • We seek approaches to privacy preservation of enterprise data that empower reasoning on this data, while at the same time providing privacy guarantees.
  • We desire to understand new technologies as they emerge. Internet of Things and supply chains all form elements of the modern and growing ecosystem. We want to develop ways to discover and automatically understand these new technologies and the data they produce. What would be the relevant data to bring into scope?
  • Automatic modeling to provide new insights is critical. How can we empower enterprises with less or no experience to bring powerful AI to bear on further understanding the ecosystem?

Trustworthy machine learning for industry

The aim is for researchers to collaborate with software developers, machine learning (ML) engineers, and security engineers/incident responders to conduct joint research in the design, development, and deployment of secure ML systems .

  • The reliability of machine learning systems in the presence of active adversaries has become especially important in recent years. As ML is used for more security-sensitive applications, and is trained with larger amounts of data, the ability for learning algorithms to tolerate worst-case noise is critical. How can we identify the risk to the confidentiality, integrity, and availability of ML models? Can we develop offline and online analysis-based tools to test ML failure modes against adversarial attacks, such as backdoor data poisoning, model inversion, perturbations, and many others, as described in the taxonomy? How can these test harnesses help us validate the trustworthiness and reliability of ML models? How can we design and train ML models to be robust against such attacks?
  • As ML models are trained and deployed frequently to capture the latest data insights, how can we effectively perform input validations at scale to our data in order to identify and reject specially crafted adversarial queries, both during training and inference? How can we detect whether an attacker is injecting synthetic traffic to influence the model’s decision boundaries? How can we attribute changes in data distributions to adversaries and not to other factors involved with a production ML pipeline?
  • Once a model compromise is detected, what should the data and model provenance look like? Can we effectively “patch” the compromised ML model/dataset or rollback to a previously known good model without compromising the existing model performance?
  • Given the black-box nature of ML systems, how do we meaningfully interrogate ML systems under attack to ascertain the root cause of failure? How do we ascertain the blast radius, and threat attribution? What steps can an incident responder perform to respond to such threats?
  • Models that are deployed on client machines are highly susceptible to model stealing and tampering attacks. How can we validate the authenticity and integrity of ML models and protect them against such adversarial tampering? What sort of guidelines can an ML engineer follow when designing and deploying such models?
  • For additional research areas, please refer to this paper.

Understanding the threat landscape

  • Can we devise methods for analysts to understand how an AI-based anomaly-, intrusion-, or malware-detection system came to its conclusions? If users are blocked from performing particular actions, AI must offer compelling reasons for its decisions, particularly if this interferes with productivity. On the other hand, some contexts involve irreducible computation that might not be amenable to simpler explanations. In such situations, how do we instill confidence in a decision being justified?
  • Attackers are continuously innovating, and new techniques and campaigns are detected after they are hypothesized or observed in the wild. How can this cycle be broken to discover techniques and campaigns before they’re used? Can we build AI-powered defensive and offensive agents that can stay ahead of adversary innovation?
  • Open Source Intelligence (OSINT) combined with proprietary telemetry serves as the basis of threat intelligence. Hunting followed by threat intelligence is the process analysts use to gain insights into tactics, techniques, and procedures (TTPs), assess enterprise risk, and prioritize defensive measures. How can this be done in real-time in a more scalable manner?

AI in supporting defenders

  • How can AI-supported security decisions be effectively balanced with minimal impact on productivity? This applies to both end users and security operation teams.
  • How can we use dynamic quantification of risk to create visibility for our customers into areas of potential concern, and how can we proactively reduce and mitigate risk as driven by AI predictions?
  • How can we use human-in-the-loop AI to enable user feedback through advanced user experiences to update defenses automatically? How do we avoid bias in this process to ensure human input does not preclude the discovery of new malicious behavior? A balance between exploration and exploitation must be achieved.
  • How can AI be used to increase the efficacy and agility of threat hunter teams? How do we present high quality actionable insight from AI to humans with an emphasis on recall (finding the breach) rather than precision? The focus should be on behaviors most likely to be related to attacks.
  • Given current exponential progress in technology, what major disruptions might shake up the threat landscape, defensive arsenal, or both? During the march towards Artificial General Intelligence (AGI), intermediate discoveries and advancements may well provide both attackers and defenders with game-changing techniques. Quantum computing may offer new capabilities.

Microsoft funding

Microsoft will provide up to $150,000 USD of funding for each approved proposal (maximum funding for this RFP, $300,000 USD). Microsoft will also consider an additional award of Azure cloud computing credits if warranted by the research and specified in the proposal. The selected winning team of this RFP shall receive funding in the form of an unrestricted gift. A second round of funding pending initial progress and outcomes (see Timeline below) may be considered at some point during this collaboration. All funding decisions will be at the sole discretion of Microsoft. Proposals for this RFP should provide an initial budget and workplan for the research based on the Timeline section below.

Microsoft encourages potential university partners to consider using resources outlined in the RFP in the following manner:

  • PhD scholarship stipends.
  • Post-doctoral researcher funding.
  • Software and hardware research engineer funding.
  • Limited but essential hardware and software needed to conduct the research.

Proposal plans should include any of these, or other items, that directly support the proposed research.

Microsoft research collaborators, at no cost to the winning teams, may visit the university partners one or more times to foster collaborative planning and research. These visits will be agreed upon and scheduled after an award decision is made. Likewise, a cadence of meetings will be mutually agreed upon at the start of the collaboration. Proposals are welcome to include other suggestions about how to foster an effective collaborative research engagement.

Eligibility

This RFP is not restricted to any one discipline or tailored to any methodology. Universities are welcome to submit cross-disciplinary proposals if that contributes to answering the proposed research question(s).

To be eligible for this RFP, your institution and proposal must meet the following requirements:

  • Institutions must have access to the knowledge, resources, and skills necessary to carry out the proposed research.
  • Institutions must be either an accredited or otherwise degree-granting university with non-profit status, or a research organization with non-profit status.
  • Proposals that are incomplete or request funds more than the maximum award will be excluded from the selection process.
  • The proposal budget must reflect your university’s policies toward receiving unrestricted gifts, and should emphasize allocation of funds toward completing the research proposed.

Additionally:

  • Proposals should include a timeline (approximately 12-18 months) or workplan that begins in summer 2020 and ends in fall of 2021.
  • To optimize the chances of receiving an award, we encourage researchers from the same university to consider submitting a single, joint proposal (rather than multiple individual proposals) that leverages their various skills and interests to create the strongest possible proposal.
  • Multiple universities can submit a joint/single proposal together. Please clearly indicate in the budget section how the budget, not to exceed $150,000 USD, will be shared.

Selection process and criteria

All proposals received by the submission deadline and in compliance with the eligibility criteria will be evaluated by a panel of subject-matter experts chosen by Microsoft. Drawing from evaluations by the review panel, Microsoft will select which proposals will receive the awards. Microsoft reserves the right to fund the winning proposal at an amount greater or lower than the amount requested, up to the stated maximum amount. Note: Microsoft will not provide individual feedback on proposals that are not funded.

All proposals will be evaluated based on the following criteria:

  • Addresses an important research area identified above that, if answered, has the potential to have a significant impact on that domain.
  • Expected value and potential impact of the research on relevant information security fields.
  • Potential for wide dissemination and use of knowledge, including specific plans for scholarly publications, public presentations, and white papers.
  • Ability to complete the project based upon adequate available resources, reasonable timelines, and the identified contributors’ qualifications.
  • Qualifications of the research team, including previous history of work in the area, successful completion of previous projects, research or teaching awards, and scholarly publications.
  • Diversity is highly valued and research teams should strive to reflect a diversity of backgrounds, experiences, and talent reflected in the research teams.
  • Evidence of university support contributed in-kind to directly support and supplement the research efforts.
  • Budget is strategic to maximize impact of research.
  • Possible additional information as requested by the review panel, which might be requested via a conference call.

Timeline

  • May 1, 2020: Proposals due.
  • May 31, 2020: Winners announced.
  • Summer 2020: Awards made, and planning begins with regularly scheduled meetings, calls, and visit(s) by Microsoft to MSecAI winning university.
  • Spring 2021: Review of progress for potential second round of funding (pending progress and availability of funds).
  • Fall 2021: Report back.

Conditions

  • As a condition of accepting an award, principal investigators agree that Microsoft may use their name and likeness to publicize their proposals (including all proposal content except detailed budget information) in connection with the promotion of the research awards in all media now known or later developed.
  • Researchers will be willing to engage with Microsoft about their project and experience, and provide updates via monthly or quarterly calls.
  • The review process is internal, and no review feedback will be given to submitters.
  • Microsoft encourages researchers to publish their work in scholarly venues such as journals and conferences. Researchers must provide Microsoft a copy of any work prior to publication. So long as accurate, such publications are not subject to Microsoft’s approval except that, at Microsoft’s request, researcher will delete any Microsoft Confidential Information identified or delay publication to enable Microsoft to file for appropriate intellectual property (IP) protection for any project IP disclosed in such work.
  • All data sets and any new IP resulting from this effort will be made public and publicly available for any researcher, developer, or interested party to access to help further the goals of this initiative.
  • Funded researchers must seek approval of their institution’s review board for any work that involves human subjects.
  • At the completion of the project, the funded researchers will be required to submit to Microsoft a report describing project learnings.
  • Any security issues in Microsoft products or services discovered during this research must be reported to the Microsoft Security Response Center.

Proposal Requirements

Proposals must be submitted no later than 5:00 PM PT, May 1, 2020. Questions should be sent to MSecAI@Microsoft.com and must be received by April 1st in order to allow adequate time for response. Responses to questions will be posted in the RFP’s FAQ section withing 2 business days.

Microsoft shall have no obligation to maintain the confidentiality of any submitted proposals. Therefore, proposals should not contain information that is confidential, proprietary, restricted, or sensitive. Proposals will be evaluated by a panel of subject-matter experts chosen from Microsoft. Microsoft reserves the right to make the winning proposals publicly available, except those portions containing budgetary information.

Length

The proposal should not be more than seven pages in length of Times New Roman 11-point font. Any documentation beyond that length will not be included as part of the proposal review.

The seven-page limit includes the cover page but the proposal can start on the cover page if additional space is needed. Scholarly references/bibliography can be submitted in addition to the seven pages and will not count toward the seven-page limit.

Cover page

The proposal should have a cover page that provides the following information:

  • Biographical information and contact information: This should include a brief description of any relevant prior research, publications, or other professional experience.
    • Faculty with deep technical experience related to the research areas described above are encouraged to apply. Indicate estimated level of effort/amount of time each faculty member will spend on the project.
    • Post-doctoral researchers and mid- to late-stage PhD students with deep technical experience related to the research should be included in proposals. Indicate the estimated level of effort/amount of time each post-doctoral researcher and PhD student will spend on the project.
  • Project proposal abstract: The abstract should contain the following:
    • A nontechnical description of the project that states the problem to be studied and explains the project’s broader significance and importance.
    • A technical description of the project that states the goals and scope of the research, and the methods and approaches to be used.

Proposal body

The proposal body should include the following information.

  • Project description: Include what set of questions based on the identified research scenarios above, will be addressed and how they will be addressed. Describe how answering these questions will help advance the state-of-the-art in security AI research.
  • Approach: Describe the methodological and theoretical approach that the researchers will use. Explain exactly how the researchers will go about answering the question. Describe how the researchers will handle the legal and ethical challenges of doing work in this area. This section should also describe how the university MSecAI team proposes to work with Microsoft counterparts (researchers and engineers) to ensure an effective and positive collaboration.
  • Resources: Proposals should specify if and how Microsoft technologies will be used, namely (1) APIs, (2) Data sets, etc. if applicable.
  • Expected results: Briefly describe what new knowledge is likely to be generated as a result of this research, why these results would be significant, and how this could benefit defenders of tomorrow.
  • Related research: Briefly summarize related research, including references where appropriate.
  • Researcher roles: Describe the role of each researcher involved in the project and explain how their skills and knowledge enable them to address the proposed research.
  • ~12-18-month Timeline/Workplan and Schedule: Describe what milestones will be used to measure progress of the project during the year and when they will be completed. If the project is part of a larger ongoing research program, estimate the time for completion of this project only. It is expected that the award will be made on or after June 15, 2020. Project timelines should reflect starting times on or shortly after this date.
  • Use of funds: Provide a budget (in U.S. dollars) describing how the award will be used. The budget should be presented as a table with the total budget request clearly indicated. Microsoft will consider requests for Azure credits necessary to conduct research. Value of Azure credits will not be considered a part of the budget request. Azure requests should be included in the budget table.
  • Other support: Include other contributions to this project (cash, goods, and services) by your university or other sources, if any, but do not include the use of university/organization facilities that are otherwise provided on an ongoing basis. Describe other grants or funded research that may be leveraged to add value to this research effort. Note: authors of the selected proposal will be required to submit an original letter on their institution’s letterhead, certifying the commitment of any additional or matching support described in the proposal.

 

FAQ

Can multiple universities submit a joint/single proposal?

Yes, multiple universities can submit a joint/single proposal together. Please clearly indicate in the budget section how the budget, not to exceed $150,000 USD, will be shared.

If a proposal is submitted by more than one university, jointly, is it possible for Microsoft to pay each university directly or do we need to subcontract to each other?

Yes, Microsoft will pay each university directly provided the budget clearly illustrates the amount to be paid to each university with a total not to exceed $150,000 USD.

How long can my proposed collaboration with Microsoft last?

Project timelines should be approximately 12-18 months. They should reflect the total time estimated to complete the research proposed.

Are proposals required to choose one of the research areas described in the RFP?

Yes, proposals must indicate which of the listed research areas will be investigated as part of the proposed research to be eligible for consideration.

Is it a requirement or advisable to have a Microsoft champion who supports our proposal?

It would be considered a positive for the proposal to have a researcher in Microsoft who is supportive but we don’t require it or expect it. If a researcher in Microsoft is interested in expressing support for your proposal, they should send an email of support to MSecAI@Microsoft.com with the university PI(s) on cc when the proposal is submitted.

Can proposal budget requests be less than $150,000 USD?

Yes, proposal budget requests can be of any amount up to $150,000 USD.

Does the budget table specified in the Proposal Requirements section count toward the seven-page limit?

The budget is part of the seven-page limit. Scholarly references/bibliography can be submitted in addition to the seven pages and will not count toward the seven-page limit but all of the other required components will count toward the seven-page limit.

If we are to include a letter of support from our university, would this count towards the seven-page limit?

No, letters of support will not count toward the seven-page limit.

Is it an issue if our cover page is more than one page if our proposal is still within the seven-page limit?

As long as the full proposal doesn’t exceed seven pages the rest of the section lengths are flexible.

The Selection Process and Criteria identifies “Evidence of university support contributed in-kind to directly support and supplement the research efforts”. Is Microsoft looking for cost-share commitments, and if so, is the cost-share considered mandatory or voluntary per the terms of the award?

We would be looking for cost-share. This is not a mandatory requirement.

Will Microsoft consider indirect costs (since they are not allowed) evidence of university support?

We would be looking for contributions that directly support the research efforts here so indirect-costs that cover items such as facilities and infrastructure would not count toward university support/cost-share/in-kind contribution.

Is there a percentage or dollar amount that is expected or required as evidence of university support?

Since this is not a requirement, there is no expected amount.

Is the money considered a 'gift'? Are there conditions put on the funds?

The funds will be considered a gift that has no restrictions on how it is used. Budgets should reflect university’s own policies for accepting unrestricted gifts

Can the grant money can be used by the receiving institutions freely, e.g. to pay an expert postdoc affiliated to one of the participating universities, however, residing in a different country during the project?

There are no restrictions on how the funds are used. We do request that how the funds will be used is clearly illustrated in the required budget portion of the proposal.

Can funds be used to cover costs for Master’s students?

There are no restrictions on how the funds are used. We do request that how the funds will be used is clearly illustrated in the required budget portion of the proposal.

Are overhead and indirect costs allowable in the budget?

The proposal budget should reflect your university’s policies toward receiving unrestricted gifts and should emphasize allocation of funds toward completing the research proposed.

Is it possible to budget for some of the PI's time as part of the Microsoft Security AI Research Award?

As unrestricted gifts, it will be entirely up to the winners to decide how to spend the award to achieve the research goals in the proposal.

To further improve and facilitate our research, would it be possible to access Microsoft's internal corpus of enterprise and/or security relevant data ?

We will not be able to provide access to any data that is not already publicly available.

Can the data and the results of the project be used for future research by the authors, as it is common in the context of commercial research grants?

Yes, the results of this research are meant to be open and public for unrestricted use by future researchers and technologists.

We plan to have one mid-stage PhD student and three professors in the proposal. Is it advisable to have an additional professor, PhD student, or postdoc in our university MSecAI team?

You are encouraged to assemble a team that is most likely to achieve the greatest results within the time and budget parameters required.

How implementation-centered should the planned research be? Is it also valuable for Microsoft to i) receive insights on how people work together using their current technology leading to implications for the design of their future tools or should be ii) more focus on creating tool prototypes, per se?

Both of these scenarios are valuable. The results of this research will be open and public and so they are meant to drive future research and technology development. More insight on how people work together leading to implications for designs of future tools – though not designed just by Microsoft but others as well that are working in these topic areas would be of interest.

However, if you feel you can develop breakthrough prototypes that also inform future research then that would also be interesting.

Recipients

2020 Microsoft Security Research AI RFP Winners

Dawn Song and Peng Gao from UC BerkleyDawn Song and Peng Gao

University of California, Berkeley

Microsoft lead collaborator: M365 Security + Compliance Research

Title: A Security Knowledge Graph for Automated Threat Intelligence Gathering and Management

Abstract: Sophisticated cyber-attacks have plagued many high-profile businesses. To gain visibility into the fast-evolving threat landscape, open-source Cyber Threat Intelligence (OSCTI) has received growing attention from the community. Commonly, knowledge about a threat is presented in a vast number of OSCTI reports, detailing how the threat unfolds into multiple steps. Despite the pressing need for high-quality OSCTI, existing approaches, however, have primarily operated on fragmented threat indicators (e.g., Indicators of Compromise). On the other hand, descriptive relationships between threat indicators have been overlooked, which contain essential information on the threat behaviors that is critical to uncovering the complete threat scenario. Recognizing the limitation, this proposal seeks to design and develop an intelligent and scalable system for automated threat intelligence gathering and management. The proposed system will use a combination of AI-based methods to collect heterogeneous OSCTI data from various sources, extract comprehensive knowledge about threat behaviors in the form of security-related entities and their relations, construct a security knowledge graph from the extracted information, and update the knowledge graph by continuously learning from its deployment. We will also pursue possible security defensive applications that can be further empowered by OSCTI. The proposed work has a broad impact for advancing the state-of-the-art in threat intelligence gathering, management, and applications.


 

Nick Heard, Department of Mathematics, Imperial College LondonNick Heard

Department of Mathematics, Imperial College London

Microsoft lead collaborator: M365 Security + Compliance Research

Title: Understanding the enterprise: Host-based event prediction for automatic defence in cyber-security

Abstract: The next generation of cyber-security challenges will demonstrate an increase in complexity and sophistication, aided by artificial intelligence. To counter this AI-driven threat, we propose to develop Bayesian statistical methodologies for adaptively designing robust, interpretable mathematical models of normal behaviour in new environments. These methodologies will provide new insights into enterprise systems, providing detailed under-standing of network assets and their relationships. These insights will inform enterprise risk-based assessments and enhance the detection and response to cyber threats. Challenges will include the fusion of diverse data sources, collected both within the network environment and externally, and securely sharing intelligence obtained from other platforms. To address these challenges, the proposed workflows will construct modelling frameworks for adaptively building probability distributions for predicting the future activity of a network host. Perspectives in both discrete time and continuous time, along with hybrids of the two, will be considered. Central to the model-building challenge will be developing principled methods for automatically identifying the quantity (either in terms of counts, or in time horizons) of historical data which should be conditioned upon in forming short-term and longer-term predictions. The principal modelling paradigm will be centered on a host-based approach, which has both the capacity to scale and be most sensitive to the protection of sensitive data. Additionally, there will be important scope for making inferences about large-scale network structure, to inform these host-based AI technologies about the position, importance and likely connectivity of the node within the network.


 

Nicolas Papernot, University of Toronto, Department of Electrical and Computer EngineeringNicolas Papernot

University of Toronto, Department of Electrical and Computer Engineering

Microsoft lead collaborator: Azure Trustworthy Machine Learning + Microsoft Security Response Center (MSRC)

Title: Towards Machine Learning Governance

Abstract: The predictions of machine learning (ML) systems often appear fragile, with no hint as to the reasoning behind them—and may be dangerously wrong. This is unacceptable: society must be able to trust and hold to account ML. This proposal seeks to empower ML developers and engineers to develop and design ML systems that are secure and provide the tools that enable its users to manage security, legal, and regulatory standards. Our efforts achieve this through the development of machine learning governance. We focus our efforts around two attack vectors: (1) input manipulations at training and test time that target the ML system’s integrity and (2) model inversion and extraction that target the privacy of training data and the confidentiality of model architectural details. We propose to tackle the first attack vector through the development of robust model uncertainty estimates, the identification of coresets in ML, and the creation of computationally efficient influence metrics. We approach the second attack vector by focusing on the life of ML systems after they have been trained: we will pursue model watermarking, machine unlearning, and the identifiability of ML outputs.