Microsoft Security AI Academic Program is launching an academic grants program. We will fund one or more projects (up to $150K in total funding for this RFP) in new collaborative research efforts with university partners so that we can invent the future of security together.<\/p>\n\n\n\n
Timeline<\/h2>\n\n\n\n\n- April 30, 2021:<\/strong> RFP published.<\/li>\n\n\n\n
- June 6, 2021:<\/strong> Proposals due.<\/li>\n\n\n\n
- June 18, 2021:<\/strong> Winners announced.<\/li>\n\n\n\n
- Summer 2021:<\/strong> Awards made, and planning begins with regularly scheduled meetings, calls, and visit(s) by Microsoft to MSecAI winning university.<\/li>\n\n\n\n
- Spring 2022:<\/strong> Review of progress for potential second round of funding (pending progress and availability of funds).<\/li>\n\n\n\n
- Fall 2022:<\/strong> Report back.<\/li>\n<\/ul>\n\n\n\n
Research Goals<\/h2>\n\n\n\n
Research is an integral part of the innovation loop. Most of the exciting research is happening in universities around the world. The goal of the Microsoft Security AI (MSecAI) RFP is to develop new knowledge and capabilities that can provide a robust defense against future attacks. Through our grants program, we hope not only to support academic research, but also to develop long-term collaborations with researchers around the world who share the same goal of protecting private data from unauthorized access.<\/p>\n\n\n\n
Proposals are invited on all areas of computing related to phish protection and AI, particularly in the following areas of interest:<\/p>\n\n\n\n
Understanding the communication graph<\/h3>\n\n\n\n
A communication graph is a collection of entities including user accounts, applications, websites, shared infrastructure and the relationships between those entities such as emails, P2P messages, login attempts, etc. How do we leverage this dynamic graph at scale to extract key insights while providing privacy guarantees? Can we understand user interaction profiles over time and identify deviations to detect compromised accounts, phish emails from spoofed domains, bulk emails, etc.?<\/p>\n\n\n\n
Understanding the content<\/h3>\n\n\n\n
90% of large enterprise customer breaches start from email that tricks users into revealing sensitive information. Most of these emails leverage some part of psychological manipulation that displays a sense of authority or urgency to take immediate action, threat, opportunity for monetary gain or loss, etc. Assuming clear text email data is available, what are some approaches that help machines understand the high level intention of a given email while providing privacy guarantees? How can we effectively group known phish emails into high level campaigns based on the content topics and exploitation techniques?<\/p>\n\n\n\n
Fairness and accountability for security<\/h3>\n\n\n\n
As ML is used for more security-sensitive applications, the ability for these systems to generalize globally, not be disruptive to end users, especially any specific segment of user population is quite important. How do we define fairness in security and identify related issues when developing AI systems? Can we develop offline and online experimentation tools to test that our ML models are not biased with respect to attributes such as geo locations, language, industry verticals, etc. How do these test cases help us validate the fairness of ML models?<\/p>\n\n\n\n
When it comes to accountability, how can we identify and assign responsibility for a decision made by an AI system? What steps can an incident responder take to respond to the business disruptions caused by misclassifications from AI system? How can we validate that the same misclassifications do not reoccur as ML systems are retrained? In addition, some of the ML systems may work with complex obfuscated data sources that might not generate human understandable explanations. How do we justify the decisions made by AI systems in such cases?<\/p>\n\n\n\n
Verifying the authenticity of modern communication channels<\/h3>\n\n\n\n
While industry phishing attempts are predominantly carried out through email, many of these attacks have migrated to modern communication channels like professional networks, p2p messaging, search and ads. Phish attempts are becoming increasingly convincing to end users with the advancement of techniques like deep fakes for audio and video generation, content morphing, fake replies. How do we leverage AI systems to verify the authenticity of such content? Moreover, how do we differentiate legitimate user accounts from adversarial\/ tester accounts setup to test defense systems or pollute backend telemetry?<\/p>\n\n\n\n
Protecting patient zero<\/h3>\n\n\n\n
Based on this (opens in new tab)<\/span><\/a> paper, an average phishing attack spans 21 hours between the first and last victim and the detection of each attack occurs an average 9 hours after the first victim. This gives attackers a window of opportunity during which most of the damage is done. How do we leverage AI systems to adapt to the adversarial temporal drift and prevent the first victim\/ patient zero from being compromised? How can we use human-in-the-loop AI systems to enable experts to update defenses automatically? How can AI systems be leveraged to identify and learn from discovery of new attack campaigns? How can we augment supervised ML approaches with unlabeled, noisy data to ensure a good feature distribution coverage in training our ML models?<\/p>\n\n\n\nEconomics of phishing<\/h3>\n\n\n\n
Phishing can be seen as an economic problem. Attackers operate like businesses by making investments in campaign inputs to generate returns by selling stolen credentials, using stolen credentials to gain network access, or committing direct fraud. Firms and users invest hundreds of billions of dollars annually in security protection and expect returns on those investments through reduced cyber risk or increased productivity gains. These markets are rich in common economic complications like externalities, asymmetric information, and uncertainty. However, they remain poorly understood. Can we categorize the attacker ecosystem by business model? What are the returns to firms\u2019 security investments? How do security investments impact the attacker ecosystem and vice versa?<\/p>\n\n\n\n\n\n
Microsoft funding<\/h2>\n\n\n\n
Microsoft will fund one or more projects (up to $150K in total funding for this RFP). A second round of funding pending initial progress and outcomes (see Timeline above) may be considered at some point during this collaboration. All funding decisions will be at the sole discretion of Microsoft. Proposals for this RFP should provide an initial budget and workplan for the research based on the Timeline section below.<\/p>\n\n\n\n
Microsoft encourages potential university partners to consider using resources outlined in the RFP in the following manner:<\/p>\n\n\n\n
\n- PhD scholarship stipends.<\/li>\n\n\n\n
- Post-doctoral researcher funding.<\/li>\n\n\n\n
- Software and hardware research engineer funding.<\/li>\n\n\n\n
- Limited but essential hardware and software needed to conduct the research.<\/li>\n<\/ul>\n\n\n\n
Proposal plans should include any of these, or other items, that directly support the proposed research.<\/p>\n\n\n\n
Microsoft research collaborators, at no cost to the winning teams, may visit the university partners one or more times to foster collaborative planning and research. These visits will be agreed upon and scheduled after an award decision is made. Likewise, a cadence of meetings will be mutually agreed upon at the start of the collaboration. Proposals are welcome to include other suggestions about how to foster an effective collaborative research engagement.<\/p>\n\n\n\n\n\n
Eligibility<\/h2>\n\n\n\n
This RFP is not restricted to any one discipline or tailored to any methodology. Universities are welcome to submit cross-disciplinary proposals if that contributes to answering the proposed research question(s).<\/p>\n\n\n\n
To be eligible for this RFP, your institution and proposal must meet the following requirements:<\/p>\n\n\n\n
\n- Institutions must have access to the knowledge, resources, and skills necessary to carry out the proposed research.<\/li>\n\n\n\n
- Institutions must be either an accredited or otherwise degree-granting university with non-profit status, or a research organization with non-profit status.<\/li>\n\n\n\n
- Proposals that are incomplete or request funds more than the maximum award will be excluded from the selection process.<\/li>\n\n\n\n
- The proposal budget must reflect your university\u2019s policies toward receiving unrestricted gifts and should emphasize allocation of funds toward completing the research proposed.<\/li>\n<\/ul>\n\n\n\n
Additionally:<\/p>\n\n\n\n
\n- Proposals should include a timeline (approximately 12-18 months) or workplan that begins in summer 2021 and ends in fall of 2022.<\/li>\n\n\n\n
- To optimize the chances of receiving an award, we encourage researchers from the same university to consider submitting a single, joint proposal (rather than multiple individual proposals) that leverages their various skills and interests to create the strongest possible proposal.<\/li>\n\n\n\n
- Multiple universities can submit a joint\/single proposal together. Please clearly indicate in the budget section how the budget, not to exceed $150K USD, will be shared.<\/li>\n<\/ul>\n\n\n\n\n\n
Selection process and criteria<\/h2>\n\n\n\n
All proposals received by the submission deadline and in compliance with the eligibility criteria will be evaluated by a panel of subject-matter experts chosen by Microsoft. Drawing from evaluations by the review panel, Microsoft will select which proposals will receive the awards. Microsoft reserves the right to fund the winning proposal at an amount greater or lower than the amount requested, up to the stated maximum amount. Note: Microsoft will not provide individual feedback on proposals that are not funded.<\/p>\n\n\n\n
All proposals will be evaluated based on the following criteria:<\/p>\n\n\n\n
\n- Addresses an important research area identified above<\/strong> that, if answered, has the potential to have a significant impact on that domain.<\/li>\n\n\n\n
- Expected value and potential impact<\/strong> of the research on relevant information security fields.<\/li>\n\n\n\n
- Potential for wide dissemination and use of knowledge<\/strong>, including specific plans for scholarly publications, public presentations, and white papers.<\/li>\n\n\n\n
- Ability to complete the project<\/strong> based upon adequate available resources, reasonable timelines, and the identified contributors\u2019 qualifications.<\/li>\n\n\n\n
- Qualifications of the research team<\/strong>, including previous history of work in the area, successful completion of previous projects, research or teaching awards, and scholarly publications.<\/li>\n\n\n\n
- Diversity<\/strong> is highly valued and research teams should strive to reflect a diversity of backgrounds, experiences, and talent reflected in the research teams.<\/li>\n\n\n\n
- Evidence of university support<\/strong> contributed in-kind to directly support and supplement the research efforts.<\/li>\n\n\n\n
- Budget<\/strong> is strategic to maximize impact of research.<\/li>\n\n\n\n
- Possible additional information<\/strong> as requested by the review panel, which might be requested via a conference call.<\/li>\n<\/ul>\n\n\n\n\n\n
Conditions<\/h2>\n\n\n\n\n- As a condition of accepting an award, principal investigators agree that Microsoft may use their name and likeness to publicize their proposals (including all proposal content except detailed budget information) in connection with the promotion of the research awards in all media now known or later developed.<\/li>\n\n\n\n
- Researchers will be willing to engage with Microsoft about their project and experience, and provide updates via monthly or quarterly calls.<\/li>\n\n\n\n
- The review process is internal, and no review feedback will be given to submitters.<\/li>\n\n\n\n
- Microsoft encourages researchers to publish their work in scholarly venues such as journals and conferences. Researchers must provide Microsoft a copy of any work prior to publication. So long as accurate, such publications are not subject to Microsoft\u2019s approval except that, at Microsoft\u2019s request, researcher will delete any Microsoft Confidential Information identified or delay publication to enable Microsoft to file for appropriate intellectual property (IP) protection for any project IP disclosed in such work.<\/li>\n\n\n\n
- All data sets and any new IP resulting from this effort will be made public and publicly available for any researcher, developer, or interested party to access to help further the goals of this initiative in providing higher quality and better access to technology services that empowers people and organizations to be more productive.<\/li>\n\n\n\n
- Funded researchers must seek approval of their institution\u2019s review board for any work that involves human subjects.<\/li>\n\n\n\n
- At the completion of the project, the funded researchers will be required to submit to Microsoft a report describing project learnings.<\/li>\n\n\n\n
- Any security issues in Microsoft products or services discovered during this research must be reported to the Microsoft Security Response Center (opens in new tab)<\/span><\/a>.<\/li>\n<\/ul>\n\n\n\n\n\n\n\n
Collaborative Research Proposal Requirements<\/h2>\n\n\n\n
The deadline for proposal submissions has now passed.<\/strong><\/p>\n\n\n\nMicrosoft shall have no obligation to maintain the confidentiality of any submitted proposals. Therefore, proposals should not contain information that is confidential, proprietary, restricted, or sensitive. Proposals will be evaluated by a panel of subject-matter experts chosen from Microsoft. Microsoft reserves the right to make the winning proposals publicly available, except those portions containing budgetary information.<\/p>\n\n\n\n
Length<\/h3>\n\n\n\n
The proposal should not be more than seven pages in length of Times New Roman 11-point font. Any documentation beyond that length will not be included as part of the proposal review.<\/p>\n\n\n\n
The seven-page limit includes the cover page but the proposal can start on the cover page if additional space is needed. Scholarly references\/bibliography can be submitted in addition to the seven pages and will not count toward the seven-page limit.<\/p>\n\n\n\n
Cover page<\/h3>\n\n\n\n
The proposal should have a cover page that provides the following information:<\/p>\n\n\n\n
\n- Biographical information and contact information: <\/strong>This should include a brief description of any relevant prior research, publications, or other professional experience.\n
\n- Faculty with deep technical experience related to the research areas described above are encouraged to apply. Indicate estimated level of effort\/amount of time each faculty member will spend on the project.<\/li>\n\n\n\n
- Post-doctoral researchers and mid- to late-stage PhD students with deep technical experience related to the research should be included in proposals. Indicate the estimated level of effort\/amount of time each post-doctoral researcher and PhD student will spend on the project.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
- Project proposal abstract: <\/strong>The abstract should contain the following:\n
\n- A nontechnical description of the project that states the problem to be studied and explains the project\u2019s broader significance and importance.<\/li>\n\n\n\n
- A technical description of the project that states the goals and scope of the research, and the methods and approaches to be used.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
Proposal body:<\/strong> The proposal body should include the following information:<\/p>\n\n\n\n\n- Project description: <\/strong>Include what set of questions based on the identified research scenarios above, will be addressed and how they will be addressed. Describe how answering these questions will help advance the state-of-the-art in security research.<\/li>\n\n\n\n
- Approach: <\/strong>Describe the methodological and theoretical approach that the researchers will use. Explain exactly how the researchers will go about answering the question. Describe how the researchers will handle the legal and ethical challenges of doing work in this area. This section should also describe how the university MSecAI team proposes to work with Microsoft counterparts (researchers and engineers) to ensure an effective and positive collaboration.<\/li>\n\n\n\n
- Resources: <\/strong>Proposals should specify if and how Microsoft technologies will be used, namely (1) APIs, (2) Data sets, etc. if applicable.<\/li>\n\n\n\n
- Expected results: <\/strong>Briefly describe what new knowledge is likely to be generated as a result of this research, why these results would be significant, and how this could benefit information workers of tomorrow.<\/li>\n\n\n\n
- Related research: <\/strong>Briefly summarize related research, including references where appropriate.<\/li>\n\n\n\n
- Researcher roles: <\/strong>Describe the role of each researcher involved in the project and explain how their skills and knowledge enable them to address the proposed research.<\/li>\n\n\n\n
- ~12-18-month Timeline\/Workplan and Schedule: <\/strong>Describe what milestones will be used to measure progress of the project during the year and when they will be completed. If the project is part of a larger ongoing research program, estimate the time for completion of this project only. It is expected that the award will be made on or after June 18, 2021. Project timelines should reflect starting times on or shortly after this date.<\/li>\n\n\n\n
- Use of funds: <\/strong>Provide a budget (in U.S. dollars) describing how the award will be used. The budget should be presented as a table with the total budget request clearly indicated. Microsoft will consider requests for Azure credits necessary to conduct research. Value of Azure credits will not be considered a part of the budget request. Azure requests should be included in the budget table.<\/li>\n\n\n\n
- Other support: <\/strong>Include other contributions to this project (cash, goods, and services) by your university or other sources, if any, but do not include the use of university\/organization facilities that are otherwise provided on an ongoing basis. Describe other grants or funded research that may be leveraged to add value to this research effort. Note: authors of the selected proposal will be required to submit an original letter on their institution\u2019s letterhead, certifying the commitment of any additional or matching support described in the proposal.<\/li>\n<\/ul>\n\n\n\n\n\n\n\n
Yes, multiple universities can submit a joint\/single proposal together. Please clearly indicate in the budget section how the budget, not to exceed $150,000 USD, will be shared.<\/p>\n\n\n\n\n\n
Yes, Microsoft will pay each university directly provided the budget clearly illustrates the amount to be paid to each university with a total not to exceed $150,000 USD.<\/p>\n\n\n\n\n\n
Project timelines should be approximately 12-18 months. They should reflect the total time estimated to complete the research proposed.<\/p>\n\n\n\n\n\n
Yes, proposals must indicate which of the listed research areas will be investigated as part of the proposed research to be eligible for consideration.<\/p>\n\n\n\n\n\n
It would be considered a positive for the proposal to have a researcher at Microsoft who is supportive, but we don\u2019t require it or expect it. If a researcher at Microsoft is interested in expressing support for your proposal, there is an opportunity during the proposal submission process to request their letter of recommendation. When received, it will be associated with your proposal and considered during the review.<\/p>\n\n\n\n\n\n
Yes, proposal budget requests can be of any amount up to $150,000 USD.<\/p>\n\n\n\n\n\n
The budget is part of the seven-page limit. Scholarly references\/bibliography can be submitted in addition to the seven pages and will not count toward the seven-page limit but all of the other required components will count toward the seven-page limit.<\/p>\n\n\n\n\n\n
No, letters of support will not count toward the seven-page limit.<\/p>\n\n\n\n\n\n
As long as the full proposal doesn\u2019t exceed seven pages the rest of the section lengths are flexible.<\/p>\n\n\n\n\n\n
We would be looking for cost-share. This is not a mandatory requirement.<\/p>\n\n\n\n\n\n
We would be looking for contributions that directly support the research efforts here so indirect-costs that cover items such as facilities and infrastructure would not count toward university support\/cost-share\/in-kind contribution.<\/p>\n\n\n\n\n\n
Since this is not a requirement, there is no expected amount.<\/p>\n\n\n\n\n\n
The funds will be considered a gift that has no restrictions on how it is used. Budgets should reflect university\u2019s own policies for accepting unrestricted gifts<\/p>\n\n\n\n\n\n
There are no restrictions on how the funds are used. We do request that how the funds will be used is clearly illustrated in the required budget portion of the proposal.<\/p>\n\n\n\n\n\n
There are no restrictions on how the funds are used. We do request that how the funds will be used is clearly illustrated in the required budget portion of the proposal.<\/p>\n\n\n\n\n\n
The proposal budget should reflect your university\u2019s policies toward receiving unrestricted gifts and should emphasize allocation of funds toward completing the research proposed.<\/p>\n\n\n\n\n\n
As unrestricted gifts, it will be entirely up to the winners to decide how to spend the award to achieve the research goals in the proposal.<\/p>\n\n\n\n\n\n
We will not be able to provide access to any data that is not already publicly available.<\/p>\n\n\n\n\n\n
Yes, the results of this research are meant to be open and public for unrestricted use by future researchers and technologists.<\/p>\n\n\n\n\n\n
You are encouraged to assemble a team that is most likely to achieve the greatest results within the time and budget parameters required.<\/p>\n\n\n\n\n\n
Both of these scenarios are valuable. The results of this research will be open and public and so they are meant to drive future research and technology development. More insight on how people work together leading to implications for designs of future tools \u2013 though not designed just by Microsoft but others as well that are working in these topic areas would be of interest.<\/p>\n\n\n\n
However, if you feel you can develop breakthrough prototypes that also inform future research then that would also be interesting.<\/p>\n\n\n\n\n\n\n\n
2021 Microsoft Security Research AI RFP Winners<\/h2>\n\n\n\n\n\t\n\t\n\t\t\n\t\t\t![\"portrait](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/06\/Amin-Kharraz.jpeg\")
<\/figure>\n\n\n\nAmin Kharraz<\/h4>\n\n\n\n
Florida International University<\/p>\n\n\n\n
Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\nWEBHASH: A Spatio-Temporal Deep Learning Approach for Detecting Social Engineering Attacks<\/strong><\/p>\n\n\n\nSocial engineering attacks continue to remain a top security threat. The impact of these attacks is often deep and consequential. Modern social engineering attacks have evolved to deliver different classes of malicious code while collecting extensive financial and personal information. Unfortunately, current mechanisms are woefully inadequate to identify and reason about such adversarial operations, leaving organizations and end-users open to a variety of consequential attacks. The goal of this project is to design principles that will guide the development of an unsupervised approach to automatically identify temporal drifts and detect emerging trends in the social engineering attack landscape. The core insight of our research is that most of social engineering campaigns rarely change the underlying software development techniques to build their attack pages and tend to reuse specific web development patterns to generate a diverse set of attack pages. In this proposal, we develop a novel similarity hashing mechanism, called WEBHASH, which takes into account the spatio-temporal characteristics of a target website and convert them into a vector that facilitates a low-overhead attribution and similarity testing at scale. We will take advantage of advances in machine learning and incorporate Siamese Neural Networks (SNNs) to conduct unsupervised similarity testing across the vectorized data. We posit that a number of useful activities can be performed with WEBHASH. By developing low latency detection and mitigation platforms for social engineering attacks, we can better protect organizations and institutions from data breaches and reduce users\u2019 exposure to modern social engineering attacks. WEBHASH also allows approximating the prevalence of an emerging social engineering threat or the adoption of new attack techniques across different campaigns with minimal human intervention.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n
\n\t\n\t\n\t\t\n\t\t\t![\"portraits](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/06\/zhou-yanning.jpg\")
<\/figure>\n\n\n\nZhou Li and Yanning Shen<\/h4>\n\n\n\n
University of California Irvine<\/p>\n\n\n\n
Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\nScalable Graph Learning for Automatic Detection of Spearphishing<\/strong><\/p>\n\n\n\nIn this project, we will tackle the problem of automated spearphishing detection. Spearphishing has become a primary attack vector to perpetuate entities in public and private sectors, causing billions of dollars loss annually. Due to the advanced social-engineering tricks performed by the attackers, spearphishing emails are often evasive, difficult to capture by the existing approaches based on malware detection, sender\/domain blacklisting, etc. To address this urgent threat, we will explore how to adapt state-of-the-art graph learning algorithms. In particular, we will first investigate how to model the email data as a graph, such that the spearphishing impersonators can be distinguished. Then, we will build a detection system with multi-kernel learning to capture the complex relationship between email users and their sending behaviors. For timely detection, we will examine how the trained classifier can be updated online with Random Feature based function estimation. Finally, we will derive the relation between different function estimators and the privacy levels. We expect this project to have profound impact on email security and research in graph learning.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n
\n\n\n\n2020 Microsoft Security Research AI RFP Winners<\/h2>\n\n\n\n\n\t\n\t\n\t\t\n\t\t\t![\"Dawn](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/06\/Song-Gao-Berkeley.jpg\")
<\/figure>\n\n\n\nDawn Song and Peng Gao<\/h4>\n\n\n\n
University of California, Berkeley<\/p>\n\n\n\n
Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\nA Security Knowledge Graph for Automated Threat Intelligence Gathering and Management<\/strong><\/p>\n\n\n\nSophisticated cyber-attacks have plagued many high-profile businesses. To gain visibility into the fast-evolving threat landscape, open-source Cyber Threat Intelligence (OSCTI) has received growing attention from the community. Commonly, knowledge about a threat is presented in a vast number of OSCTI reports, detailing how the threat unfolds into multiple steps. Despite the pressing need for high-quality OSCTI, existing approaches, however, have primarily operated on fragmented threat indicators (e.g., Indicators of Compromise). On the other hand, descriptive relationships between threat indicators have been overlooked, which contain essential information on the threat behaviors that is critical to uncovering the complete threat scenario. Recognizing the limitation, this proposal seeks to design and develop an intelligent and scalable system for automated threat intelligence gathering and management. The proposed system will use a combination of AI-based methods to collect heterogeneous OSCTI data from various sources, extract comprehensive knowledge about threat behaviors in the form of security-related entities and their relations, construct a security knowledge graph from the extracted information, and update the knowledge graph by continuously learning from its deployment. We will also pursue possible security defensive applications that can be further empowered by OSCTI. The proposed work has a broad impact for advancing the state-of-the-art in threat intelligence gathering, management, and applications.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n
\n\t\n\t\n\t\t\n\t\t\t![\"Nick](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/03\/Nick-Heard.jpg\")
<\/figure>\n\n\n\nNick Heard<\/h4>\n\n\n\n
Department of Mathematics, Imperial College London<\/p>\n\n\n\n
Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\nUnderstanding the enterprise: Host-based event prediction for automatic defence in cyber-security<\/strong><\/p>\n\n\n\nThe next generation of cyber-security challenges will demonstrate an increase in complexity and sophistication, aided by artificial intelligence. To counter this AI-driven threat, we propose to develop Bayesian statistical methodologies for adaptively designing robust, interpretable mathematical models of normal behaviour in new environments. These methodologies will provide new insights into enterprise systems, providing detailed under-standing of network assets and their relationships. These insights will inform enterprise risk-based assessments and enhance the detection and response to cyber threats. Challenges will include the fusion of diverse data sources, collected both within the network environment and externally, and securely sharing intelligence obtained from other platforms. To address these challenges, the proposed workflows will construct modelling frameworks for adaptively building probability distributions for predicting the future activity of a network host. Perspectives in both discrete time and continuous time, along with hybrids of the two, will be considered. Central to the model-building challenge will be developing principled methods for automatically identifying the quantity (either in terms of counts, or in time horizons) of historical data which should be conditioned upon in forming short-term and longer-term predictions. The principal modelling paradigm will be centered on a host-based approach, which has both the capacity to scale and be most sensitive to the protection of sensitive data. Additionally, there will be important scope for making inferences about large-scale network structure, to inform these host-based AI technologies about the position, importance and likely connectivity of the node within the network.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n
\n\t\n\t\n\t\t\n\t\t\t![\"Nicolas](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/03\/nicolas.jpg\")
<\/figure>\n\n\n\nNicolas Papernot<\/h4>\n\n\n\n
University of Toronto, Department of Electrical and Computer Engineering<\/p>\n\n\n\n
Microsoft lead collaborator:<\/strong> Azure Trustworthy Machine Learning + Microsoft Security Response Center (MSRC)<\/p>\n\n\n\nTowards Machine Learning Governance<\/strong><\/p>\n\n\n\nThe predictions of machine learning (ML) systems often appear fragile, with no hint as to the reasoning behind them\u2014and may be dangerously wrong. This is unacceptable: society must be able to trust and hold to account ML. This proposal seeks to empower ML developers and engineers to develop and design ML systems that are secure and provide the tools that enable its users to manage security, legal, and regulatory standards. Our efforts achieve this through the development of machine learning governance. We focus our efforts around two attack vectors: (1) input manipulations at training and test time that target the ML system\u2019s integrity and (2) model inversion and extraction that target the privacy of training data and the confidentiality of model architectural details. We propose to tackle the first attack vector through the development of robust model uncertainty estimates, the identification of coresets in ML, and the creation of computationally efficient influence metrics. We approach the second attack vector by focusing on the life of ML systems after they have been trained: we will pursue model watermarking, machine unlearning, and the identifiability of ML outputs.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n","protected":false},"featured_media":498056,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":true,"_classifai_error":"","msr_hide_image_in_river":0,"footnotes":""},"msr-opportunity-type":[155533],"msr-region":[256048],"msr-locale":[268875],"msr-program-audience":[243727],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-640278","msr-academic-program","type-msr-academic-program","status-publish","has-post-thumbnail","hentry","msr-opportunity-type-grants-and-fellowships","msr-region-global","msr-locale-en_us","msr-program-audience-faculty"],"msr_description":"","msr_social_media":[],"related-researchers":[],"tab-content":[{"id":0,"name":"About","content":"
What it is<\/h2>\r\nFunding for collaborative research between Microsoft and universities working together to make advances in artificial intelligence to solve computing security problems.\r\nAbout<\/h2>\r\nMicrosoft is committed to pushing the boundaries of technology to empower every person and every organization on the planet to achieve more. The cornerstone of how Microsoft does this is by building systems that are secure and by providing tools that enable customers to manage security, legal, and regulatory standards.\r\n\r\nThe goal of this request for proposals (RFP) is to spark new AI research in different areas of phish protection that will expand our understanding of the communication graph, email and web content, economics of phishing and how to secure our customer\u2019s assets in the face of increasingly sophisticated attacks while providing fairness and privacy guarantees.\r\n\r\nAs our cyber defense systems grow more complex in the face of ever evolving and sophisticated attackers, the human element remains the weakest link with few effective protections. Humans are targeted through various modern communication channels and tricked into disclosing sensitive information that may include credentials, financial details, PII data and certificates. According to the FBI\u2019s 2020 IC3 Report<\/a> social engineering attacks including phishing, vishing, smishing, etc. have gone up by 110% from 2019 alone.\r\n\r\nMicrosoft Security AI Academic Program is launching an academic grants program. We will fund one or more projects (up to $150K in total funding for this RFP) in new collaborative research efforts with university partners so that we can invent the future of security together.\r\nTimeline<\/h2>\r\n\r\n \t- April 30, 2021:<\/strong> RFP published.<\/li>\r\n \t
- June 6, 2021:<\/strong> Proposals due.<\/li>\r\n \t
- June 18, 2021:<\/strong> Winners announced.<\/li>\r\n \t
- Summer 2021:<\/strong> Awards made, and planning begins with regularly scheduled meetings, calls, and visit(s) by Microsoft to MSecAI winning university.<\/li>\r\n \t
- Spring 2022:<\/strong> Review of progress for potential second round of funding (pending progress and availability of funds).<\/li>\r\n \t
- Fall 2022:<\/strong> Report back.<\/li>\r\n<\/ul>\r\n
Research Goals<\/h2>\r\nResearch is an integral part of the innovation loop. Most of the exciting research is happening in universities around the world. The goal of the Microsoft Security AI (MSecAI) RFP is to develop new knowledge and capabilities that can provide a robust defense against future attacks. Through our grants program, we hope not only to support academic research, but also to develop long-term collaborations with researchers around the world who share the same goal of protecting private data from unauthorized access.\r\n\r\nProposals are invited on all areas of computing related to phish protection and AI, particularly in the following areas of interest:\r\nUnderstanding the communication graph<\/h3>\r\nA communication graph is a collection of entities including user accounts, applications, websites, shared infrastructure and the relationships between those entities such as emails, P2P messages, login attempts, etc. How do we leverage this dynamic graph at scale to extract key insights while providing privacy guarantees? Can we understand user interaction profiles over time and identify deviations to detect compromised accounts, phish emails from spoofed domains, bulk emails, etc.?\r\nUnderstanding the content<\/h3>\r\n90% of large enterprise customer breaches start from email that tricks users into revealing sensitive information. Most of these emails leverage some part of psychological manipulation that displays a sense of authority or urgency to take immediate action, threat, opportunity for monetary gain or loss, etc. Assuming clear text email data is available, what are some approaches that help machines understand the high level intention of a given email while providing privacy guarantees? How can we effectively group known phish emails into high level campaigns based on the content topics and exploitation techniques?\r\nFairness and accountability for security<\/h3>\r\nAs ML is used for more security-sensitive applications, the ability for these systems to generalize globally, not be disruptive to end users, especially any specific segment of user population is quite important. How do we define fairness in security and identify related issues when developing AI systems? Can we develop offline and online experimentation tools to test that our ML models are not biased with respect to attributes such as geo locations, language, industry verticals, etc.\u00a0 How do these test cases help us validate the fairness of ML models?\r\n\r\nWhen it comes to accountability, how can we identify and assign responsibility for a decision made by an AI system? What steps can an incident responder take to respond to the business disruptions caused by misclassifications from AI system? How can we validate that the same misclassifications do not reoccur as ML systems are retrained? In addition, some of the ML systems may work with complex obfuscated data sources that might not generate human understandable explanations. How do we justify the decisions made by AI systems in such cases?\r\nVerifying the authenticity of modern communication channels<\/h3>\r\nWhile industry phishing attempts are predominantly carried out through email, many of these attacks have migrated to modern communication channels like professional networks, p2p messaging, search and ads. Phish attempts are becoming increasingly convincing to end users with the advancement of techniques like deep fakes for audio and video generation, content morphing, fake replies.\u00a0 How do we leverage AI systems to verify the authenticity of such content? Moreover, how do we differentiate legitimate user accounts from adversarial\/ tester accounts setup to test defense systems or pollute backend telemetry?\r\nProtecting patient zero<\/h3>\r\nBased on this<\/a> paper, an average phishing attack spans 21 hours between the first and last victim and the detection of each attack occurs an average 9 hours after the first victim. This gives attackers a window of opportunity during which most of the damage is done. How do we leverage AI systems to adapt to the adversarial temporal drift and prevent the first victim\/ patient zero from being compromised? How can we use human-in-the-loop AI systems to enable experts to update defenses automatically? How can AI systems be leveraged to identify and learn from discovery of new attack campaigns? How can we augment supervised ML approaches with unlabeled, noisy data to ensure a good feature distribution coverage in training our ML models?\r\nEconomics of phishing<\/h3>\r\nPhishing can be seen as an economic problem. Attackers operate like businesses by making investments in campaign inputs to generate returns by selling stolen credentials, using stolen credentials to gain network access, or committing direct fraud. Firms and users invest hundreds of billions of dollars annually in security protection and expect returns on those investments through reduced cyber risk or increased productivity gains. These markets are rich in common economic complications like externalities, asymmetric information, and uncertainty. However, they remain poorly understood. Can we categorize the attacker ecosystem by business model? What are the returns to firms\u2019 security investments? How do security investments impact the attacker ecosystem and vice versa?\r\n\r\n[accordion] [panel header=\"Funding\"]\r\nMicrosoft funding<\/h2>\r\nMicrosoft will fund one or more projects (up to $150K in total funding for this RFP). A second round of funding pending initial progress and outcomes (see Timeline above) may be considered at some point during this collaboration. All funding decisions will be at the sole discretion of Microsoft. Proposals for this RFP should provide an initial budget and workplan for the research based on the Timeline section below.\r\n\r\nMicrosoft encourages potential university partners to consider using resources outlined in the RFP in the following manner:\r\n\r\n \t- PhD scholarship stipends.<\/li>\r\n \t
- Post-doctoral researcher funding.<\/li>\r\n \t
- Software and hardware research engineer funding.<\/li>\r\n \t
- Limited but essential hardware and software needed to conduct the research.<\/li>\r\n<\/ul>\r\nProposal plans should include any of these, or other items, that directly support the proposed research.\r\n\r\nMicrosoft research collaborators, at no cost to the winning teams, may visit the university partners one or more times to foster collaborative planning and research. These visits will be agreed upon and scheduled after an award decision is made. Likewise, a cadence of meetings will be mutually agreed upon at the start of the collaboration. Proposals are welcome to include other suggestions about how to foster an effective collaborative research engagement.\r\n\r\n[\/panel][panel header=\"Eligibility\"]\r\n
Eligibility<\/h2>\r\nThis RFP is not restricted to any one discipline or tailored to any methodology. Universities are welcome to submit cross-disciplinary proposals if that contributes to answering the proposed research question(s).\r\n\r\nTo be eligible for this RFP, your institution and proposal must meet the following requirements:\r\n\r\n \t- Institutions must have access to the knowledge, resources, and skills necessary to carry out the proposed research.<\/li>\r\n \t
- Institutions must be either an accredited or otherwise degree-granting university with non-profit status, or a research organization with non-profit status.<\/li>\r\n \t
- Proposals that are incomplete or request funds more than the maximum award will be excluded from the selection process.<\/li>\r\n \t
- The proposal budget must reflect your university\u2019s policies toward receiving unrestricted gifts and should emphasize allocation of funds toward completing the research proposed.<\/li>\r\n<\/ul>\r\nAdditionally:\r\n
\r\n \t- Proposals should include a timeline (approximately 12-18 months) or workplan that begins in summer 2021 and ends in fall of 2022.<\/li>\r\n \t
- To optimize the chances of receiving an award, we encourage researchers from the same university to consider submitting a single, joint proposal (rather than multiple individual proposals) that leverages their various skills and interests to create the strongest possible proposal.<\/li>\r\n \t
- Multiple universities can submit a joint\/single proposal together. Please clearly indicate in the budget section how the budget, not to exceed $150K USD, will be shared.<\/li>\r\n<\/ul>\r\n[\/panel] [panel header=\"Selection process & criteria\"]\r\n
Selection process and criteria<\/h2>\r\nAll proposals received by the submission deadline and in compliance with the eligibility criteria will be evaluated by a panel of subject-matter experts chosen by Microsoft. Drawing from evaluations by the review panel, Microsoft will select which proposals will receive the awards. Microsoft reserves the right to fund the winning proposal at an amount greater or lower than the amount requested, up to the stated maximum amount. Note: Microsoft will not provide individual feedback on proposals that are not funded.\r\n\r\nAll proposals will be evaluated based on the following criteria:\r\n\r\n \t- Addresses an important research area identified above<\/strong> that, if answered, has the potential to have a significant impact on that domain.<\/li>\r\n \t
- Expected value and potential impact<\/strong> of the research on relevant information security fields.<\/li>\r\n \t
- Potential for wide dissemination and use of knowledge<\/strong>, including specific plans for scholarly publications, public presentations, and white papers.<\/li>\r\n \t
- Ability to complete the project<\/strong> based upon adequate available resources, reasonable timelines, and the identified contributors\u2019 qualifications.<\/li>\r\n \t
- Qualifications of the research team<\/strong>, including previous history of work in the area, successful completion of previous projects, research or teaching awards, and scholarly publications.<\/li>\r\n \t
- Diversity<\/strong> is highly valued and research teams should strive to reflect a diversity of backgrounds, experiences, and talent reflected in the research teams.<\/li>\r\n \t
- Evidence of university support<\/strong> contributed in-kind to directly support and supplement the research efforts.<\/li>\r\n \t
- Budget<\/strong> is strategic to maximize impact of research.<\/li>\r\n \t
- Possible additional information<\/strong> as requested by the review panel, which might be requested via a conference call.<\/li>\r\n<\/ul>\r\n[\/panel][panel header=\"Conditions\"]\r\n
Conditions<\/h2>\r\n
Research Goals<\/h2>\n\n\n\n
Research is an integral part of the innovation loop. Most of the exciting research is happening in universities around the world. The goal of the Microsoft Security AI (MSecAI) RFP is to develop new knowledge and capabilities that can provide a robust defense against future attacks. Through our grants program, we hope not only to support academic research, but also to develop long-term collaborations with researchers around the world who share the same goal of protecting private data from unauthorized access.<\/p>\n\n\n\n
Proposals are invited on all areas of computing related to phish protection and AI, particularly in the following areas of interest:<\/p>\n\n\n\n
Understanding the communication graph<\/h3>\n\n\n\n
A communication graph is a collection of entities including user accounts, applications, websites, shared infrastructure and the relationships between those entities such as emails, P2P messages, login attempts, etc. How do we leverage this dynamic graph at scale to extract key insights while providing privacy guarantees? Can we understand user interaction profiles over time and identify deviations to detect compromised accounts, phish emails from spoofed domains, bulk emails, etc.?<\/p>\n\n\n\n
Understanding the content<\/h3>\n\n\n\n
90% of large enterprise customer breaches start from email that tricks users into revealing sensitive information. Most of these emails leverage some part of psychological manipulation that displays a sense of authority or urgency to take immediate action, threat, opportunity for monetary gain or loss, etc. Assuming clear text email data is available, what are some approaches that help machines understand the high level intention of a given email while providing privacy guarantees? How can we effectively group known phish emails into high level campaigns based on the content topics and exploitation techniques?<\/p>\n\n\n\n
Fairness and accountability for security<\/h3>\n\n\n\n
As ML is used for more security-sensitive applications, the ability for these systems to generalize globally, not be disruptive to end users, especially any specific segment of user population is quite important. How do we define fairness in security and identify related issues when developing AI systems? Can we develop offline and online experimentation tools to test that our ML models are not biased with respect to attributes such as geo locations, language, industry verticals, etc. How do these test cases help us validate the fairness of ML models?<\/p>\n\n\n\n
When it comes to accountability, how can we identify and assign responsibility for a decision made by an AI system? What steps can an incident responder take to respond to the business disruptions caused by misclassifications from AI system? How can we validate that the same misclassifications do not reoccur as ML systems are retrained? In addition, some of the ML systems may work with complex obfuscated data sources that might not generate human understandable explanations. How do we justify the decisions made by AI systems in such cases?<\/p>\n\n\n\n
Verifying the authenticity of modern communication channels<\/h3>\n\n\n\n
While industry phishing attempts are predominantly carried out through email, many of these attacks have migrated to modern communication channels like professional networks, p2p messaging, search and ads. Phish attempts are becoming increasingly convincing to end users with the advancement of techniques like deep fakes for audio and video generation, content morphing, fake replies. How do we leverage AI systems to verify the authenticity of such content? Moreover, how do we differentiate legitimate user accounts from adversarial\/ tester accounts setup to test defense systems or pollute backend telemetry?<\/p>\n\n\n\n
Protecting patient zero<\/h3>\n\n\n\n
Based on this (opens in new tab)<\/span><\/a> paper, an average phishing attack spans 21 hours between the first and last victim and the detection of each attack occurs an average 9 hours after the first victim. This gives attackers a window of opportunity during which most of the damage is done. How do we leverage AI systems to adapt to the adversarial temporal drift and prevent the first victim\/ patient zero from being compromised? How can we use human-in-the-loop AI systems to enable experts to update defenses automatically? How can AI systems be leveraged to identify and learn from discovery of new attack campaigns? How can we augment supervised ML approaches with unlabeled, noisy data to ensure a good feature distribution coverage in training our ML models?<\/p>\n\n\n\n Phishing can be seen as an economic problem. Attackers operate like businesses by making investments in campaign inputs to generate returns by selling stolen credentials, using stolen credentials to gain network access, or committing direct fraud. Firms and users invest hundreds of billions of dollars annually in security protection and expect returns on those investments through reduced cyber risk or increased productivity gains. These markets are rich in common economic complications like externalities, asymmetric information, and uncertainty. However, they remain poorly understood. Can we categorize the attacker ecosystem by business model? What are the returns to firms\u2019 security investments? How do security investments impact the attacker ecosystem and vice versa?<\/p>\n\n\n\n\n\n Microsoft will fund one or more projects (up to $150K in total funding for this RFP). A second round of funding pending initial progress and outcomes (see Timeline above) may be considered at some point during this collaboration. All funding decisions will be at the sole discretion of Microsoft. Proposals for this RFP should provide an initial budget and workplan for the research based on the Timeline section below.<\/p>\n\n\n\n Microsoft encourages potential university partners to consider using resources outlined in the RFP in the following manner:<\/p>\n\n\n\n Proposal plans should include any of these, or other items, that directly support the proposed research.<\/p>\n\n\n\n Microsoft research collaborators, at no cost to the winning teams, may visit the university partners one or more times to foster collaborative planning and research. These visits will be agreed upon and scheduled after an award decision is made. Likewise, a cadence of meetings will be mutually agreed upon at the start of the collaboration. Proposals are welcome to include other suggestions about how to foster an effective collaborative research engagement.<\/p>\n\n\n\n\n\n This RFP is not restricted to any one discipline or tailored to any methodology. Universities are welcome to submit cross-disciplinary proposals if that contributes to answering the proposed research question(s).<\/p>\n\n\n\n To be eligible for this RFP, your institution and proposal must meet the following requirements:<\/p>\n\n\n\n Additionally:<\/p>\n\n\n\n All proposals received by the submission deadline and in compliance with the eligibility criteria will be evaluated by a panel of subject-matter experts chosen by Microsoft. Drawing from evaluations by the review panel, Microsoft will select which proposals will receive the awards. Microsoft reserves the right to fund the winning proposal at an amount greater or lower than the amount requested, up to the stated maximum amount. Note: Microsoft will not provide individual feedback on proposals that are not funded.<\/p>\n\n\n\n All proposals will be evaluated based on the following criteria:<\/p>\n\n\n\n The deadline for proposal submissions has now passed.<\/strong><\/p>\n\n\n\n Microsoft shall have no obligation to maintain the confidentiality of any submitted proposals. Therefore, proposals should not contain information that is confidential, proprietary, restricted, or sensitive. Proposals will be evaluated by a panel of subject-matter experts chosen from Microsoft. Microsoft reserves the right to make the winning proposals publicly available, except those portions containing budgetary information.<\/p>\n\n\n\n The proposal should not be more than seven pages in length of Times New Roman 11-point font. Any documentation beyond that length will not be included as part of the proposal review.<\/p>\n\n\n\n The seven-page limit includes the cover page but the proposal can start on the cover page if additional space is needed. Scholarly references\/bibliography can be submitted in addition to the seven pages and will not count toward the seven-page limit.<\/p>\n\n\n\n The proposal should have a cover page that provides the following information:<\/p>\n\n\n\n Proposal body:<\/strong> The proposal body should include the following information:<\/p>\n\n\n\n Yes, multiple universities can submit a joint\/single proposal together. Please clearly indicate in the budget section how the budget, not to exceed $150,000 USD, will be shared.<\/p>\n\n\n\n\n\n Yes, Microsoft will pay each university directly provided the budget clearly illustrates the amount to be paid to each university with a total not to exceed $150,000 USD.<\/p>\n\n\n\n\n\n Project timelines should be approximately 12-18 months. They should reflect the total time estimated to complete the research proposed.<\/p>\n\n\n\n\n\n Yes, proposals must indicate which of the listed research areas will be investigated as part of the proposed research to be eligible for consideration.<\/p>\n\n\n\n\n\n It would be considered a positive for the proposal to have a researcher at Microsoft who is supportive, but we don\u2019t require it or expect it. If a researcher at Microsoft is interested in expressing support for your proposal, there is an opportunity during the proposal submission process to request their letter of recommendation. When received, it will be associated with your proposal and considered during the review.<\/p>\n\n\n\n\n\n Yes, proposal budget requests can be of any amount up to $150,000 USD.<\/p>\n\n\n\n\n\n The budget is part of the seven-page limit. Scholarly references\/bibliography can be submitted in addition to the seven pages and will not count toward the seven-page limit but all of the other required components will count toward the seven-page limit.<\/p>\n\n\n\n\n\n No, letters of support will not count toward the seven-page limit.<\/p>\n\n\n\n\n\n As long as the full proposal doesn\u2019t exceed seven pages the rest of the section lengths are flexible.<\/p>\n\n\n\n\n\n We would be looking for cost-share. This is not a mandatory requirement.<\/p>\n\n\n\n\n\n We would be looking for contributions that directly support the research efforts here so indirect-costs that cover items such as facilities and infrastructure would not count toward university support\/cost-share\/in-kind contribution.<\/p>\n\n\n\n\n\n Since this is not a requirement, there is no expected amount.<\/p>\n\n\n\n\n\n The funds will be considered a gift that has no restrictions on how it is used. Budgets should reflect university\u2019s own policies for accepting unrestricted gifts<\/p>\n\n\n\n\n\n There are no restrictions on how the funds are used. We do request that how the funds will be used is clearly illustrated in the required budget portion of the proposal.<\/p>\n\n\n\n\n\n There are no restrictions on how the funds are used. We do request that how the funds will be used is clearly illustrated in the required budget portion of the proposal.<\/p>\n\n\n\n\n\n The proposal budget should reflect your university\u2019s policies toward receiving unrestricted gifts and should emphasize allocation of funds toward completing the research proposed.<\/p>\n\n\n\n\n\n As unrestricted gifts, it will be entirely up to the winners to decide how to spend the award to achieve the research goals in the proposal.<\/p>\n\n\n\n\n\n We will not be able to provide access to any data that is not already publicly available.<\/p>\n\n\n\n\n\n Yes, the results of this research are meant to be open and public for unrestricted use by future researchers and technologists.<\/p>\n\n\n\n\n\n You are encouraged to assemble a team that is most likely to achieve the greatest results within the time and budget parameters required.<\/p>\n\n\n\n\n\n Both of these scenarios are valuable. The results of this research will be open and public and so they are meant to drive future research and technology development. More insight on how people work together leading to implications for designs of future tools \u2013 though not designed just by Microsoft but others as well that are working in these topic areas would be of interest.<\/p>\n\n\n\n However, if you feel you can develop breakthrough prototypes that also inform future research then that would also be interesting.<\/p>\n\n\n\n\n\n\n\n Florida International University<\/p>\n\n\n\n Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\n WEBHASH: A Spatio-Temporal Deep Learning Approach for Detecting Social Engineering Attacks<\/strong><\/p>\n\n\n\n Social engineering attacks continue to remain a top security threat. The impact of these attacks is often deep and consequential. Modern social engineering attacks have evolved to deliver different classes of malicious code while collecting extensive financial and personal information. Unfortunately, current mechanisms are woefully inadequate to identify and reason about such adversarial operations, leaving organizations and end-users open to a variety of consequential attacks. The goal of this project is to design principles that will guide the development of an unsupervised approach to automatically identify temporal drifts and detect emerging trends in the social engineering attack landscape. The core insight of our research is that most of social engineering campaigns rarely change the underlying software development techniques to build their attack pages and tend to reuse specific web development patterns to generate a diverse set of attack pages. In this proposal, we develop a novel similarity hashing mechanism, called WEBHASH, which takes into account the spatio-temporal characteristics of a target website and convert them into a vector that facilitates a low-overhead attribution and similarity testing at scale. We will take advantage of advances in machine learning and incorporate Siamese Neural Networks (SNNs) to conduct unsupervised similarity testing across the vectorized data. We posit that a number of useful activities can be performed with WEBHASH. By developing low latency detection and mitigation platforms for social engineering attacks, we can better protect organizations and institutions from data breaches and reduce users\u2019 exposure to modern social engineering attacks. WEBHASH also allows approximating the prevalence of an emerging social engineering threat or the adoption of new attack techniques across different campaigns with minimal human intervention.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n University of California Irvine<\/p>\n\n\n\n Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\n Scalable Graph Learning for Automatic Detection of Spearphishing<\/strong><\/p>\n\n\n\n In this project, we will tackle the problem of automated spearphishing detection. Spearphishing has become a primary attack vector to perpetuate entities in public and private sectors, causing billions of dollars loss annually. Due to the advanced social-engineering tricks performed by the attackers, spearphishing emails are often evasive, difficult to capture by the existing approaches based on malware detection, sender\/domain blacklisting, etc. To address this urgent threat, we will explore how to adapt state-of-the-art graph learning algorithms. In particular, we will first investigate how to model the email data as a graph, such that the spearphishing impersonators can be distinguished. Then, we will build a detection system with multi-kernel learning to capture the complex relationship between email users and their sending behaviors. For timely detection, we will examine how the trained classifier can be updated online with Random Feature based function estimation. Finally, we will derive the relation between different function estimators and the privacy levels. We expect this project to have profound impact on email security and research in graph learning.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n University of California, Berkeley<\/p>\n\n\n\n Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\n A Security Knowledge Graph for Automated Threat Intelligence Gathering and Management<\/strong><\/p>\n\n\n\n Sophisticated cyber-attacks have plagued many high-profile businesses. To gain visibility into the fast-evolving threat landscape, open-source Cyber Threat Intelligence (OSCTI) has received growing attention from the community. Commonly, knowledge about a threat is presented in a vast number of OSCTI reports, detailing how the threat unfolds into multiple steps. Despite the pressing need for high-quality OSCTI, existing approaches, however, have primarily operated on fragmented threat indicators (e.g., Indicators of Compromise). On the other hand, descriptive relationships between threat indicators have been overlooked, which contain essential information on the threat behaviors that is critical to uncovering the complete threat scenario. Recognizing the limitation, this proposal seeks to design and develop an intelligent and scalable system for automated threat intelligence gathering and management. The proposed system will use a combination of AI-based methods to collect heterogeneous OSCTI data from various sources, extract comprehensive knowledge about threat behaviors in the form of security-related entities and their relations, construct a security knowledge graph from the extracted information, and update the knowledge graph by continuously learning from its deployment. We will also pursue possible security defensive applications that can be further empowered by OSCTI. The proposed work has a broad impact for advancing the state-of-the-art in threat intelligence gathering, management, and applications.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n Department of Mathematics, Imperial College London<\/p>\n\n\n\n Microsoft lead collaborator:<\/strong> M365 Security + Compliance Research<\/p>\n\n\n\n Understanding the enterprise: Host-based event prediction for automatic defence in cyber-security<\/strong><\/p>\n\n\n\n The next generation of cyber-security challenges will demonstrate an increase in complexity and sophistication, aided by artificial intelligence. To counter this AI-driven threat, we propose to develop Bayesian statistical methodologies for adaptively designing robust, interpretable mathematical models of normal behaviour in new environments. These methodologies will provide new insights into enterprise systems, providing detailed under-standing of network assets and their relationships. These insights will inform enterprise risk-based assessments and enhance the detection and response to cyber threats. Challenges will include the fusion of diverse data sources, collected both within the network environment and externally, and securely sharing intelligence obtained from other platforms. To address these challenges, the proposed workflows will construct modelling frameworks for adaptively building probability distributions for predicting the future activity of a network host. Perspectives in both discrete time and continuous time, along with hybrids of the two, will be considered. Central to the model-building challenge will be developing principled methods for automatically identifying the quantity (either in terms of counts, or in time horizons) of historical data which should be conditioned upon in forming short-term and longer-term predictions. The principal modelling paradigm will be centered on a host-based approach, which has both the capacity to scale and be most sensitive to the protection of sensitive data. Additionally, there will be important scope for making inferences about large-scale network structure, to inform these host-based AI technologies about the position, importance and likely connectivity of the node within the network.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n\n University of Toronto, Department of Electrical and Computer Engineering<\/p>\n\n\n\n Microsoft lead collaborator:<\/strong> Azure Trustworthy Machine Learning + Microsoft Security Response Center (MSRC)<\/p>\n\n\n\n Towards Machine Learning Governance<\/strong><\/p>\n\n\n\n The predictions of machine learning (ML) systems often appear fragile, with no hint as to the reasoning behind them\u2014and may be dangerously wrong. This is unacceptable: society must be able to trust and hold to account ML. This proposal seeks to empower ML developers and engineers to develop and design ML systems that are secure and provide the tools that enable its users to manage security, legal, and regulatory standards. Our efforts achieve this through the development of machine learning governance. We focus our efforts around two attack vectors: (1) input manipulations at training and test time that target the ML system\u2019s integrity and (2) model inversion and extraction that target the privacy of training data and the confidentiality of model architectural details. We propose to tackle the first attack vector through the development of robust model uncertainty estimates, the identification of coresets in ML, and the creation of computationally efficient influence metrics. We approach the second attack vector by focusing on the life of ML systems after they have been trained: we will pursue model watermarking, machine unlearning, and the identifiability of ML outputs.<\/p>\t\t<\/div>\n\t<\/div>\n\n\t<\/div>\n\n\n","protected":false},"featured_media":498056,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":true,"_classifai_error":"","msr_hide_image_in_river":0,"footnotes":""},"msr-opportunity-type":[155533],"msr-region":[256048],"msr-locale":[268875],"msr-program-audience":[243727],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-640278","msr-academic-program","type-msr-academic-program","status-publish","has-post-thumbnail","hentry","msr-opportunity-type-grants-and-fellowships","msr-region-global","msr-locale-en_us","msr-program-audience-faculty"],"msr_description":"","msr_social_media":[],"related-researchers":[],"tab-content":[{"id":0,"name":"About","content":"Economics of phishing<\/h3>\n\n\n\n
Microsoft funding<\/h2>\n\n\n\n
\n
Eligibility<\/h2>\n\n\n\n
\n
\n
Selection process and criteria<\/h2>\n\n\n\n
\n
Conditions<\/h2>\n\n\n\n
\n
Collaborative Research Proposal Requirements<\/h2>\n\n\n\n
Length<\/h3>\n\n\n\n
Cover page<\/h3>\n\n\n\n
\n
\n
\n
\n
2021 Microsoft Security Research AI RFP Winners<\/h2>\n\n\n\n
<\/figure>\n\n\n\nAmin Kharraz<\/h4>\n\n\n\n
<\/figure>\n\n\n\nZhou Li and Yanning Shen<\/h4>\n\n\n\n
\n\n\n\n2020 Microsoft Security Research AI RFP Winners<\/h2>\n\n\n\n
<\/figure>\n\n\n\nDawn Song and Peng Gao<\/h4>\n\n\n\n
<\/figure>\n\n\n\nNick Heard<\/h4>\n\n\n\n
<\/figure>\n\n\n\nNicolas Papernot<\/h4>\n\n\n\n
What it is<\/h2>\r\nFunding for collaborative research between Microsoft and universities working together to make advances in artificial intelligence to solve computing security problems.\r\n
About<\/h2>\r\nMicrosoft is committed to pushing the boundaries of technology to empower every person and every organization on the planet to achieve more. The cornerstone of how Microsoft does this is by building systems that are secure and by providing tools that enable customers to manage security, legal, and regulatory standards.\r\n\r\nThe goal of this request for proposals (RFP) is to spark new AI research in different areas of phish protection that will expand our understanding of the communication graph, email and web content, economics of phishing and how to secure our customer\u2019s assets in the face of increasingly sophisticated attacks while providing fairness and privacy guarantees.\r\n\r\nAs our cyber defense systems grow more complex in the face of ever evolving and sophisticated attackers, the human element remains the weakest link with few effective protections. Humans are targeted through various modern communication channels and tricked into disclosing sensitive information that may include credentials, financial details, PII data and certificates. According to the FBI\u2019s 2020 IC3 Report<\/a> social engineering attacks including phishing, vishing, smishing, etc. have gone up by 110% from 2019 alone.\r\n\r\nMicrosoft Security AI Academic Program is launching an academic grants program. We will fund one or more projects (up to $150K in total funding for this RFP) in new collaborative research efforts with university partners so that we can invent the future of security together.\r\n
Timeline<\/h2>\r\n
\r\n \t
Research Goals<\/h2>\r\nResearch is an integral part of the innovation loop. Most of the exciting research is happening in universities around the world. The goal of the Microsoft Security AI (MSecAI) RFP is to develop new knowledge and capabilities that can provide a robust defense against future attacks. Through our grants program, we hope not only to support academic research, but also to develop long-term collaborations with researchers around the world who share the same goal of protecting private data from unauthorized access.\r\n\r\nProposals are invited on all areas of computing related to phish protection and AI, particularly in the following areas of interest:\r\n
Understanding the communication graph<\/h3>\r\nA communication graph is a collection of entities including user accounts, applications, websites, shared infrastructure and the relationships between those entities such as emails, P2P messages, login attempts, etc. How do we leverage this dynamic graph at scale to extract key insights while providing privacy guarantees? Can we understand user interaction profiles over time and identify deviations to detect compromised accounts, phish emails from spoofed domains, bulk emails, etc.?\r\n
Understanding the content<\/h3>\r\n90% of large enterprise customer breaches start from email that tricks users into revealing sensitive information. Most of these emails leverage some part of psychological manipulation that displays a sense of authority or urgency to take immediate action, threat, opportunity for monetary gain or loss, etc. Assuming clear text email data is available, what are some approaches that help machines understand the high level intention of a given email while providing privacy guarantees? How can we effectively group known phish emails into high level campaigns based on the content topics and exploitation techniques?\r\n
Fairness and accountability for security<\/h3>\r\nAs ML is used for more security-sensitive applications, the ability for these systems to generalize globally, not be disruptive to end users, especially any specific segment of user population is quite important. How do we define fairness in security and identify related issues when developing AI systems? Can we develop offline and online experimentation tools to test that our ML models are not biased with respect to attributes such as geo locations, language, industry verticals, etc.\u00a0 How do these test cases help us validate the fairness of ML models?\r\n\r\nWhen it comes to accountability, how can we identify and assign responsibility for a decision made by an AI system? What steps can an incident responder take to respond to the business disruptions caused by misclassifications from AI system? How can we validate that the same misclassifications do not reoccur as ML systems are retrained? In addition, some of the ML systems may work with complex obfuscated data sources that might not generate human understandable explanations. How do we justify the decisions made by AI systems in such cases?\r\n
Verifying the authenticity of modern communication channels<\/h3>\r\nWhile industry phishing attempts are predominantly carried out through email, many of these attacks have migrated to modern communication channels like professional networks, p2p messaging, search and ads. Phish attempts are becoming increasingly convincing to end users with the advancement of techniques like deep fakes for audio and video generation, content morphing, fake replies.\u00a0 How do we leverage AI systems to verify the authenticity of such content? Moreover, how do we differentiate legitimate user accounts from adversarial\/ tester accounts setup to test defense systems or pollute backend telemetry?\r\n
Protecting patient zero<\/h3>\r\nBased on this<\/a> paper, an average phishing attack spans 21 hours between the first and last victim and the detection of each attack occurs an average 9 hours after the first victim. This gives attackers a window of opportunity during which most of the damage is done. How do we leverage AI systems to adapt to the adversarial temporal drift and prevent the first victim\/ patient zero from being compromised? How can we use human-in-the-loop AI systems to enable experts to update defenses automatically? How can AI systems be leveraged to identify and learn from discovery of new attack campaigns? How can we augment supervised ML approaches with unlabeled, noisy data to ensure a good feature distribution coverage in training our ML models?\r\n
Economics of phishing<\/h3>\r\nPhishing can be seen as an economic problem. Attackers operate like businesses by making investments in campaign inputs to generate returns by selling stolen credentials, using stolen credentials to gain network access, or committing direct fraud. Firms and users invest hundreds of billions of dollars annually in security protection and expect returns on those investments through reduced cyber risk or increased productivity gains. These markets are rich in common economic complications like externalities, asymmetric information, and uncertainty. However, they remain poorly understood. Can we categorize the attacker ecosystem by business model? What are the returns to firms\u2019 security investments? How do security investments impact the attacker ecosystem and vice versa?\r\n\r\n[accordion] [panel header=\"Funding\"]\r\n
Microsoft funding<\/h2>\r\nMicrosoft will fund one or more projects (up to $150K in total funding for this RFP). A second round of funding pending initial progress and outcomes (see Timeline above) may be considered at some point during this collaboration. All funding decisions will be at the sole discretion of Microsoft. Proposals for this RFP should provide an initial budget and workplan for the research based on the Timeline section below.\r\n\r\nMicrosoft encourages potential university partners to consider using resources outlined in the RFP in the following manner:\r\n
\r\n \t
Eligibility<\/h2>\r\nThis RFP is not restricted to any one discipline or tailored to any methodology. Universities are welcome to submit cross-disciplinary proposals if that contributes to answering the proposed research question(s).\r\n\r\nTo be eligible for this RFP, your institution and proposal must meet the following requirements:\r\n
\r\n \t
\r\n \t
Selection process and criteria<\/h2>\r\nAll proposals received by the submission deadline and in compliance with the eligibility criteria will be evaluated by a panel of subject-matter experts chosen by Microsoft. Drawing from evaluations by the review panel, Microsoft will select which proposals will receive the awards. Microsoft reserves the right to fund the winning proposal at an amount greater or lower than the amount requested, up to the stated maximum amount. Note: Microsoft will not provide individual feedback on proposals that are not funded.\r\n\r\nAll proposals will be evaluated based on the following criteria:\r\n
\r\n \t
Conditions<\/h2>\r\n