{"id":558867,"date":"2019-01-03T13:12:10","date_gmt":"2019-01-03T21:12:10","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-event&p=558867"},"modified":"2025-08-06T11:56:39","modified_gmt":"2025-08-06T18:56:39","slug":"wsdm-2019","status":"publish","type":"msr-event","link":"https:\/\/www.microsoft.com\/en-us\/research\/event\/wsdm-2019\/","title":{"rendered":"Microsoft @ WSDM 2019"},"content":{"rendered":"\n\n

Venue:<\/strong> Melbourne Convention and Exhibition Centre (opens in new tab)<\/span><\/a>
\n1 Convention Centre Place, South Wharf 3006<\/p>\n

Website:<\/strong> WSDM 2019 (opens in new tab)<\/span><\/a>Opens in a new tab<\/span><\/p>\n

Microsoft is excited to be a Silver sponsor of the 12th<\/sup> ACM International conference on Web Search and Data Mining (opens in new tab)<\/span><\/a>. Come by our booth to chat with our experts, see demos of our latest research and find out about career opportunities with Microsoft.<\/p>\n

Microsoft Attendees<\/h3>\n

Puneet Agrawal
\nAhmed Awadallah<\/a>
\nMurat Bayir
\nPaul Bennett<\/a>
\nSusan Dumais<\/a>
\nManish Gupta<\/a>
\nEmre Kiciman<\/a>
\nWidad Machmouchi
\nTobias Schnabel<\/a>
\nAmit Sharma<\/a>
\nRaghav Somani
\nJaime Teevan<\/a>
\nKuansan Wang<\/a>
\nRyen White<\/a>
\nWei Wu<\/a>
\nRoy Zimmermann<\/a><\/p>\n

Demonstration Chair<\/h3>\n

Paul Thomas<\/a><\/p>\n

Program Committee Co-Chair<\/h3>\n

Paul Bennett<\/a><\/p>\n

Senior Program Committee Members<\/h3>\n

Nick Craswell<\/a>
\nEmre Kiciman<\/a>
\nMilad Shokouhi<\/a>
\nRyen White<\/a>
\nXing Xie<\/a><\/p>\n

Program Committee Members<\/h3>\n

Omar Alonso
\nJiang Bian<\/a>
\nYuxiao Dong<\/a>
\nAdam Fourney<\/a>
\nAhmed Hassan Awadallah<\/a>
\nJulia Kiseleva
\nArnd Christian K\u00f6nig<\/a>
\nJonathan Litz
\nHao Ma
\nElnaz Nouri<\/a>
\nRobert Sim<\/a>
\nRuihua Song<\/a>
\nAdith Swaminathan<\/a>
\nChi Wang<\/a>
\nFangzhao Wu<\/a>
\nChenyan Xiong<\/a>
\nElad Yom-Tov<\/a>
\nYizhe Zhang<\/a>
\nImed Zitouni<\/a><\/p>\n

Workshop Organizers<\/h3>\n

Task Intelligence Workshop<\/h4>\n

Ahmed Hassan Awadallah<\/a>
\nRyen W. White<\/a><\/p>\n

Keynote<\/h3>\n

Tuesday, February 12, 2019
\nAttending to What Matters<\/h4>\n

Jaime Teevan<\/a>Opens in a new tab<\/span><\/p>\n

Tuesday, February 12, 2019 | 10:00 AM\u201310:30 AM | Session 1: Search and Ranking<\/h3>\n

MSA: Jointly Detecting Drug Name and Adverse Drug Reaction Mentioning Tweets with Multi-Head Self-Attention<\/h4>\n

Chuhan Wu, Fangzhao Wu<\/a>,<\/strong> Zhigang Yuan, Junxin Liu, Yongfeng Huang,<\/strong> Xing Xie<\/a><\/p>\n

 <\/p>\n

Tuesday, February 12, 2019 | 11:00 AM\u201312:30 PM | Session 2: Knowledge Graphs and Analytics<\/h3>\n

Clustered Monotone Transforms for Rating Factorization<\/h4>\n

Raghav Somani,<\/strong> Gaurush Hiranandani, Oluwasanmi Koyejo, Sreangsu Acharyya<\/strong><\/p>\n

 <\/p>\n

Wednesday, February 13, 2019 | 10:00 AM\u201310:30 PM | Session 5: Understanding Conversation, Discussion, and Opinions<\/h3>\n

Multi-Representation Fusion Network for Multi-Turn Response Selection in Retrieval-Based Chatbots<\/h4>\n

Chongyang Tao, Wei Wu<\/a>, Can Xu,<\/strong> Wenpeng Hu, Dongyan Zhao, Rui Yan<\/p>\n

Attitude Detection for One-Round Conversation: Jointly Extracting Target-Polarity Pairs<\/h4>\n

Zhaohao Zeng, Ruihua Song<\/a>, Pingping Lin,<\/strong> Tetsuya Sakai<\/p>\n

 <\/p>\n

Wednesday, February 13, 2019 | 4:15 PM-5:30 PM | Session 8: Counterfactual and Causal Learning<\/h3>\n

Genie: An Open Box Counterfactual Policy Estimator for Optimizing Sponsored Search Marketplace<\/h4>\n

Murat Ali Bayir<\/strong>, Mingsen Xu<\/strong>, Yaojia Zhu<\/strong>, Yifan Shi<\/strong><\/p>\n

 <\/p>\n

Thursday, February 14, 2019 | 9:00 AM\u201310:30 AM | Session 9: Recommendation<\/h3>\n

Slice: Scalable Linear Extreme Classifiers trained on 100 Million Labels for Related Searches<\/a><\/h4>\n

*Best Paper Award<\/em><\/p>\n

Himanshu Jain, Venkatesh Balasubramanian<\/strong>, Bhanu Teja Chunduri<\/strong>, Manik Varma<\/a>
\n<\/strong><\/p>\n

Neural Tensor Factorization for Temporal Interaction Learning<\/h4>\n

Xian Wu, Baoxu Shi, Yuxiao Dong<\/a>, Chao Huang, Nitesh Chawla<\/p>\n

Shaping Feedback Data in Recommender Systems with Interventions Based on Information Foraging Theory<\/h4>\n

Tobias Schnabel<\/a>, Paul Bennett<\/a>,<\/strong> Thorsten Joachims<\/p>\n

 <\/p>\n

Thursday, February 14, 2019 | 11:00 AM\u201311:45 AM | Session 10: Personalization and Characterizing User Behavior<\/h3>\n

Characterizing and Predicting Email Deferral Behaviour<\/a><\/h4>\n

Bahareh Sarrafzadeh, Ahmed Hassan Awadallah<\/a>, Christopher Lin<\/strong>, Chia-Jung Lee<\/strong>, Milad Shokouhi<\/a>, Susan Dumais<\/a><\/p>\n

Task Duration Estimation<\/a><\/h4>\n

Ryen W. White<\/a>, Ahmed Hassan Awadallah<\/a><\/p>\n

Neural Demographic Prediction using Search Query<\/h4>\n

Chuhan Wu, Fangzhao Wu<\/a>, Junxin Liu, Shaojian He, <\/strong>Yongfeng Huang, Xing Xie<\/a>
\n<\/strong><\/p>\n

 <\/p>\n

Thursday, February 14, 2019 | 2:00 PM\u20133:25 PM | Session 11: Domain Transfer and Representation Learning<\/h3>\n

Domain Adaptation for Commitment Detection in Email<\/h4>\n

Hosein Azarbonyad, Robert Sim<\/a>, Ryen W. White<\/a>
\n<\/strong><\/p>\n

Opens in a new tab<\/span><\/p>\n

Demonstrations<\/h2>\n

clstk: The Cross-Lingual Summarization Tool-Kit<\/h3>\n

Tuesday, February 12 | 3:30 PM\u20134:15 PM
\nNisarg Jhaveri<\/strong>, Manish Gupta<\/a>, Vasudeva Varma<\/p>\n

Half-Day Tutorials<\/h2>\n

Causal Inference and Counterfactual Reasoning<\/h3>\n

Monday, February 11 | 9:00 AM\u201312:30 PM
\nEmre Kiciman<\/a>, Amit Sharma<\/a><\/p>\n

As computing systems are more frequently and more actively intervening to improve people\u2019s work and daily lives, it is critical to correctly predict and understand the causal effects of these interventions. This tutorial will introduce participants to concepts in causal inference and counterfactual reasoning, drawing from abroad literature from statistics, social sciences and machine learning. To tackle such questions, we will introduce the key ingredient that causal analysis depends on\u2014counterfactual reasoning\u2014and describe the two most popular frameworks based on Bayesian graphical models and potential outcomes. Based on this, we will cover a range of methods suitable for doing causal inference with large-scale online data, including randomized experiments, observational methods like matching and stratification, and natural experiment-based methods such as instrumental variables and regression discontinuity. We will also focus on best practices for evaluation and validation of causal inference techniques, drawing from our own experiences.<\/p>\n

Fairness-Aware Machine Learning: Practical Challenges and Lessons Learned<\/h3>\n

Monday, February 11 | 1:30 PM\u20135:00 PM
\nSarah Bird, Krishnaram Kenthapadi, Emre Kiciman<\/a>, Margaret Mitchell<\/p>\n

Researchers and practitioners from different disciplines have highlighted the ethical and legal challenges posed by the use of machine-learned models and data-driven systems, and the potential for such systems to discriminate against certain population groups, due to biases in algorithmic decision-making systems. This tutorial presents an overview of algorithmic bias\/discrimination issues observed over the last few years and the lessons learned key regulations and laws, and evolution of techniques for achieving fairness in machine learning systems. We will motivate the need for adopting a “fairness by design” approach (as opposed to viewing algorithmic bias\/fairness considerations as an afterthought) when developing machine learning based models and systems for different consumer and enterprise applications. Then, we will focus on the application of fairness-aware machine learning techniques in practice, by presenting non-proprietary case studies from different technology companies. Finally, based on our experiences working on fairness in machine learning at companies such as Facebook, Google, LinkedIn, and Microsoft, we will present open problems and research directions for the data mining\/machine learning community.Opens in a new tab<\/span><\/p>\n

Industry Day (opens in new tab)<\/span><\/a><\/h2>\n

A Case Study on Microsoft\u2019s Ruuh.ai: Is User Growth a Peril to Research Progress?<\/h3>\n

Monday, February 11 | 11:00 AM\u201312:30 PM
\nPuneet Agrawal<\/strong><\/p>\n

Striking a balance between business goals such as user growth and deep meaningful research is always a challenging task in an industrial research setting. In this talk, taking Microsoft\u2019s Ruuh as a case study, we will discuss the challenges and opportunities in the industry when it comes to research. Microsoft\u2019s Ruuh was conceptualized about 2.5 years back and the main product promise of Ruuh is to be able to talk to its users on any subject they choose. We realized that the promise meant thinking beyond the utilitarian notion of merely generating \u201crelevant\u201d responses and enabling Ruuh to comprehend and meet a wider range of user social needs, like expressing happiness when user’s favorite team wins, sharing a cute comment on showing the pictures of the user’s pet and so on. At the onset, this seems an impossible task to achieve coupled with aggressive release deadline and pressure to grow usage. However, in this talk we will discuss how our research progress helped the user growth and vice versa, and also discuss scenarios where we suffered setbacks. A good quality product leads to high usage which in turn provides the much-needed data to improve the research and understand the flaws in the current approach. At the same time, high usage of the product forces the team to focus on the efficiency, cost per query and other infrastructure related workloads. This talk will take real-world examples and explain these tradeoffs. More details of the talk are presented in last section.<\/p>\n

\u2018No Interaction\u2019 as Indicator of Search Satisfaction: Accounting for Good Abandonment in User Success Metrics<\/h3>\n

Monday, February 11 | 11:00 AM\u201312:30 PM
\nWidad Machmouchi<\/strong><\/p>\n

At Bing, measuring user success has always been a deciding factor as to which feature or change is shipped to production. Testing such changes is carried out through randomized controlled experiments, where success metrics are used to measure the treatment effect on user satisfaction. Over the years, we have designed and refined our metrics to capture various user interactions, from search queries to clicks and hovers, and interpreted them to predict users\u2019 satisfaction with the search engine. One of the main scenarios that is hard to interpret is search result page abandonment, where the user doesn\u2019t click on the page or interact with any specific element. In this scenario of abandonment, we need to differentiate cases where the user abandoned due to getting the information they need without clicking on any results, from those where the user abandoned due to a defective and\/or unsatisfactory search result page. In this talk, we outline Bing\u2019s journey in addressing this measurement problem. We talk about our initial effort of considering the presence of specific elements on the page as indicator of success; to our offline\/online hybrid approach to identify good abandonment; and finally, to a fully-online solution that relies on a user\u2019s behavior across their search session. We also cover the pitfalls of the different approaches, how we evaluate them and the current challenges and problems left to solve.Opens in a new tab<\/span><\/p>\n

Half-Day Workshops<\/h2>\n

Task Intelligence Workshop (opens in new tab)<\/span><\/a><\/h3>\n

Friday, February 15 | 9:00 AM\u20132:00 PM<\/p>\n

Organizers: Ahmed Hassan-Awadallah<\/a>, Cathal Gurrin, Mark Sanderson, Ryen W. White<\/a><\/p>\n

Invited Speaker: Paul Bennett<\/a><\/p>\n

Accepted Paper: Supporting Complex Tasks Using Multiple Devices<\/a> (Elnaz Nouri<\/a>, Adam Fourney<\/a>, Robert Sim<\/a>, Ryen W. White<\/a>)<\/p>\n

Tasks are defined pieces of work that range in scope from specific (sending an email) to broad (planning a wedding) and are central to all aspects of information access and use. Task intelligence spans technologies and experiences to extract, understand, and support the completion of short- and long-term tasks. Helping users complete tasks is a key capability of search systems, digital assistants, and productivity applications and poses core challenges in data mining and knowledge representation and draws on additional research from areas such as machine learning and natural language processing. The workshop will comprise a mixture of research paper presentations, reports from data challenge participants, including system demonstrations if available.Opens in a new tab<\/span><\/p>\n

\n

\n\t
\n\t\t

\n\t\t\t\t\t\t\tWSDM Conference Analytics<\/a>\n\t\t\t\t\t<\/h3>\n\n\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\tMicrosoft Academic | February 1, 2019<\/p>\n

The Microsoft Academic Graph makes it possible to gain analytic insights about any of the entities within it: publications, authors, institutions, topics, journals, and conferences. \t\t\t<\/div>\n\t\t<\/div>\n\n\t<\/div>\n<\/article>\nOpens in a new tab<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Microsoft is excited to be a Silver sponsor of the 12th ACM International conference on Web Search and Data Mining Come by our booth to chat with our experts, see demos of our latest research and find out about career opportunities with Microsoft.<\/p>\n","protected":false},"featured_media":558870,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_startdate":"2019-02-11","msr_enddate":"2019-02-15","msr_location":"Melbourne, Australia","msr_expirationdate":"","msr_event_recording_link":"","msr_event_link":"http:\/\/www.wsdm-conference.org\/2019\/registration.php","msr_event_link_redirect":false,"msr_event_time":"","msr_hide_region":false,"msr_private_event":false,"msr_hide_image_in_river":0,"footnotes":""},"research-area":[13555],"msr-region":[197903],"msr-event-type":[197941],"msr-video-type":[],"msr-locale":[268875],"msr-program-audience":[],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-558867","msr-event","type-msr-event","status-publish","has-post-thumbnail","hentry","msr-research-area-search-information-retrieval","msr-region-asia-pacific","msr-event-type-conferences","msr-locale-en_us"],"msr_about":"\n\n

Venue:<\/strong> Melbourne Convention and Exhibition Centre (opens in new tab)<\/span><\/a>
\n1 Convention Centre Place, South Wharf 3006<\/p>\n

Website:<\/strong> WSDM 2019 (opens in new tab)<\/span><\/a>Opens in a new tab<\/span><\/p>\n

Microsoft is excited to be a Silver sponsor of the 12th<\/sup> ACM International conference on Web Search and Data Mining<\/a>. Come by our booth to chat with our experts, see demos of our latest research and find out about career opportunities with Microsoft.<\/p>\n

Microsoft Attendees<\/h3>\n

Puneet Agrawal
\nAhmed Awadallah<\/a>
\nMurat Bayir
\nPaul Bennett<\/a>
\nSusan Dumais<\/a>
\nManish Gupta<\/a>
\nEmre Kiciman<\/a>
\nWidad Machmouchi
\nTobias Schnabel<\/a>
\nAmit Sharma<\/a>
\nRaghav Somani
\nJaime Teevan<\/a>
\nKuansan Wang<\/a>
\nRyen White<\/a>
\nWei Wu<\/a>
\nRoy Zimmermann<\/a><\/p>\n

Demonstration Chair<\/h3>\n

Paul Thomas<\/a><\/p>\n

Program Committee Co-Chair<\/h3>\n

Paul Bennett<\/a><\/p>\n

Senior Program Committee Members<\/h3>\n

Nick Craswell<\/a>
\nEmre Kiciman<\/a>
\nMilad Shokouhi<\/a>
\nRyen White<\/a>
\nXing Xie<\/a><\/p>\n

Program Committee Members<\/h3>\n

Omar Alonso
\nJiang Bian<\/a>
\nYuxiao Dong<\/a>
\nAdam Fourney<\/a>
\nAhmed Hassan Awadallah<\/a>
\nJulia Kiseleva
\nArnd Christian K\u00f6nig<\/a>
\nJonathan Litz
\nHao Ma
\nElnaz Nouri<\/a>
\nRobert Sim<\/a>
\nRuihua Song<\/a>
\nAdith Swaminathan<\/a>
\nChi Wang<\/a>
\nFangzhao Wu<\/a>
\nChenyan Xiong<\/a>
\nElad Yom-Tov<\/a>
\nYizhe Zhang<\/a>
\nImed Zitouni<\/a><\/p>\n

Workshop Organizers<\/h3>\n

Task Intelligence Workshop<\/h4>\n

Ahmed Hassan Awadallah<\/a>
\nRyen W. White<\/a><\/p>\n

Keynote<\/h3>\n

Tuesday, February 12, 2019
\nAttending to What Matters<\/h4>\n

Jaime Teevan<\/a>Opens in a new tab<\/span><\/p>\n

Tuesday, February 12, 2019 | 10:00 AM\u201310:30 AM | Session 1: Search and Ranking<\/h3>\n

MSA: Jointly Detecting Drug Name and Adverse Drug Reaction Mentioning Tweets with Multi-Head Self-Attention<\/h4>\n

Chuhan Wu, Fangzhao Wu<\/a>,<\/strong> Zhigang Yuan, Junxin Liu, Yongfeng Huang,<\/strong> Xing Xie<\/a><\/p>\n

 <\/p>\n

Tuesday, February 12, 2019 | 11:00 AM\u201312:30 PM | Session 2: Knowledge Graphs and Analytics<\/h3>\n

Clustered Monotone Transforms for Rating Factorization<\/h4>\n

Raghav Somani,<\/strong> Gaurush Hiranandani, Oluwasanmi Koyejo, Sreangsu Acharyya<\/strong><\/p>\n

 <\/p>\n

Wednesday, February 13, 2019 | 10:00 AM\u201310:30 PM | Session 5: Understanding Conversation, Discussion, and Opinions<\/h3>\n

Multi-Representation Fusion Network for Multi-Turn Response Selection in Retrieval-Based Chatbots<\/h4>\n

Chongyang Tao, Wei Wu<\/a>, Can Xu,<\/strong> Wenpeng Hu, Dongyan Zhao, Rui Yan<\/p>\n

Attitude Detection for One-Round Conversation: Jointly Extracting Target-Polarity Pairs<\/h4>\n

Zhaohao Zeng, Ruihua Song<\/a>, Pingping Lin,<\/strong> Tetsuya Sakai<\/p>\n

 <\/p>\n

Wednesday, February 13, 2019 | 4:15 PM-5:30 PM | Session 8: Counterfactual and Causal Learning<\/h3>\n

Genie: An Open Box Counterfactual Policy Estimator for Optimizing Sponsored Search Marketplace<\/h4>\n

Murat Ali Bayir<\/strong>, Mingsen Xu<\/strong>, Yaojia Zhu<\/strong>, Yifan Shi<\/strong><\/p>\n

 <\/p>\n

Thursday, February 14, 2019 | 9:00 AM\u201310:30 AM | Session 9: Recommendation<\/h3>\n

Slice: Scalable Linear Extreme Classifiers trained on 100 Million Labels for Related Searches<\/a><\/h4>\n

*Best Paper Award<\/em><\/p>\n

Himanshu Jain, Venkatesh Balasubramanian<\/strong>, Bhanu Teja Chunduri<\/strong>, Manik Varma<\/a>
\n<\/strong><\/p>\n

Neural Tensor Factorization for Temporal Interaction Learning<\/h4>\n

Xian Wu, Baoxu Shi, Yuxiao Dong<\/a>, Chao Huang, Nitesh Chawla<\/p>\n

Shaping Feedback Data in Recommender Systems with Interventions Based on Information Foraging Theory<\/h4>\n

Tobias Schnabel<\/a>, Paul Bennett<\/a>,<\/strong> Thorsten Joachims<\/p>\n

 <\/p>\n

Thursday, February 14, 2019 | 11:00 AM\u201311:45 AM | Session 10: Personalization and Characterizing User Behavior<\/h3>\n

Characterizing and Predicting Email Deferral Behaviour<\/a><\/h4>\n

Bahareh Sarrafzadeh, Ahmed Hassan Awadallah<\/a>, Christopher Lin<\/strong>, Chia-Jung Lee<\/strong>, Milad Shokouhi<\/a>, Susan Dumais<\/a><\/p>\n

Task Duration Estimation<\/a><\/h4>\n

Ryen W. White<\/a>, Ahmed Hassan Awadallah<\/a><\/p>\n

Neural Demographic Prediction using Search Query<\/h4>\n

Chuhan Wu, Fangzhao Wu<\/a>, Junxin Liu, Shaojian He, <\/strong>Yongfeng Huang, Xing Xie<\/a>
\n<\/strong><\/p>\n

 <\/p>\n

Thursday, February 14, 2019 | 2:00 PM\u20133:25 PM | Session 11: Domain Transfer and Representation Learning<\/h3>\n

Domain Adaptation for Commitment Detection in Email<\/h4>\n

Hosein Azarbonyad, Robert Sim<\/a>, Ryen W. White<\/a>
\n<\/strong><\/p>\n

Opens in a new tab<\/span><\/p>\n

Demonstrations<\/h2>\n

clstk: The Cross-Lingual Summarization Tool-Kit<\/h3>\n

Tuesday, February 12 | 3:30 PM\u20134:15 PM
\nNisarg Jhaveri<\/strong>, Manish Gupta<\/a>, Vasudeva Varma<\/p>\n

Half-Day Tutorials<\/h2>\n

Causal Inference and Counterfactual Reasoning<\/h3>\n

Monday, February 11 | 9:00 AM\u201312:30 PM
\nEmre Kiciman<\/a>, Amit Sharma<\/a><\/p>\n

As computing systems are more frequently and more actively intervening to improve people\u2019s work and daily lives, it is critical to correctly predict and understand the causal effects of these interventions. This tutorial will introduce participants to concepts in causal inference and counterfactual reasoning, drawing from abroad literature from statistics, social sciences and machine learning. To tackle such questions, we will introduce the key ingredient that causal analysis depends on\u2014counterfactual reasoning\u2014and describe the two most popular frameworks based on Bayesian graphical models and potential outcomes. Based on this, we will cover a range of methods suitable for doing causal inference with large-scale online data, including randomized experiments, observational methods like matching and stratification, and natural experiment-based methods such as instrumental variables and regression discontinuity. We will also focus on best practices for evaluation and validation of causal inference techniques, drawing from our own experiences.<\/p>\n

Fairness-Aware Machine Learning: Practical Challenges and Lessons Learned<\/h3>\n

Monday, February 11 | 1:30 PM\u20135:00 PM
\nSarah Bird, Krishnaram Kenthapadi, Emre Kiciman<\/a>, Margaret Mitchell<\/p>\n

Researchers and practitioners from different disciplines have highlighted the ethical and legal challenges posed by the use of machine-learned models and data-driven systems, and the potential for such systems to discriminate against certain population groups, due to biases in algorithmic decision-making systems. This tutorial presents an overview of algorithmic bias\/discrimination issues observed over the last few years and the lessons learned key regulations and laws, and evolution of techniques for achieving fairness in machine learning systems. We will motivate the need for adopting a “fairness by design” approach (as opposed to viewing algorithmic bias\/fairness considerations as an afterthought) when developing machine learning based models and systems for different consumer and enterprise applications. Then, we will focus on the application of fairness-aware machine learning techniques in practice, by presenting non-proprietary case studies from different technology companies. Finally, based on our experiences working on fairness in machine learning at companies such as Facebook, Google, LinkedIn, and Microsoft, we will present open problems and research directions for the data mining\/machine learning community.Opens in a new tab<\/span><\/p>\n

Industry Day (opens in new tab)<\/span><\/a><\/h2>\n

A Case Study on Microsoft\u2019s Ruuh.ai: Is User Growth a Peril to Research Progress?<\/h3>\n

Monday, February 11 | 11:00 AM\u201312:30 PM
\nPuneet Agrawal<\/strong><\/p>\n

Striking a balance between business goals such as user growth and deep meaningful research is always a challenging task in an industrial research setting. In this talk, taking Microsoft\u2019s Ruuh as a case study, we will discuss the challenges and opportunities in the industry when it comes to research. Microsoft\u2019s Ruuh was conceptualized about 2.5 years back and the main product promise of Ruuh is to be able to talk to its users on any subject they choose. We realized that the promise meant thinking beyond the utilitarian notion of merely generating \u201crelevant\u201d responses and enabling Ruuh to comprehend and meet a wider range of user social needs, like expressing happiness when user’s favorite team wins, sharing a cute comment on showing the pictures of the user’s pet and so on. At the onset, this seems an impossible task to achieve coupled with aggressive release deadline and pressure to grow usage. However, in this talk we will discuss how our research progress helped the user growth and vice versa, and also discuss scenarios where we suffered setbacks. A good quality product leads to high usage which in turn provides the much-needed data to improve the research and understand the flaws in the current approach. At the same time, high usage of the product forces the team to focus on the efficiency, cost per query and other infrastructure related workloads. This talk will take real-world examples and explain these tradeoffs. More details of the talk are presented in last section.<\/p>\n

\u2018No Interaction\u2019 as Indicator of Search Satisfaction: Accounting for Good Abandonment in User Success Metrics<\/h3>\n

Monday, February 11 | 11:00 AM\u201312:30 PM
\nWidad Machmouchi<\/strong><\/p>\n

At Bing, measuring user success has always been a deciding factor as to which feature or change is shipped to production. Testing such changes is carried out through randomized controlled experiments, where success metrics are used to measure the treatment effect on user satisfaction. Over the years, we have designed and refined our metrics to capture various user interactions, from search queries to clicks and hovers, and interpreted them to predict users\u2019 satisfaction with the search engine. One of the main scenarios that is hard to interpret is search result page abandonment, where the user doesn\u2019t click on the page or interact with any specific element. In this scenario of abandonment, we need to differentiate cases where the user abandoned due to getting the information they need without clicking on any results, from those where the user abandoned due to a defective and\/or unsatisfactory search result page. In this talk, we outline Bing\u2019s journey in addressing this measurement problem. We talk about our initial effort of considering the presence of specific elements on the page as indicator of success; to our offline\/online hybrid approach to identify good abandonment; and finally, to a fully-online solution that relies on a user\u2019s behavior across their search session. We also cover the pitfalls of the different approaches, how we evaluate them and the current challenges and problems left to solve.Opens in a new tab<\/span><\/p>\n

Half-Day Workshops<\/h2>\n

Task Intelligence Workshop<\/a><\/h3>\n

Friday, February 15 | 9:00 AM\u20132:00 PM<\/p>\n

Organizers: Ahmed Hassan-Awadallah<\/a>, Cathal Gurrin, Mark Sanderson, Ryen W. White<\/a><\/p>\n

Invited Speaker: Paul Bennett<\/a><\/p>\n

Accepted Paper: Supporting Complex Tasks Using Multiple Devices<\/a> (Elnaz Nouri<\/a>, Adam Fourney<\/a>, Robert Sim<\/a>, Ryen W. White<\/a>)<\/p>\n

Tasks are defined pieces of work that range in scope from specific (sending an email) to broad (planning a wedding) and are central to all aspects of information access and use. Task intelligence spans technologies and experiences to extract, understand, and support the completion of short- and long-term tasks. Helping users complete tasks is a key capability of search systems, digital assistants, and productivity applications and poses core challenges in data mining and knowledge representation and draws on additional research from areas such as machine learning and natural language processing. The workshop will comprise a mixture of research paper presentations, reports from data challenge participants, including system demonstrations if available.Opens in a new tab<\/span><\/p>\n

\n

\n\t
\n\t\t

\n\t\t\t\t\t\t\tWSDM Conference Analytics<\/a>\n\t\t\t\t\t<\/h3>\n\n\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\tMicrosoft Academic | February 1, 2019<\/p>\n

The Microsoft Academic Graph makes it possible to gain analytic insights about any of the entities within it: publications, authors, institutions, topics, journals, and conferences. \t\t\t<\/div>\n\t\t<\/div>\n\n\t<\/div>\n<\/article>\nOpens in a new tab<\/span><\/p>\n","tab-content":[{"id":0,"name":"About","content":"Microsoft is excited to be a Silver sponsor of the 12th<\/sup> ACM International conference on Web Search and Data Mining<\/a>. Come by our booth to chat with our experts, see demos of our latest research and find out about career opportunities with Microsoft.\r\n

Microsoft Attendees<\/h3>\r\nPuneet Agrawal\r\nAhmed Awadallah<\/a>\r\nMurat Bayir\r\nPaul Bennett<\/a>\r\nSusan Dumais<\/a>\r\nManish Gupta<\/a>\r\nEmre Kiciman<\/a>\r\nWidad Machmouchi\r\nTobias Schnabel<\/a>\r\nAmit Sharma<\/a>\r\nRaghav Somani\r\nJaime Teevan<\/a>\r\nKuansan Wang<\/a>\r\nRyen White<\/a>\r\nWei Wu<\/a>\r\nRoy Zimmermann<\/a>\r\n

Demonstration Chair<\/h3>\r\nPaul Thomas<\/a>\r\n

Program Committee Co-Chair<\/h3>\r\nPaul Bennett<\/a>\r\n

Senior Program Committee Members<\/h3>\r\nNick Craswell<\/a>\r\nEmre Kiciman<\/a>\r\nMilad Shokouhi<\/a>\r\nRyen White<\/a>\r\nXing Xie<\/a>\r\n

Program Committee Members<\/h3>\r\nOmar Alonso\r\nJiang Bian<\/a>\r\nYuxiao Dong<\/a>\r\nAdam Fourney<\/a>\r\nAhmed Hassan Awadallah<\/a>\r\nJulia Kiseleva\r\nArnd Christian K\u00f6nig<\/a>\r\nJonathan Litz\r\nHao Ma\r\nElnaz Nouri<\/a>\r\nRobert Sim<\/a>\r\nRuihua Song<\/a>\r\nAdith Swaminathan<\/a>\r\nChi Wang<\/a>\r\nFangzhao Wu<\/a>\r\nChenyan Xiong<\/a>\r\nElad Yom-Tov<\/a>\r\nYizhe Zhang<\/a>\r\nImed Zitouni<\/a>\r\n

Workshop Organizers<\/h3>\r\n

Task Intelligence Workshop<\/h4>\r\nAhmed Hassan Awadallah<\/a>\r\nRyen W. White<\/a>\r\n

Keynote<\/h3>\r\n

Tuesday, February 12, 2019\r\nAttending to What Matters<\/h4>\r\nJaime Teevan<\/a>"},{"id":1,"name":"Accepted Papers","content":"

Tuesday, February 12, 2019 | 10:00 AM\u201310:30 AM | Session 1: Search and Ranking<\/h3>\r\n

MSA: Jointly Detecting Drug Name and Adverse Drug Reaction Mentioning Tweets with Multi-Head Self-Attention<\/h4>\r\n

Chuhan Wu, Fangzhao Wu<\/a>,<\/strong> Zhigang Yuan, Junxin Liu, Yongfeng Huang,<\/strong> Xing Xie<\/a><\/p>\r\n \r\n

Tuesday, February 12, 2019 | 11:00 AM\u201312:30 PM | Session 2: Knowledge Graphs and Analytics<\/h3>\r\n

Clustered Monotone Transforms for Rating Factorization<\/h4>\r\n

Raghav Somani,<\/strong> Gaurush Hiranandani, Oluwasanmi Koyejo, Sreangsu Acharyya<\/strong><\/p>\r\n \r\n

Wednesday, February 13, 2019 | 10:00 AM\u201310:30 PM | Session 5: Understanding Conversation, Discussion, and Opinions<\/h3>\r\n

Multi-Representation Fusion Network for Multi-Turn Response Selection in Retrieval-Based Chatbots<\/h4>\r\n

Chongyang Tao, Wei Wu<\/a>, Can Xu,<\/strong> Wenpeng Hu, Dongyan Zhao, Rui Yan<\/p>\r\n\r\n

Attitude Detection for One-Round Conversation: Jointly Extracting Target-Polarity Pairs<\/h4>\r\n

Zhaohao Zeng, Ruihua Song<\/a>, Pingping Lin,<\/strong> Tetsuya Sakai<\/p>\r\n \r\n

Wednesday, February 13, 2019 | 4:15 PM-5:30 PM | Session 8: Counterfactual and Causal Learning<\/h3>\r\n

Genie: An Open Box Counterfactual Policy Estimator for Optimizing Sponsored Search Marketplace<\/h4>\r\n

Murat Ali Bayir<\/strong>, Mingsen Xu<\/strong>, Yaojia Zhu<\/strong>, Yifan Shi<\/strong><\/p>\r\n \r\n

Thursday, February 14, 2019 | 9:00 AM\u201310:30 AM | Session 9: Recommendation<\/h3>\r\n

Slice: Scalable Linear Extreme Classifiers trained on 100 Million Labels for Related Searches<\/a><\/h4>\r\n*Best Paper Award<\/em>\r\n

Himanshu Jain, Venkatesh Balasubramanian<\/strong>, Bhanu Teja Chunduri<\/strong>, Manik Varma<\/a>\r\n<\/strong><\/p>\r\n\r\n

Neural Tensor Factorization for Temporal Interaction Learning<\/h4>\r\n

Xian Wu, Baoxu Shi, Yuxiao Dong<\/a>, Chao Huang, Nitesh Chawla<\/p>\r\n\r\n

Shaping Feedback Data in Recommender Systems with Interventions Based on Information Foraging Theory<\/h4>\r\n

Tobias Schnabel<\/a>, Paul Bennett<\/a>,<\/strong> Thorsten Joachims<\/p>\r\n \r\n

Thursday, February 14, 2019 | 11:00 AM\u201311:45 AM | Session 10: Personalization and Characterizing User Behavior<\/h3>\r\n

Characterizing and Predicting Email Deferral Behaviour<\/a><\/h4>\r\n

Bahareh Sarrafzadeh, Ahmed Hassan Awadallah<\/a>, Christopher Lin<\/strong>, Chia-Jung Lee<\/strong>, Milad Shokouhi<\/a>, Susan Dumais<\/a><\/p>\r\n\r\n

Task Duration Estimation<\/a><\/h4>\r\n

Ryen W. White<\/a>, Ahmed Hassan Awadallah<\/a><\/p>\r\n\r\n

Neural Demographic Prediction using Search Query<\/h4>\r\n

Chuhan Wu, Fangzhao Wu<\/a>, Junxin Liu, Shaojian He, <\/strong>Yongfeng Huang, Xing Xie<\/a>\r\n<\/strong><\/p>\r\n \r\n

Thursday, February 14, 2019 | 2:00 PM\u20133:25 PM | Session 11: Domain Transfer and Representation Learning<\/h3>\r\n

Domain Adaptation for Commitment Detection in Email<\/h4>\r\n

Hosein Azarbonyad, Robert Sim<\/a>, Ryen W. White<\/a>\r\n<\/strong><\/p>"},{"id":2,"name":"Demos & Tutorials","content":"

Demonstrations<\/h2>\r\n

clstk: The Cross-Lingual Summarization Tool-Kit<\/h3>\r\nTuesday, February 12 | 3:30 PM\u20134:15 PM\r\nNisarg Jhaveri<\/strong>, Manish Gupta<\/a>, Vasudeva Varma\r\n

Half-Day Tutorials<\/h2>\r\n

Causal Inference and Counterfactual Reasoning<\/h3>\r\nMonday, February 11 | 9:00 AM\u201312:30 PM\r\nEmre Kiciman<\/a>, Amit Sharma<\/a>\r\n\r\nAs computing systems are more frequently and more actively intervening to improve people\u2019s work and daily lives, it is critical to correctly predict and understand the causal effects of these interventions. This tutorial will introduce participants to concepts in causal inference and counterfactual reasoning, drawing from abroad literature from statistics, social sciences and machine learning. To tackle such questions, we will introduce the key ingredient that causal analysis depends on\u2014counterfactual reasoning\u2014and describe the two most popular frameworks based on Bayesian graphical models and potential outcomes. Based on this, we will cover a range of methods suitable for doing causal inference with large-scale online data, including randomized experiments, observational methods like matching and stratification, and natural experiment-based methods such as instrumental variables and regression discontinuity. We will also focus on best practices for evaluation and validation of causal inference techniques, drawing from our own experiences.\r\n

Fairness-Aware Machine Learning: Practical Challenges and Lessons Learned<\/h3>\r\nMonday, February 11 | 1:30 PM\u20135:00 PM\r\nSarah Bird, Krishnaram Kenthapadi, Emre Kiciman<\/a>, Margaret Mitchell\r\n\r\nResearchers and practitioners from different disciplines have highlighted the ethical and legal challenges posed by the use of machine-learned models and data-driven systems, and the potential for such systems to discriminate against certain population groups, due to biases in algorithmic decision-making systems. This tutorial presents an overview of algorithmic bias\/discrimination issues observed over the last few years and the lessons learned key regulations and laws, and evolution of techniques for achieving fairness in machine learning systems. We will motivate the need for adopting a \"fairness by design\" approach (as opposed to viewing algorithmic bias\/fairness considerations as an afterthought) when developing machine learning based models and systems for different consumer and enterprise applications. Then, we will focus on the application of fairness-aware machine learning techniques in practice, by presenting non-proprietary case studies from different technology companies. Finally, based on our experiences working on fairness in machine learning at companies such as Facebook, Google, LinkedIn, and Microsoft, we will present open problems and research directions for the data mining\/machine learning community."},{"id":3,"name":"Industry Day","content":"

Industry Day<\/a><\/h2>\r\n

A Case Study on Microsoft\u2019s Ruuh.ai: Is User Growth a Peril to Research Progress?<\/h3>\r\nMonday, February 11 | 11:00 AM\u201312:30 PM\r\nPuneet Agrawal<\/strong>\r\n\r\nStriking a balance between business goals such as user growth and deep meaningful research is always a challenging task in an industrial research setting. In this talk, taking Microsoft\u2019s Ruuh as a case study, we will discuss the challenges and opportunities in the industry when it comes to research. Microsoft\u2019s Ruuh was conceptualized about 2.5 years back and the main product promise of Ruuh is to be able to talk to its users on any subject they choose. We realized that the promise meant thinking beyond the utilitarian notion of merely generating \u201crelevant\u201d responses and enabling Ruuh to comprehend and meet a wider range of user social needs, like expressing happiness when user's favorite team wins, sharing a cute comment on showing the pictures of the user's pet and so on. At the onset, this seems an impossible task to achieve coupled with aggressive release deadline and pressure to grow usage. However, in this talk we will discuss how our research progress helped the user growth and vice versa, and also discuss scenarios where we suffered setbacks. A good quality product leads to high usage which in turn provides the much-needed data to improve the research and understand the flaws in the current approach. At the same time, high usage of the product forces the team to focus on the efficiency, cost per query and other infrastructure related workloads. This talk will take real-world examples and explain these tradeoffs. More details of the talk are presented in last section.\r\n

\u2018No Interaction\u2019 as Indicator of Search Satisfaction: Accounting for Good Abandonment in User Success Metrics<\/h3>\r\nMonday, February 11 | 11:00 AM\u201312:30 PM\r\nWidad Machmouchi<\/strong>\r\n\r\nAt Bing, measuring user success has always been a deciding factor as to which feature or change is shipped to production. Testing such changes is carried out through randomized controlled experiments, where success metrics are used to measure the treatment effect on user satisfaction. Over the years, we have designed and refined our metrics to capture various user interactions, from search queries to clicks and hovers, and interpreted them to predict users\u2019 satisfaction with the search engine. One of the main scenarios that is hard to interpret is search result page abandonment, where the user doesn\u2019t click on the page or interact with any specific element. In this scenario of abandonment, we need to differentiate cases where the user abandoned due to getting the information they need without clicking on any results, from those where the user abandoned due to a defective and\/or unsatisfactory search result page. In this talk, we outline Bing\u2019s journey in addressing this measurement problem. We talk about our initial effort of considering the presence of specific elements on the page as indicator of success; to our offline\/online hybrid approach to identify good abandonment; and finally, to a fully-online solution that relies on a user\u2019s behavior across their search session. We also cover the pitfalls of the different approaches, how we evaluate them and the current challenges and problems left to solve."},{"id":4,"name":"Workshops","content":"

Half-Day Workshops<\/h2>\r\n

Task Intelligence Workshop<\/a><\/h3>\r\nFriday, February 15 | 9:00 AM\u20132:00 PM\r\n\r\nOrganizers: Ahmed Hassan-Awadallah<\/a>, Cathal Gurrin, Mark Sanderson, Ryen W. White<\/a>\r\n\r\nInvited Speaker: Paul Bennett<\/a>\r\n\r\nAccepted Paper: Supporting Complex Tasks Using Multiple Devices<\/a> (Elnaz Nouri<\/a>, Adam Fourney<\/a>, Robert Sim<\/a>, Ryen W. White<\/a>)\r\n\r\nTasks are defined pieces of work that range in scope from specific (sending an email) to broad (planning a wedding) and are central to all aspects of information access and use. Task intelligence spans technologies and experiences to extract, understand, and support the completion of short- and long-term tasks. Helping users complete tasks is a key capability of search systems, digital assistants, and productivity applications and poses core challenges in data mining and knowledge representation and draws on additional research from areas such as machine learning and natural language processing. The workshop will comprise a mixture of research paper presentations, reports from data challenge participants, including system demonstrations if available."},{"id":5,"name":"Analytics","content":"[card title=\"WSDM Conference Analytics\" url=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/academic\/articles\/wsdm-conference-analytics\" ]Microsoft Academic | February 1, 2019\r\n\r\nThe Microsoft Academic Graph makes it possible to gain analytic insights about any of the entities within it: publications, authors, institutions, topics, journals, and conferences. [\/card]"}],"msr_startdate":"2019-02-11","msr_enddate":"2019-02-15","msr_event_time":"","msr_location":"Melbourne, Australia","msr_event_link":"http:\/\/www.wsdm-conference.org\/2019\/registration.php","msr_event_recording_link":"","msr_startdate_formatted":"February 11, 2019","msr_register_text":"Watch now","msr_cta_link":"http:\/\/www.wsdm-conference.org\/2019\/registration.php","msr_cta_text":"Watch now","msr_cta_bi_name":"Event Register","featured_image_thumbnail":"\"Sunrise","event_excerpt":"Microsoft is excited to be a Silver sponsor of the 12th ACM International conference on Web Search and Data Mining Come by our booth to chat with our experts, see demos of our latest research and find out about career opportunities with Microsoft.","msr_research_lab":[199560,199562,199563,199565,212740],"related-researchers":[],"msr_impact_theme":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-opportunities":[],"related-publications":[],"related-videos":[],"related-posts":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/558867","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-event"}],"version-history":[{"count":4,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/558867\/revisions"}],"predecessor-version":[{"id":1147058,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event\/558867\/revisions\/1147058"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/558870"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=558867"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=558867"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=558867"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=558867"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=558867"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=558867"},{"taxonomy":"msr-program-audience","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-program-audience?post=558867"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=558867"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=558867"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}