{"id":656325,"date":"2020-06-02T09:40:48","date_gmt":"2020-06-02T16:40:48","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=656325"},"modified":"2025-03-31T18:33:10","modified_gmt":"2025-04-01T01:33:10","slug":"econml","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/econml\/","title":{"rendered":"EconML"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background-catalina-blue card-background--inset-right\">\n\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"460\" height=\"120\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/06\/econml-logo-inverse.png\" class=\"attachment-full size-full\" alt=\"EconML logo, with a sequence of rounded colored bars on the left, and the text \"EconML\" on the right in white\" style=\"\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/06\/econml-logo-inverse.png 460w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/06\/econml-logo-inverse-300x78.png 300w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/06\/econml-logo-inverse-240x63.png 240w\" sizes=\"auto, (max-width: 460px) 100vw, 460px\" \/>\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 \">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 class=\"wp-block-heading h2\" id=\"econml\">EconML<\/h1>\n\n\n\n<p>Estimate causal effects with ML<\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/Microsoft\/EconML\" target=\"_blank\" rel=\"noreferrer noopener\">EconML on GitHub<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-outline is-style-outline--1\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/econml.azurewebsites.net\/\" target=\"_blank\" rel=\"noreferrer noopener\">Documentation on EconML<\/a><\/div>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"overview\">Overview<\/h2>\n\n\n\n<p><strong>EconML<\/strong>&nbsp;is a Python package that applies the power of machine learning techniques to estimate individualized causal responses from observational or experimental data. The suite of estimation methods provided in EconML represents the latest advances in causal machine learning. By incorporating individual machine learning steps into interpretable causal models, these methods improve the reliability of what-if predictions and make causal analysis quicker and easier for a broad set of users.<\/p>\n\n\n\n<p>EconML is open-source software developed by the&nbsp;<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/alice\/\">ALICE team<\/a>&nbsp;at Microsoft Research.<\/p>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"92\" height=\"92\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/imgFlexible.png\" alt=\"Flexible icon\" class=\"wp-image-656358\" \/><\/figure>\n\n\n\n<div style=\"height:10px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"has-text-align-center\"><b>Flexible<\/b>\nAllows for flexible model forms that do not impose strong assumptions, including models of heterogenous responses to treatment.\n\n<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"92\" height=\"92\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/imgUnified.png\" alt=\"Unified icon\" class=\"wp-image-656355\" \/><\/figure>\n\n\n\n<div style=\"height:10px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"has-text-align-center\"><b>Unified<\/b>\nBroad set of methods representing latest advances in the econometrics and machine learning literature within a unified API.\n\n<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"92\" height=\"92\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/imgFamiliar.png\" alt=\"Familiar icon\" class=\"wp-image-656352\" \/><\/figure>\n\n\n\n<div style=\"height:10px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"has-text-align-center\"><b>Familiar Interface<\/b>\nBuilt on standard Python packages for machine learning and data analysis.\n\n<\/p>\n<\/div>\n<\/div>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"use-cases\">Use cases<\/h2>\n\n\n\n<p>This toolkit is designed to measure the causal effect of some treatment variable(s) T on an outcome variable Y, controlling for a set of features X. <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/econml\/#!use-cases\">Use cases include<\/a>:<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"340\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Recomendation.png\" alt=\"illustration of woman with an A left of her and a B right of her\" class=\"wp-image-656442\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Recomendation.png 490w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Recomendation-300x208.png 300w\" sizes=\"auto, (max-width: 490px) 100vw, 490px\" \/><\/figure>\n\n\n\n<div style=\"height:7px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"recommendation-a-b-testing\">Recommendation A\/B testing<\/h3>\n\n\n\n<p><em>Interpret experiments with imperfect compliance<\/em><\/p>\n\n\n\n<p><strong>Question:<\/strong>\u00a0A travel website would like to know whether joining a membership program causes users to spend more time engaging with the website.\u00a0<\/p>\n\n\n\n<p><strong>Problem:<\/strong>\u00a0They can\u2019t look directly at existing data, comparing members and non-members, because the customers who chose to become members are likely already more engaged than other users. Nor can they run a direct A\/B test because they can\u2019t force users to sign up for membership.\u00a0<\/p>\n\n\n\n<p><strong>Solution:<\/strong>\u00a0The company had run an earlier experiment to test the value of a new, faster sign-up process. EconML\u2019s\u00a0<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/reference.html#doubly-robust-dr-iv\" target=\"_blank\" rel=\"noopener noreferrer\">DRIV estimator(opens in new tab)<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u00a0uses this experimental nudge towards membership as an instrument that generates random variation in the likelihood of membership. The DRIV model adjusts for the fact that not every customer who was offered the easier sign-up became a member and returns the effect of membership rather than the effect of receiving the quick sign-up.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--2\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/04\/MSR_ALICE_casestudy_2020.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">Trip Advisor Case Study<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Recommendation%20AB%20Testing%20at%20An%20Online%20Travel%20Company.ipynb\" target=\"_blank\" rel=\"noreferrer noopener\">Jupyter Notebook<\/a><\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"340\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Segmentation.png\" alt=\"illustration of a world map, the bust of a man, and an arrow hitting the bullseye of a target\" class=\"wp-image-656445\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Segmentation.png 490w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Segmentation-300x208.png 300w\" sizes=\"auto, (max-width: 490px) 100vw, 490px\" \/><\/figure>\n\n\n\n<div style=\"height:7px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"customer-segmentation\">Customer segmentation<\/h3>\n\n\n\n<p><em>Estimate individualized responses to incentives<\/em><\/p>\n\n\n\n<p><strong>Question:<\/strong>&nbsp;A media subscription service would like to offer targeted discounts through a personalized pricing plan.&nbsp;<\/p>\n\n\n\n<p><strong>Problem:<\/strong>&nbsp;They observe many features of their customers but are not sure which customers will respond most to a lower price.&nbsp;<\/p>\n\n\n\n<p><strong>Solution:<\/strong>\u00a0EconML\u2019s\u00a0<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/spec\/estimation\/dml.html\" target=\"_blank\" rel=\"noopener noreferrer\">DML estimator(opens in new tab)<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u00a0uses price variations in existing data, along with a rich set of user features, to estimate heterogeneous price sensitivities that vary with multiple customer features. The\u00a0<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/reference.html#cate-interpreters\" target=\"_blank\" rel=\"noopener noreferrer\">tree interpreter(opens in new tab)<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u00a0provides a presentation-ready summary of the key features that explain the biggest differences in responsiveness to a discount.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Customer%20Segmentation%20at%20An%20Online%20Media%20Company.ipynb\" target=\"_blank\" rel=\"noreferrer noopener\">Jupyter Notebook<\/a><\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"340\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Attribution.png\" alt=\"illustration of 3 different colored hands each picking a piece from a pie chart\" class=\"wp-image-656448\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Attribution.png 490w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Attribution-300x208.png 300w\" sizes=\"auto, (max-width: 490px) 100vw, 490px\" \/><\/figure>\n\n\n\n<div style=\"height:7px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"multi-investment-attribution\">Multi-investment attribution<\/h3>\n\n\n\n<p><em>Distinguish the effects of multiple outreach efforts<\/em><\/p>\n\n\n\n<p><strong>Question:<\/strong>\u00a0A startup would like to know the most effective approach for recruiting new customers: price discounts, technical support to ease adoption, or a combination of the two.\u00a0<\/p>\n\n\n\n<p><strong>Problem:<\/strong>\u00a0The risk of losing customers makes experiments across outreach efforts too expensive. So far, customers have been offered incentives strategically, for example larger businesses are more likely to get technical support.\u00a0<\/p>\n\n\n\n<p><strong>Solution:<\/strong>\u00a0EconML\u2019s\u00a0<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/spec\/estimation\/dr.html\" target=\"_blank\" rel=\"noopener noreferrer\">Doubly Robust Learner(opens in new tab)<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u00a0model jointly estimates the effects of multiple discrete treatments. The model uses flexible functions of observed customer features to filter out confounding correlations in existing data and deliver the causal effect of each effort on revenue.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--3\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/arxiv.org\/abs\/2103.08390\" target=\"_blank\" rel=\"noreferrer noopener\">Technical Paper<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Multi-investment%20Attribution%20at%20A%20Software%20Company.ipynb\" target=\"_blank\" rel=\"noreferrer noopener\">Jupyter Notebook<\/a><\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/Microsoft\/EconML\" target=\"_blank\" rel=\"noreferrer noopener\">EconML on GitHub<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-outline is-style-outline--4\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/econml.azurewebsites.net\/\" target=\"_blank\" rel=\"noreferrer noopener\">Documentation on EconML<\/a><\/div>\n<\/div>\n\n\n\n\n\n<div>\n<div style=\"background-color: #fff8f2 !important;padding: 25px 25px 10px 25px\">\n<figure><img loading=\"lazy\" decoding=\"async\" class=\"alignleft wp-image-664095 size-full\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/06\/imgGear.png\" alt=\"orange gear icon\" width=\"60\" height=\"60\"><\/figure><p><\/p>\n<h2>Quick Installation<\/h2>\n<p>Install the latest release for <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/pypi.org\/project\/econml\/\" target=\"_blank\" rel=\"noopener noreferrer\">PyPI<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>:&nbsp;<code style=\"background-color: #ffeada;padding: 4px\">pip install econml<\/code><\/p>\n<\/div>\n<div style=\"height: 20px\">&nbsp;<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"for-developers\">For developers<\/h2>\n\n\n\n<p>You can get started by cloning this repository. We use&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/setuptools.readthedocs.io\/en\/latest\/index.html\" target=\"_blank\" rel=\"noopener noreferrer\">setuptools<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;for building and distributing our package. We rely on some recent features of setuptools, so make sure to upgrade to a recent version with&nbsp;<code>pip install setuptools --upgrade<\/code>. Then from your local copy of the repository you can run&nbsp;<code>python setup.py develop<\/code>&nbsp;to get started.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"running-the-tests\">Running the tests<\/h2>\n\n\n\n<p>This project uses&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/docs.pytest.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">pytest<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;for testing. To run tests locally after installing the package, you can use&nbsp;<code>python setup.py pytest<\/code>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"generating-the-documentation\">Generating the documentation<\/h2>\n\n\n\n<p>This project\u2019s documentation is generated via&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/www.sphinx-doc.org\/en\/master\/index.html\" target=\"_blank\" rel=\"noopener noreferrer\">Sphinx<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>. Note that we use&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/graphviz.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">graphviz<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>\u2018s&nbsp;<code>dot<\/code>&nbsp;application to produce some of the images in our documentation, so you should make sure that&nbsp;<code>dot<\/code>&nbsp;is installed and in your path.<\/p>\n\n\n\n<p>To generate a local copy of the documentation from a clone of this repository, just run&nbsp;<code>python setup.py build_sphinx -W -E -a<\/code>, which will build the documentation and place it under the&nbsp;<code>build\/sphinx\/html<\/code>&nbsp;path.<\/p>\n\n\n\n<p>The reStructuredText files that make up the documentation are stored in the&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Microsoft\/EconML\/tree\/master\/doc\" target=\"_blank\" rel=\"noopener noreferrer\">docs directory<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>; module documentation is automatically generated by the Sphinx build process.<\/p>\n\n\n\n<div style=\"background-color: #f2f9ff !important;padding: 25px 25px 10px 25px\">\n<h2>Enviroment<\/h2>\n<p>The econml package works on macOS, Windows, and Linux, and supports Python versions 3.5-3.7. The econml package relies on numpy, scipy, and scikit-learn for most of its underlying numerical computation and machine learning routines, and uses keras for the components built on deep neural networks. If you don\u2019t already have these dependencies installed locally, then installing the econml package from PyPI via pip will also install them.<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/Microsoft\/EconML\" target=\"_blank\" rel=\"noreferrer noopener\">EconML on GitHub<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-outline is-style-outline--5\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/econml.azurewebsites.net\/\" target=\"_blank\" rel=\"noreferrer noopener\">Documentation on EconML<\/a><\/div>\n<\/div>\n\n\n\n\n\n<h2 class=\"wp-block-heading\" id=\"if-you-are-new-to-causal-inference\">If you are new to causal inference<\/h2>\n\n\n\n<p>Causal analysis is used to answer what-if questions. Unlike forecasting, which answers the question of what will happen next if everyone keeps behaving as they have in the past, causal inference answers the question of what would happen next if someone changes behavior, such as pursuing a new pricing strategy for a product or a new treatment for a patient. If you are new to causal inference, our guide will introduce you to the biggest challenges in answering causal questions, how EconML addresses those challenges, and the standard causal terminology we use throughout our SDK and documentation.<\/p>\n\n\n\n<p><iframe loading=\"lazy\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/09\/Causal-Inference-User-Guide-v4-022520.pdf\" width=\"900\" height=\"500\"> <\/iframe><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"have-a-question-in-mind\">Have a question in mind?<\/h2>\n\n\n\n<p>If you are ready to start estimating, our <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/spec\/flowchart.html\" target=\"_blank\" rel=\"noopener noreferrer\">flow chart<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> can guide you to an appropriate estimation model for your question. Also check out our Case Studies for examples that apply these models to real world questions.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/Microsoft\/EconML\" target=\"_blank\" rel=\"noreferrer noopener\">EconML on GitHub<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-outline is-style-outline--6\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/econml.azurewebsites.net\/index.html\" target=\"_blank\" rel=\"noreferrer noopener\">Documentation on EconML<\/a><\/div>\n<\/div>\n\n\n\n\n\n<div class=\"wp-block-media-text has-vertical-margin-small  has-vertical-padding-none  has-media-on-the-right is-stacked-on-mobile is-vertically-aligned-top\" style=\"grid-template-columns:auto 30%\"><div class=\"wp-block-media-text__content\">\n<h2 class=\"wp-block-heading\" id=\"recommendation-a-b-testing-1\">Recommendation A\/B testing<\/h2>\n\n\n\n<p><em>Interpret experiments with imperfect compliance<\/em><\/p>\n\n\n\n<p><strong>Question:<\/strong>&nbsp;A travel website would like to know whether joining a membership program causes users to spend more time engaging with the website.&nbsp;<\/p>\n\n\n\n<p><strong>Problem:<\/strong>&nbsp;They can\u2019t look directly at existing data, comparing members and non-members, because the customers who chose to become members are likely already more engaged than other users. Nor can they run a direct A\/B test because they can\u2019t force users to sign up for membership.&nbsp;<\/p>\n\n\n\n<p><strong>Solution:<\/strong>&nbsp;The company had run an earlier experiment to test the value of a new, faster sign-up process. EconML\u2019s&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.ortho_iv.html\" target=\"_blank\" rel=\"noopener noreferrer\">DRIV estimator<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;uses this experimental nudge towards membership as an instrument that generates random variation in the likelihood of membership. The DRIV model adjusts for the fact that not every customer who was offered the easier sign-up became a member and returns the effect of membership rather than the effect of receiving the quick sign-up.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--7\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/04\/MSR_ALICE_casestudy_2020.pdf\">Trip Advisor Case Study<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Recommendation%20AB%20Testing%20at%20An%20Online%20Travel%20Company.ipynb\">Jupyter Notebook<\/a><\/div>\n<\/div>\n<\/div><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"340\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Recomendation.png\" alt=\"EconML | illustration of woman with an A left of her and a B right of her\" class=\"wp-image-656442 size-full\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Recomendation.png 490w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Recomendation-300x208.png 300w\" sizes=\"auto, (max-width: 490px) 100vw, 490px\" \/><\/figure><\/div>\n\n\n\n<div class=\"wp-block-media-text has-vertical-margin-small  has-vertical-padding-none  is-stacked-on-mobile is-vertically-aligned-top\" style=\"grid-template-columns:30% auto\"><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"340\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Segmentation.png\" alt=\"EconML | segmentation | illustration of a world map, the bust of a man, and an arrow hitting the bullseye of a target\" class=\"wp-image-656445 size-full\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Segmentation.png 490w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Segmentation-300x208.png 300w\" sizes=\"auto, (max-width: 490px) 100vw, 490px\" \/><\/figure><div class=\"wp-block-media-text__content\">\n<h2 class=\"wp-block-heading\" id=\"customer-segmentation-1\">Customer segmentation<\/h2>\n\n\n\n<p><em>Estimate individualized responses to incentives<\/em><\/p>\n\n\n\n<p><strong>Question:<\/strong>&nbsp;A media subscription service would like to offer targeted discounts through a personalized pricing plan.&nbsp;<\/p>\n\n\n\n<p><strong>Problem:<\/strong>&nbsp;They observe many features of their customers but are not sure which customers will respond most to a lower price.&nbsp;<\/p>\n\n\n\n<p><strong>Solution:<\/strong>&nbsp;EconML\u2019s&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.dml.html#module-econml.dml\" target=\"_blank\" rel=\"noopener noreferrer\">DML estimator<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;uses price variations in existing data, along with a rich set of user features, to estimate heterogeneous price sensitivities that vary with multiple customer features. The&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.cate_interpreter.html\" target=\"_blank\" rel=\"noopener noreferrer\">tree interpreter<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;provides a presentation-ready summary of the key features that explain the biggest differences in responsiveness to a discount.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Customer%20Segmentation%20at%20An%20Online%20Media%20Company.ipynb\" target=\"_blank\" rel=\"noreferrer noopener\">Jupyter Notebook<\/a><\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-media-text has-vertical-margin-small  has-vertical-padding-none  has-media-on-the-right is-stacked-on-mobile is-vertically-aligned-top\" style=\"grid-template-columns:auto 30%\"><div class=\"wp-block-media-text__content\">\n<h2 class=\"wp-block-heading\" id=\"multi-investment-attribution-1\">Multi-investment attribution<\/h2>\n\n\n\n<p><em>Distinguish the effects of multiple outreach efforts<\/em><\/p>\n\n\n\n<p><strong>Question:<\/strong>&nbsp;A startup would like to know the most effective approach for recruiting new customers: price discounts, technical support to ease adoption, or a combination of the two.&nbsp;<\/p>\n\n\n\n<p><strong>Problem:<\/strong>&nbsp;The risk of losing customers makes experiments across outreach efforts too expensive. So far, customers have been offered incentives strategically, for example larger businesses are more likely to get technical support.&nbsp;<\/p>\n\n\n\n<p><strong>Solution:<\/strong>&nbsp;EconML\u2019s&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.drlearner.html\" target=\"_blank\" rel=\"noopener noreferrer\">Doubly Robust Learner<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&nbsp;model jointly estimates the effects of multiple discrete treatments. The model uses flexible functions of observed customer features to filter out confounding correlations in existing data and deliver the causal effect of each effort on revenue.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-outline is-style-outline--8\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/arxiv.org\/abs\/2103.08390\" target=\"_blank\" rel=\"noreferrer noopener\">Technical Paper<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Multi-investment%20Attribution%20at%20A%20Software%20Company.ipynb\">Jupyter Notebook<\/a><\/div>\n<\/div>\n<\/div><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"490\" height=\"340\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Attribution.png\" alt=\"EconML | attribution | illustration of 3 different colored hands each picking a piece from a pie chart\" class=\"wp-image-656448 size-full\" srcset=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Attribution.png 490w, https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Attribution-300x208.png 300w\" sizes=\"auto, (max-width: 490px) 100vw, 490px\" \/><\/figure><\/div>\n\n\n\n\n\n<h2 class=\"wp-block-heading\" id=\"frequently-asked-questions\">Frequently asked questions<\/h2>\n\n\n\n\n\n<p>\nEconML is designed to answer causal questions: what will happen in response to some change in behavior, prices, or conditions? These questions require different methods than forecasting questions: what will happen next if everything continues as it has been?\n\n<\/p>\n\n\n\n\n\n<p>\nEconML offers the broadest range of cutting-edge AI models designed specifically to answer causal questions. The EconML models also build on familiar Python packages, allowing users to easily select the best model for their question. Finally, EconML includes custom interpreters to create presentation-ready output.\n\n<\/p>\n\n\n\n\n\n<p>\nTry comparing the consistency of your estimates across multiple models, including some that make stronger structural assumptions like linear relationships and some that do not. Pay attention to the standard errors as well as the point estimates\u2014imprecise estimates should be interpreted accordingly. While researchers can introduce bias by narrowly fishing for estimates that match their prior, it is also important to use your expertise to evaluate results. If you estimate that a 5% decrease in price generates an implausible 5000% increase in sales you should carefully review your code!\n\n<\/p>\n\n\n\n\n\n<p>\nFirst carefully check your code for errors and try several causal models. If your estimates are consistent, but implausible, you may have a confounding variable that hasn\u2019t been measured in your data. Think carefully about the source of the data you are using: was there something unusual going on during the period when the data were collected (for example a holiday or an economic downturn)? Is there something unusual about your sample (for example, all men with pre-existing heart conditions)?\n\n<\/p>\n\n\n\n\n\n<p>\nIn this case, no statistical approach can perfectly isolate the causal effect of the treatment on the outcome. DML, OrthoForest, or MetaLearners, all including all the confounders you can observe, will deliver the best approximation of the causal effect that minimizes the bias from confounders. Be aware of some remaining bias when using these estimates.\n\n<\/p>\n\n\n\n\n\n<p>\nYou are identifying a valid causal effect if and only if the underlying assumptions of the causal model assumed by the estimation routine are correct. Those are often hard to test (though the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/Microsoft\/dowhy\" target=\"_blank\" rel=\"noopener noreferrer\">DoWhy package<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> may help). Having made those assumptions, the EconML package allows you to fit the best causal model you can. Many models will store a final stage fit metric that can be used to validate how well the causal model predicts out of sample, which is a good diagnostic as to the quality of your model.\n\n<\/p>\n\n\n\n\n\n<p>\nThis project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/cla.microsoft.com\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/cla.microsoft.com<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>.\n\nWhen you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.\n\nThis project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact <a href=\"mailto:opencode@microsoft.com\">opencode@microsoft.com<\/a> with any additional questions or comments.\n\n<\/p>\n\n\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/github.com\/Microsoft\/EconML\" target=\"_blank\" rel=\"noreferrer noopener\">EconML on GitHub<\/a><\/div>\n\n\n\n<div class=\"wp-block-button is-style-outline is-style-outline--9\"><a data-bi-type=\"button\" class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/econml.azurewebsites.net\/index.html\" target=\"_blank\" rel=\"noreferrer noopener\">Documentation on EconML<\/a><\/div>\n<\/div>\n\n\n","protected":false},"excerpt":{"rendered":"<p>EconML\u00a0is an open source Python package developed by the ALICE team at Microsoft Research that applies the power of machine learning techniques to estimate individualized causal responses from observational or experimental data. The suite of estimation methods provided in EconML represents the latest advances in causal machine learning. By incorporating individual machine learning steps into interpretable causal models, these methods improve the reliability of what-if predictions and make causal analysis quicker and easier for a broad set of users.<\/p>\n","protected":false},"featured_media":810628,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556,13548],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-656325","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-research-area-economics","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[471336,471504,471510,475107,475113,493301,493316,626067,626082,626151,626160,771889,900081],"related-downloads":[],"related-videos":[887568],"related-groups":[330695,470706,656316],"related-events":[836827],"related-opportunities":[],"related-posts":[749191,848023],"related-articles":[],"tab-content":[{"id":0,"name":"Get Started","content":"<div>\r\n<div style=\"background-color: #fff8f2 !important;padding: 25px 25px 10px 25px\">\r\n\r\n<img class=\"alignleft wp-image-664095 size-full\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/06\/imgGear.png\" alt=\"orange gear icon\" width=\"60\" height=\"60\" \/>\r\n<h2>Quick Installation<\/h2>\r\nInstall the latest release for <a href=\"https:\/\/pypi.org\/project\/econml\/\" target=\"_blank\" rel=\"nofollow noopener\">PyPI<\/a>:\u00a0<code style=\"background-color: #ffeada;padding: 4px\">pip install econml<\/code>\r\n\r\n<\/div>\r\n<div style=\"height: 20px\"><\/div>\r\n<h2>For Developers<\/h2>\r\nYou can get started by cloning this repository. We use\u00a0<a href=\"https:\/\/setuptools.readthedocs.io\/en\/latest\/index.html\" target=\"_blank\" rel=\"nofollow noopener\">setuptools<\/a>\u00a0for building and distributing our package. We rely on some recent features of setuptools, so make sure to upgrade to a recent version with\u00a0<code style=\"background-color: #f7f8fa;padding: 4px\">pip install setuptools --upgrade<\/code>. Then from your local copy of the repository you can run\u00a0<code style=\"background-color: #f7f8fa;padding: 4px\">python setup.py develop<\/code>\u00a0to get started.\r\n<h2>Running the Tests<\/h2>\r\nThis project uses\u00a0<a href=\"https:\/\/docs.pytest.org\/\" target=\"_blank\" rel=\"nofollow noopener\">pytest<\/a>\u00a0for testing. To run tests locally after installing the package, you can use\u00a0<code style=\"background-color: #f7f8fa;padding: 4px\">python setup.py pytest<\/code>.\r\n<h2>Generating the documentation<\/h2>\r\nThis project's documentation is generated via\u00a0<a href=\"https:\/\/www.sphinx-doc.org\/en\/master\/index.html\" target=\"_blank\" rel=\"nofollow noopener\">Sphinx<\/a>. Note that we use\u00a0<a href=\"https:\/\/graphviz.org\/\" target=\"_blank\" rel=\"nofollow noopener\">graphviz<\/a>'s\u00a0<code style=\"background-color: #f7f8fa;padding: 4px\">dot<\/code>\u00a0application to produce some of the images in our documentation, so you should make sure that\u00a0<code style=\"background-color: #f7f8fa;padding: 4px\">dot<\/code>\u00a0is installed and in your path.\r\n\r\nTo generate a local copy of the documentation from a clone of this repository, just run\u00a0<code style=\"background-color: #f7f8fa;padding: 4px\">python setup.py build_sphinx -W -E -a<\/code>, which will build the documentation and place it under the\u00a0<code style=\"background-color: #f7f8fa;padding: 4px\">build\/sphinx\/html<\/code>\u00a0path.\r\n<p class=\"\">The reStructuredText files that make up the documentation are stored in the\u00a0<a href=\"https:\/\/github.com\/Microsoft\/EconML\/tree\/master\/doc\" target=\"_blank\" rel=\"nofollow noopener\">docs directory<\/a>; module documentation is automatically generated by the Sphinx build process.<\/p>\r\n\r\n<div style=\"background-color: #f2f9ff !important;padding: 25px 25px 10px 25px\">\r\n<h2>Enviroment<\/h2>\r\nThe econml package works on macOS, Windows, and Linux, and supports Python versions 3.5-3.7. The econml package relies on numpy, scipy, and scikit-learn for most of its underlying numerical computation and machine learning routines, and uses keras for the components built on deep neural networks. If you don\u2019t already have these dependencies installed locally, then installing the econml package from PyPI via pip will also install them.\r\n\r\n<\/div>\r\n<\/div>\r\n<div style=\"height: 20px\"><\/div>\r\n<div>[msr-button text=\"EconML on GitHub\" url=\"https:\/\/github.com\/Microsoft\/EconML\" new-window=\"true\" ][msr-button text=\"Documentation on EconML\" url=\"https:\/\/econml.azurewebsites.net\/\" new-window=\"true\" ]<\/div>"},{"id":1,"name":"How To","content":"<h2>If You Are New to Causal Inference<\/h2>\r\nCausal analysis is used to answer what-if questions. Unlike forecasting, which answers the question of what will happen next if everyone keeps behaving as they have in the past, causal inference answers the question of what would happen next if someone changes behavior, such as pursuing a new pricing strategy for a product or a new treatment for a patient. If you are new to causal inference, our guide will introduce you to the biggest challenges in answering causal questions, how EconML addresses those challenges, and the standard causal terminology we use throughout our SDK and documentation.\r\n\r\n<iframe src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/09\/Causal-Inference-User-Guide-v4-022520.pdf\" width=\"900\" height=\"500\"> <\/iframe>\r\n<h2>Have a Question in Mind?<\/h2>\r\nIf you are ready to start estimating, our <a href=\"https:\/\/econml.azurewebsites.net\/spec\/flowchart.html\" target=\"_blank\" rel=\"noopener\">flow chart<\/a> can guide you to an appropriate estimation model for your question. Also check out our Case Studies for examples that apply these models to real world questions.\r\n<div style=\"height: 20px\"><\/div>\r\n<div>[msr-button text=\"EconML on GitHub\" url=\"https:\/\/github.com\/Microsoft\/EconML\" new-window=\"true\" ][msr-button text=\"Documentation on EconML\" url=\"https:\/\/econml.azurewebsites.net\/index.html\" new-window=\"true\" ]<\/div>"},{"id":2,"name":"Use Cases","content":"<div>\r\n<h2><img class=\"alignright wp-image-656442 \" style=\"padding: 10px\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Recomendation.png\" alt=\"illustration of woman with an A left of her and a B right of her\" width=\"432\" height=\"300\" \/>Recommendation A\/B testing<\/h2>\r\n<p class=\"subtitletxt\"><em>Interpret experiments with imperfect compliance<\/em><\/p>\r\n<b>Question:<\/b>\u00a0A travel website would like to know whether joining a membership program causes users to spend more time engaging with the website. <b>Problem:<\/b>\u00a0They can\u2019t look directly at existing data, comparing members and non-members, because the customers who chose to become members are likely already more engaged than other users. Nor can they run a direct A\/B test because they can\u2019t force users to sign up for membership. <b>Solution:<\/b>\u00a0The company had run an earlier experiment to test the value of a new, faster sign-up process. EconML\u2019s\u00a0<a href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.ortho_iv.html\" target=\"_blank\" rel=\"noopener\">DRIV estimator<\/a>\u00a0uses this experimental nudge towards membership as an instrument that generates random variation in the likelihood of membership. The DRIV model adjusts for the fact that not every customer who was offered the easier sign-up became a member and returns the effect of membership rather than the effect of receiving the quick sign-up.\r\n<div>[msr-button text=\"Trip Advisor Case Study\" url=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/04\/MSR_ALICE_casestudy_2020.pdf\" new-window=\"true\" ][msr-button text=\"Jupyter Notebook\" url=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Recommendation%20AB%20Testing%20at%20An%20Online%20Travel%20Company.ipynb\" new-window=\"true\" ]<\/div>\r\n<div style=\"height: 20px\"><\/div>\r\n<h2><img class=\"alignright wp-image-656445 \" style=\"padding: 10px\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Segmentation.png\" alt=\"illustration of a world map, the bust of a man, and an arrow hitting the bullseye of a target\" width=\"432\" height=\"300\" \/>Customer Segmentation<\/h2>\r\n<p class=\"subtitletxt\"><em>Estimate individualized responses to incentives<\/em><\/p>\r\n<b>Question:<\/b>\u00a0A media subscription service would like to offer targeted discounts through a personalized pricing plan. <b>Problem:<\/b>\u00a0They observe many features of their customers, but are not sure which customers will respond most to a lower price. <b>Solution:<\/b>\u00a0EconML\u2019s\u00a0<a href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.dml.html#module-econml.dml\" target=\"_blank\" rel=\"noopener\">DML estimator<\/a>\u00a0uses price variations in existing data, along with a rich set of user features, to estimate heterogeneous price sensitivities that vary with multiple customer features. The\u00a0<a href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.cate_interpreter.html\" target=\"_blank\" rel=\"noopener\">tree interpreter<\/a>\u00a0provides a presentation-ready summary of the key features that explain the biggest differences in responsiveness to a discount.\r\n<div>[msr-button text=\"Jupyter Notebook\" url=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Customer%20Segmentation%20at%20An%20Online%20Media%20Company.ipynb\" new-window=\"true\" ]<\/div>\r\n<div style=\"height: 20px\"><\/div>\r\n<h2><img class=\"alignright wp-image-656448 \" style=\"padding: 10px;border-radius: 3%\" src=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2020\/05\/Attribution.png\" alt=\"illustration of 3 different colored hands each picking a piece from a pie chart\" width=\"432\" height=\"300\" \/>Multi-investment Attribution<\/h2>\r\n<p class=\"subtitletxt\"><em>Distinguish the effects of multiple outreach efforts<\/em><\/p>\r\n<b>Question:<\/b>\u00a0A startup would like to know the most effective approach for recruiting new customers: price discounts, technical support to ease adoption, or a combination of the two. <b>Problem:<\/b>\u00a0The risk of losing customers makes experiments across outreach efforts too expensive. So far, customers have been offered incentives strategically, for example larger businesses are more likely to get technical support. <b>Solution:<\/b>\u00a0EconML\u2019s\u00a0<a href=\"https:\/\/econml.azurewebsites.net\/_autosummary\/econml.drlearner.html\" target=\"_blank\" rel=\"noopener\">Doubly Robust Learner<\/a> model jointly estimates the effects of multiple discrete treatments. The model uses flexible functions of observed customer features to filter out confounding correlations in existing data and deliver the causal effect of each effort on revenue.\r\n\r\n<\/div>\r\n<div>[msr-button text=\"Technical Paper\" url=\"https:\/\/arxiv.org\/abs\/2103.08390\" new-window=\"true\" ][msr-button text=\"Jupyter Notebook\" url=\"https:\/\/github.com\/microsoft\/EconML\/blob\/master\/notebooks\/CustomerScenarios\/Case%20Study%20-%20Multi-investment%20Attribution%20at%20A%20Software%20Company.ipynb\" new-window=\"true\" ]<\/div>"},{"id":3,"name":"FAQ","content":"<h2>Frequently Asked Questions<\/h2>\r\n[accordion]\r\n\r\n[panel header=\"When should I use EconML?\"]\r\nEconML is designed to answer causal questions: what will happen in response to some change in behavior, prices, or conditions? These questions require different methods than forecasting questions: what will happen next if everything continues as it has been?\r\n\r\n[\/panel]\r\n\r\n[panel header=\"What are the advantages of EconML?\"]\r\nEconML offers the broadest range of cutting-edge AI models designed specifically to answer causal questions. The EconML models also build on familiar Python packages, allowing users to easily select the best model for their question. Finally, EconML includes custom interpreters to create presentation-ready output.\r\n\r\n[\/panel]\r\n\r\n[panel header=\"How do I know if the results make sense?\"]\r\nTry comparing the consistency of your estimates across multiple models, including some that make stronger structural assumptions like linear relationships and some that do not. Pay attention to the standard errors as well as the point estimates\u2014imprecise estimates should be interpreted accordingly. While researchers can introduce bias by narrowly fishing for estimates that match their prior, it is also important to use your expertise to evaluate results. If you estimate that a 5% decrease in price generates an implausible 5000% increase in sales you should carefully review your code!\r\n\r\n[\/panel]\r\n\r\n[panel header=\"I\u2019m getting causal estimates that make no sense. What next?\"]\r\nFirst carefully check your code for errors and try several causal models. If your estimates are consistent, but implausible, you may have a confounding variable that hasn\u2019t been measured in your data. Think carefully about the source of the data you are using: was there something unusual going on during the period when the data were collected (for example a holiday or an economic downturn)? Is there something unusual about your sample (for example, all men with pre-existing heart conditions)?\r\n\r\n[\/panel]\r\n\r\n[panel header=\"What if I don\u2019t have a good instrument, can\u2019t run an experiment, and don\u2019t observe all confounders?\"]\r\nIn this case, no statistical approach can perfectly isolate the causal effect of the treatment on the outcome. DML, OrthoForest, or MetaLearners, all including all the confounders you can observe, will deliver the best approximation of the causal effect that minimizes the bias from confounders. Be aware of some remaining bias when using these estimates.\r\n\r\n[\/panel]\r\n\r\n[panel header=\"How can I test whether I'm identifying the causal effect?\"]\r\nYou are identifying a valid causal effect if and only if the underlying assumptions of the causal model assumed by the estimation routine are correct. Those are often hard to test (though the <a href=\"https:\/\/github.com\/Microsoft\/dowhy\" target=\"_blank\" rel=\"noopener\">DoWhy package<\/a> may help). Having made those assumptions, the EconML package allows you to fit the best causal model you can. Many models will store a final stage fit metric that can be used to validate how well the causal model predicts out of sample, which is a good diagnostic as to the quality of your model.\r\n\r\n[\/panel]\r\n\r\n[panel header=\"How do I give feedback?\"]\r\nThis project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit <a href=\"https:\/\/cla.microsoft.com\" target=\"_blank\" rel=\"noopener\">https:\/\/cla.microsoft.com<\/a>.\r\n\r\nWhen you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.\r\n\r\nThis project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact <a href=\"mailto:opencode@microsoft.com\">opencode@microsoft.com<\/a> with any additional questions or comments.\r\n\r\n[\/panel]\r\n\r\n[\/accordion]\r\n<div style=\"height: 20px\"><\/div>\r\n<div>[msr-button text=\"EconML on GitHub\" url=\"https:\/\/github.com\/Microsoft\/EconML\" new-window=\"true\" ][msr-button text=\"Documentation on EconML\" url=\"https:\/\/econml.azurewebsites.net\/index.html\" new-window=\"true\" ]<\/div>"}],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Keith Battocchi","user_id":36548,"people_section":"Core Team","alias":"kebatt"},{"type":"user_nicename","display_name":"Eleanor Dillon","user_id":38467,"people_section":"Core Team","alias":"eldillon"},{"type":"user_nicename","display_name":"Darya Moldavskaya","user_id":43569,"people_section":"Core Team","alias":"dmoldavskaya"},{"type":"user_nicename","display_name":"Friederike Niedtner","user_id":39919,"people_section":"Core Team","alias":"fniedtner"},{"type":"user_nicename","display_name":"Fabio Vera","user_id":41398,"people_section":"Core Team","alias":"fabiovera"}],"msr_research_lab":[199563],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/656325","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":38,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/656325\/revisions"}],"predecessor-version":[{"id":1135624,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/656325\/revisions\/1135624"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/810628"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=656325"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=656325"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=656325"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=656325"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=656325"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}