{"id":642972,"date":"2020-03-13T10:22:06","date_gmt":"2020-03-13T17:22:06","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-blog-post&p=642972"},"modified":"2020-03-13T16:59:33","modified_gmt":"2020-03-13T23:59:33","slug":"covid-19-highlights-the-wisdom-of-the-academic-crowd","status":"publish","type":"msr-blog-post","link":"https:\/\/www.microsoft.com\/en-us\/research\/articles\/covid-19-highlights-the-wisdom-of-the-academic-crowd\/","title":{"rendered":"COVID-19 Highlights the Wisdom of the Academic Crowd"},"content":{"rendered":"

For more than a quarter century, we have lived a life where our online activities are closely monitored, collected, and exploited by internet giants to offer useful services without charging fees. Today, it is unfathomable to have to pay for Apple\u2019s Facetime, Google\u2019s Hangout, Facebook\u2019s Messenger or What\u2019s App, etc., the same way that we have to pay heftily to our wireless providers for the similar services in phone calls and text messages. These internet businesses prosper because the user data are worth a lot in our connected lives. Aside from creating detailed user profiles to place more targeted advertisements, the data holds the keys to identifying trends and project needs for new products and services.<\/p>\n

One example is Google Flu Trends (GFT) which made headlines around the world in February 2013. It was reported that web search queries, with their timestamps and locations, could serve as a good indicator of the flu epidemics. Unfortunately, it was later found (opens in new tab)<\/span><\/a> that GFT\u2019s predictions often have a wide gap from the Center of Disease Control (CDC) data, even though GFT was specifically trained on CDC reports. Nevertheless, the \u201cwisdom of the crowd\u201d hidden in search queries is certainly valuable and a recent approach (opens in new tab)<\/span><\/a> finds it beneficial to combine search data with electronic health records to improve the prediction accuracy.<\/p>\n

To be sure, web search data are tricky to use correctly, especially in reliably tracking epidemics. At the time of writing, COVID-19 is rampaging around the world; with Italy and China both having to take drastic measures, such as locking down cities and cancelling schools to slow down the infections. In Microsoft\u2019s backyard the city of Kirkland, made famous by Costco as its store brand, has seen deaths exceeding 25 with worrisome evidence of community spread since the first confirmed case was reported on January 21st<\/sup>, 3 weeks after the outbreak was noticed in Wuhan China and one day after China\u2019s CDC declared an emergency.<\/p>\n

According to Google Trends, however, search volume about coronavirus was insignificant until the day China declared the emergency, but the interests subsided in 10 days. For the first three weeks of February, the query volume continued to drop. It was not until February 21st<\/sup>, 10 days after the official name COVID-19 was announced by the World Health Organization (WHO), did the search query start to increase again, as can be seen in the Figure below. To ensure the trend plot is interpreted correctly, we contrast the search for \u201ccoronavirus\u201d with \u201cgoogle.\u201d The latter query is likely a result of a portion of internet users typing \u201cgoogle\u201d into their web browser address bar, indicating a search intent. Nevertheless, it tracks the daily search activities and shows the cyclic nature of the search queries. The activity-normalized curve for \u201ccoronavirus\u201d is shown as the dotted line against the secondary axis in the Figure.<\/p>\n

\"Google<\/p>\n

 <\/p>\n

In the meantime, the research community has no illusions of the danger this novel coronavirus can pose to the world. Articles sounding the alarm began to be published in the journals the second week of January, one full week before China\u2019s emergency declaration. Scientific activities prior to January 20th<\/sup> include events highlighted below:<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\u00a0Date<\/td>\n\u00a0Events<\/td>\n<\/tr>\n
\u00a012\/27\/2019<\/td>\nAs recounted in this JAMA paper (opens in new tab)<\/span><\/a>, China CDC was first alerted 4 cases of unusual pneumonia in Wuhan, China<\/td>\n<\/tr>\n
\u00a012\/31\/2019<\/td>\nWuhan\u2019s health commission disclosed (opens in new tab)<\/span><\/a> that 27 people, all having visited a local seafood market, had developed symptoms of a viral pneumonia. The public was advised to avoid crowded areas with poor air circulation, wear masks when going out, but there were no reasons to be alarmed. WHO was notified (opens in new tab)<\/span><\/a> of the outbreak on the same day and the seafood market was closed the next day.<\/td>\n<\/tr>\n
\u00a01\/7\/2020<\/td>\nChinese authorities conducted genome sequencing and identified the disease was caused by a novel coronavirus after ruling out previously known coronaviruses that caused SARS (opens in new tab)<\/span><\/a>, MERS (opens in new tab)<\/span><\/a>, and many others (opens in new tab)<\/span><\/a>.<\/td>\n<\/tr>\n
\u00a01\/9\/2020<\/td>\nWHO announced (opens in new tab)<\/span><\/a> the discovery of a novel coronavirus by Chinese authorities, but stated the virus \u201cdoes not transmit readily between people\u201d.<\/td>\n<\/tr>\n
\u00a01\/12\/2020<\/td>\nChina reported (opens in new tab)<\/span><\/a> 1st<\/sup> death out of 41 confirmed cases and shared the genetic sequences with WHO for other countries to develop diagnostic kits.<\/td>\n<\/tr>\n
\u00a01\/14\/2020<\/td>\nA paper (opens in new tab)<\/span><\/a> published in the International Journal of Infectious Diseases (opens in new tab)<\/span><\/a> uses the term 2019-nCov to refer to the novel coronavirus. 7 of the 12 authors are members of the Pan-African Network on Emerging and Re-emerging Infections funded by European Horizon 2020.<\/td>\n<\/tr>\n
\u00a01\/15\/2020<\/td>\nWHO reported (opens in new tab)<\/span><\/a> the first confirmed case in Japan and suggested the possibility of human-to-human transmissions after a few cases with no link to the seafood market were found in China.<\/td>\n<\/tr>\n
\u00a01\/17\/2020<\/td>\nAn article (opens in new tab)<\/span><\/a> in Science (opens in new tab)<\/span><\/a> cautioned the spread of the virus after a tourist in Thailand was confirmed to be infected. Family Practice News reported (opens in new tab)<\/span><\/a> 3 US airports started screening travelers from Wuhan China as soon as US CDC introduced the measure (opens in new tab)<\/span><\/a>.<\/td>\n<\/tr>\n
\u00a01\/18\/2020<\/td>\nJournal of Hospital Infection (opens in new tab)<\/span><\/a> made available online this paper (opens in new tab)<\/span><\/a> outlining measures to prevent hospital outbreak for the novel coronavirus. The article is officially scheduled to appear in the journal\u2019s March 2020 issue.<\/td>\n<\/tr>\n
\u00a01\/19\/2020<\/td>\nA paper (opens in new tab)<\/span><\/a> proposing a mathematical model of the novel coronavirus is published on bioRxiv. The model assumes animal-to-human transmission.<\/td>\n<\/tr>\n
\u00a01\/20\/2020<\/td>\nBoth BMJ (opens in new tab)<\/span><\/a> and Science (opens in new tab)<\/span><\/a> reported surging cases of the virus infection in this (opens in new tab)<\/span><\/a> and this (opens in new tab)<\/span><\/a> article. Web search on this topic started to appear according to Google Trends.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

With editors prioritizing the reviews and publications of COVID-19 papers, the volume of publications on this topic quickly rises. Furthermore, almost all academic publishers have dropped the paywalls on COVID-19 papers, accelerating the dissemination of the scientific discoveries and the citations to these articles. In total, more than 920 articles have been published in the crucial month between January 14 and February 14, one week ahead of the turning point in Google Trends. The collective responses of the scholarly community to COVID-19 are captured in the remarkable publication node growth and the citation edges in Microsoft Academic Graph (MAG), as can be seen below:<\/p>\n

 <\/p>\n

\"COVID-19<\/p>\n

These behavioral changes have significant impacts on MAG. First, the fast appearing publications have led our machine readers to conclude not one, but two new concepts have emerged under the existing concept \u201ccoronavirus (opens in new tab)<\/span><\/a>\u201d (MAG Id=2777648638) and \u201cinfectious diseases (opens in new tab)<\/span><\/a>\u201d (MAG Id=524204448). Based on the operations described in our recent paper (opens in new tab)<\/span><\/a>, the new concept under the coronavirus is called \u201cSevere acute respiratory syndrome coronavirus 2 (SARS-CoV-2)\u201d, or simply \u201csars-cov-2 (opens in new tab)<\/span><\/a>\u201d (MAG Id=3007834351), and is mapped to this Wikipedia page (opens in new tab)<\/span><\/a>. The other new infectious disease is unsurprisingly called \u201cCoronavirus disease 2019 (COVID-19)\u201d, or simply \u201ccovid-19 (opens in new tab)<\/span><\/a>\u201d (MAG Id=3008058167), and is mapped to the corresponding Wikipedia page (opens in new tab)<\/span><\/a>. Note that the subtle distinction between a disease and its cause is an ongoing issue for our machine learning algorithm. As described in the paper, the natural outcome based on the language and the network similarities<\/a>, is to follow the classifications from the authoritative webpages which, in this case, are the two Wikipedia articles that make the distinction. However, as described in our recent post<\/a>, we also program our system to be as consistent as the Unified Medical Language System (UMLS) (opens in new tab)<\/span><\/a>, where such distinctions are sometimes not made (e.g., the \u201cHIV\/AIDS (opens in new tab)<\/span><\/a>\u201d concept originated from UMLS (opens in new tab)<\/span><\/a> lumps the disease AIDS and its cause HIV together). The differences among the authoritative sources we use lead to inconsistencies in our taxonomy. We are anxiously waiting to see how our system will resolve this issue by itself after more publications and citations are observed in the future.<\/p>\n

Secondly, the rapid rise of citation activities among these papers has also boosted their saliency scores, which are an indicator that predict the likelihoods of papers to receive citations in the next 5 years. As explained in this (opens in new tab)<\/span><\/a> and this paper (opens in new tab)<\/span><\/a>, saliency uses the same reinforcement learning (RF) algorithm as in Alpha Go and other video game playing systems to anticipate the next moves from humans. Instead of sending the RF learning agents into parallel universes to play thousands of games simultaneously, here we send the RF agents to travel back and forth in time to acquire the best strategy in predicting future citations. In this sense saliency is a leading rather than a lagging indicator, e.g. the citation count, of the research impact of any given paper. Due to the flurry of citations among them a mere month is all it takes for the COVID-19 papers to dominate the search results for the query \u201ccoronavirus china (opens in new tab)<\/span><\/a>\u201d, even though they all have yet to receive their full recognitions and their citation counts are much lower than other coronavirus papers published years ago. To see the differences in rankings by saliency and by citation counts, snapshots of the search results ranked by the saliency (relevance) and the citation counts based on data up until February 14, 2020 are shown below. Note that the term \u201cCOVID-19\u201d was adopted by WHO (opens in new tab)<\/span><\/a> only on February 11, the same day the Coronavirus Study Group published this naming paper on bioRxiv (opens in new tab)<\/span><\/a> proposing to use SARS-Cov-2 instead of 2019-nCov. Papers accepted for publications prior to this date cannot be retrieved with these keywords. However, these early publications are detected as discussing topics about \u201ccoronavirus\u201d and \u201cchina (opens in new tab)<\/span><\/a>\u201d (MAG Id = 191935318). The query is thus useful in finding these early publications.<\/p>\n

\"Saliency<\/p>\n

\"Citation<\/p>\n

So how effective are search and click behaviors underlying Google\u2019s and Bing\u2019s rankings? Our experience shows that searching behaviors are useful in capturing general consumer interests. However, for cases such as pandemics that require expert knowledge, the consumer behaviors are in fact a lagging indicator. For this reason, and to respect the privacy of our users, Microsoft Academic is a search engine that does not include user private data such as browsing and clicking activities in the search ranking. We can, however, compare the rankings in the equivalent Google Scholar to understand the effectiveness of using search behaviors. Indeed, when the search volume on the topic of \u201ccoronavirus\u201d was not enough, older papers that have accumulated larger citation counts were ranked higher. By the time of February 18, 2020, however, some papers about COVID-19 started to make it to the top 10 search results. To be fair, by this time most papers have included the terminology \u201c2019-nCov\u201d to clarify which novel coronavirus the contents are about. As the screenshots below show, Google Scholar did an excellent job for this query.<\/p>\n

The field of study known as infodemiology (opens in new tab)<\/span><\/a>, established in 2002 based on the insight of Gunther Eysenbach (opens in new tab)<\/span><\/a>\u2019s ground breaking paper (opens in new tab)<\/span><\/a>, has been studying the utilities of search queries and social media activities in capturing public health (opens in new tab)<\/span><\/a> issues in a more accurate and timely manner. To the best of our knowledge (based on this query (opens in new tab)<\/span><\/a>), we believe we are the first to observe scholarly communication activities as a potential resource in tracking epidemics. More research is needed, but one thing is for sure: without the scientists actively submitting new findings and editors prioritizing their publications, we wouldn\u2019t have been able to report this useful dataset. The credit should really go to the collaborative instincts and the scholarships of the research community that we are very proud to be a member of.<\/p>\n

Happy researching!<\/p>\n

\"Google<\/p>\n

\"Google<\/p>\n","protected":false},"excerpt":{"rendered":"

For more than a quarter century, we have lived a life where our online activities are closely monitored, collected, and exploited by internet giants to offer useful services without charging fees. Today, it is unfathomable to have to pay for Apple\u2019s Facetime, Google\u2019s Hangout, Facebook\u2019s Messenger or What\u2019s App, etc., the same way that we have to pay heftily to our wireless providers for the similar services in phone calls and text messages. These internet businesses prosper because the user data are worth a lot in our connected lives. Aside from creating detailed user profiles to place more targeted advertisements, the data holds the keys to identifying trends and project needs for new products and services.<\/p>\n","protected":false},"author":33458,"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-content-parent":170262,"msr_hide_image_in_river":0,"footnotes":""},"research-area":[],"msr-locale":[268875],"msr-post-option":[],"class_list":["post-642972","msr-blog-post","type-msr-blog-post","status-publish","hentry","msr-locale-en_us"],"msr_assoc_parent":{"id":170262,"type":"project"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/642972","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-blog-post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/33458"}],"version-history":[{"count":16,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/642972\/revisions"}],"predecessor-version":[{"id":643371,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/642972\/revisions\/643371"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=642972"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=642972"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=642972"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=642972"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}