{"id":308282,"date":"2006-07-05T05:00:39","date_gmt":"2006-07-05T12:00:39","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?p=308282"},"modified":"2016-10-19T08:25:38","modified_gmt":"2016-10-19T15:25:38","slug":"sensormap-delivers-real-time-data-go","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/sensormap-delivers-real-time-data-go\/","title":{"rendered":"SensorMap Delivers Real-Time Data on the Go"},"content":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research<\/em><\/p>\n

You\u2019re in pre-movie limbo. It\u2019s 6 p.m., and the film you want to see starts at 7. You\u2019ve got enough time to grab a quick dinner\u2014if you don\u2019t have to wait for a table. You know the neighborhood around the theater has some good restaurants, but there\u2019s no time to waste. Which ones could take your order right away?<\/p>\n

Someday soon, perhaps, SensorMap will get you to your movie, relaxed and well-fed.<\/p>\n

SensorMap is a platform for publishing and searching for real-time data, emerging from Microsoft Research\u2019s Networked Embedded Computing group, that shows promise in being able to combine static and real-time data to help users receive search results informed and enriched by the users\u2019 particular information needs.<\/p>\n

\u201cYou can go to a search engine and search for all the restaurants in Seattle,\u201d says Suman Nath<\/a>, a researcher based in Microsoft Research\u2019s Redmond lab<\/a>. \u201cBut what you can\u2019t do today is to find me all the restaurants in Seattle that have a waiting time of less than 30 minutes.<\/p>\n

\u201cThat part is a real-time component, and we want to enable this type of scenario, so that you can search based not just on static data, but also on real-time data.\u201d<\/p>\n

SensorMap will be released to the public during the Faculty Summit, an annual event sponsored by Microsoft Research\u2019s External Research & Programs group that brings together academic researchers, faculty, and Microsoft researchers and product-group personnel to discuss areas of mutual concern. This year\u2019s event will be held July 17-18 in Redmond.<\/p>\n

The SensorMap platform, part of the SenseWeb<\/a> project, will serve as a portal for those who want to publish real-time data to make it searchable. And the gathering of academics for the Faculty Summit provides a perfect setting for a coming-out party.<\/p>\n

\u201cMany researchers in different universities do research on sensor networks and have deployed sensors for their experiments,\u201d Nath explains. \u201cWhile doing experiments, they are collecting information about different things, and we think SensorMap can be a good portal for them to publish their data.\u201d<\/p>\n

It\u2019s all about collecting the data and sharing the collection.<\/p>\n

\u201cWhy isn\u2019t there this type of application now?\u201d Nath asks. \u201cThe main problem is there is not enough data. Basically, there is this chicken-and-egg problem. There is no application because there is no data, and there is no data because people don\u2019t know how useful that data is. We are trying to show people that if you give us live data, then we could do interesting stuff. Once the data is there, other people can write applications on top of it.\u201d<\/p>\n

Nath and his SensorMap colleagues\u2014Jie Liu<\/a>, a researcher, and their manager, Feng Zhao, a principal researcher\u2014are eager to encourage that awareness.<\/p>\n

\u201cSensorMap allows users to easily publish their live data from their sensors,\u201d Nath explains. \u201cWe actually don\u2019t care about what that particular data source is. It can be sensors, it can be someone typing something, it can be some other thing that provides data that change over time. We make that part simple so we can encourage people to put more and more data on the Web.\u201d<\/p>\n

Once the data is collected, SensorMap indexes that data\u2014\u201cthe most difficult part,\u201d Nath says\u2014to make it searchable. A database holds geographically indexed sensor descriptions and works in tandem with a sensor-data publishing Web service and a server-side query processor.<\/p>\n

\u201cWe are trying to make all these things easier,\u201d Nath says. \u201cWe are providing some tools for the publisher to publish data, we are providing this portal, where you can go and easily find that information, and, in the middle, the indexing and processing, this is all done by us.\u201d<\/p>\n

A few university projects elsewhere recently have used geographic Web interfaces to annotate sensors, as, for example, in a UCLA project. And there are Web-based maps that offer some sensor information, such as weather data or traffic information. But those maps are not integrated. SenseWeb is organizing the sensor data so users quickly can locate relevant information, just as a current search engine would do for text documents. The vision of searching the physical world with live sensor data was outlined in 2004 in a book by Zhao and Leonidas Guibas entitled Wireless Sensor Networks: An Information Processing Approach<\/em>.<\/p>\n

\u201cWhat we are trying to do here,\u201d Nath says, \u201cis to bring all different types of sensors together. And we\u2019re actually trying to provide more useful things\u2014search based on geographic region or based on keywords. Also, depending on the zoom level of a map, we are trying to aggregate data. If you\u2019re querying temperature sensors statewide, we can provide individual sensor information, but we also can show you the average temperature. That sort of aggregation is absent in other industry applications.\u201d<\/p>\n

The expectation is that the SensorMap platform will enable data publishers to define the uses for their contributions.<\/p>\n

\u201cIdeally,\u201d Nath says, \u201cwhat we want to do is to make this platform open in the sense that if you want to add new types of sensors, you can define your own icon and even define what types of things can be done with that sensor. For example, is it OK to compute an average with the data being shared? In the future, we\u2019ll add a control so you can specify who would be able to see your data\u2014just you, your friends, or the world.\u201d<\/p>\n

The technology behind SensorMap has come together rather quickly; work didn\u2019t really coalesce until January. The Sense Web team built a small prototype to provide a proof of concept and displayed it in early March during TechFest, Microsoft Research\u2019s annual technology fair for Microsoft employees. SensorMap received good feedback, and after some enhancements, was showcased during the International Conference on Information Processing in Sensor Networks (opens in new tab)<\/span><\/a>, sponsored by the Association for Computing Machinery and the Institute of Electrical and Electronics Engineers and held April 19-21 in Nashville, Tenn. Again, the reception was positive, as it was in early May during Microsoft Research\u2019s Silicon Valley Road Show.<\/p>\n

\u201cSo far, we have got only good responses from people,\u201d Nath smiles, \u201cand that motivated us to make it public.\u201d<\/p>\n

Not only was SensorMap received well, but it also has elicited interest from Microsoft product groups.<\/p>\n

\u201cWindows Live Local (opens in new tab)<\/span><\/a>\u2122 was the default one,\u201d Nath recalls. \u201cWe thought they might be interested in that. They are interested in the technology and the data. They said that if the technology matures, they could put it into their system.<\/p>\n

\u201cThey also were interested in the data. They thought that if we have this type of platform, then more people will be interested in putting data online. And they can probably use that data to build another application on top of it.\u201d<\/p>\n

That group also was intrigued by some of the techniques employed by the SensorMap team. The existing system does not yet handle caching for live data streams and other issues. But SensorMap can enhance that.<\/p>\n

\u201cEvery time we make a query,\u201d Nath explains, \u201cwe validate it.\u201d<\/p>\n

Thus, Sensor Map can help answer a question such as, \u201cHow many restaurants are in this region of a city?\u201d You simply click a few times on a map to create a polygon encompassing a region, then search within the prescribed area. The technology also enables searching using free text and searching by sensor type. Again, the more data available, the more useful the results.<\/p>\n

\u201cA sensor has two types of data,\u201d Nath says. \u201cOne is the live data, which changes over time. The other is metadata that describes the sensor itself. Over time, that metadata doesn\u2019t change that much.<\/p>\n

\u201cIn our system, we have one central database where we store all the metadata. When you make a query, the query first goes to that central database, and it returns a list of sensors relevant to the query. Another module talks to those sensors directly, gets the live data, processes that data, and sends it back to the user.\u201d<\/p>\n

There are challenges, of course, in coping with such a dynamic set of information.<\/p>\n

\u201cThe biggest problem,\u201d Nath says, \u201cis the query-processing part. And the biggest challenge in this whole space is scalability. If you have thousands of sensors and if you have many, many users, how can we scale efficiently to that? How can we cache data on the back end so we can reuse that data for scaling? There will be lots of issues that we need to solve.\u201d<\/p>\n

Things are progressing nicely, though. The team released the latest version of MSR Sense, the Microsoft Research Network Embedded Sensing Toolkit, in January. MSR Sense is a collection of software tools that enable users to collect, process, archive, and visualize data from a sensor network. A second toolkit is under construction that will enable users to build their own SensorMaps. And other refinements are in the works.<\/p>\n

\u201cThere are many interesting things to be done,\u201d Nath says. \u201cRight now, we search only on the metadata. We have to explore how we can search more effectively on the live data.<\/p>\n

\u201cAlso, we show all the data as points, but you can imagine a service where we generate some sort of map like a temperature-contour map or a gradient map and overlay those on top of the SensorMap.\u201d<\/p>\n

All in good time. For now, Nath and colleagues are pretty happy with what they\u2019ve achieved thus far.<\/p>\n

\u201cRight now, you search only for static data,\u201d he says, \u201cbut I hope there will someday be another box on a search engine where you can put in some real-time information. You could ask some complicated queries.<\/p>\n

\u201cThat\u2019s the most exciting part of this: You\u2019ll have lots of data. What can you do with that?\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"

By Rob Knies, Managing Editor, Microsoft Research You\u2019re in pre-movie limbo. It\u2019s 6 p.m., and the film you want to see starts at 7. You\u2019ve got enough time to grab a quick dinner\u2014if you don\u2019t have to wait for a table. You know the neighborhood around the theater has some good restaurants, but there\u2019s no […]<\/p>\n","protected":false},"author":39507,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[194474,194485],"tags":[196357,215756,186855,215753,215762,215759],"research-area":[13563,13547],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-308282","post","type-post","status-publish","format-standard","hentry","category-data-visulalization","category-networking","tag-metadata","tag-msr-sense","tag-query-processing","tag-senseweb","tag-sensormap","tag-windows-live-local","msr-research-area-data-platform-analytics","msr-research-area-systems-and-networking","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"July 5, 2006","formattedExcerpt":"By Rob Knies, Managing Editor, Microsoft Research You\u2019re in pre-movie limbo. It\u2019s 6 p.m., and the film you want to see starts at 7. You\u2019ve got enough time to grab a quick dinner\u2014if you don\u2019t have to wait for a table. You know the neighborhood…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/308282","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=308282"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/308282\/revisions"}],"predecessor-version":[{"id":308840,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/308282\/revisions\/308840"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=308282"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=308282"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=308282"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=308282"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=308282"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=308282"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=308282"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=308282"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=308282"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=308282"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=308282"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}