{"id":315,"date":"2013-06-12T11:45:00","date_gmt":"2013-06-12T11:45:00","guid":{"rendered":"https:\/\/blogs.technet.microsoft.com\/inside_microsoft_research\/2013\/06\/12\/continuous-mobile-vision-takes-a-step-forward\/"},"modified":"2016-07-20T07:31:35","modified_gmt":"2016-07-20T14:31:35","slug":"continuous-mobile-vision-takes-a-step-forward","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/research\/blog\/continuous-mobile-vision-takes-a-step-forward\/","title":{"rendered":"Continuous Mobile Vision Takes a Step Forward"},"content":{"rendered":"<div id=\"share-icons-top-of-post\">\n<ul class=\"post-social\"><!--          FACEBOOK LIKE CODE             --><\/p>\n<li class=\"post-facebook\"><iframe style=\"border: none; overflow: hidden; width: 100px; height: 21px;\" src=\"http:\/\/www.facebook.com\/plugins\/like.php?href=http%3A%2F%2Fblogs.technet.com%2Fb%2Finside_microsoft_research%2Farchive%2F2013%2F06%2F12%2Fcontinuous-mobile-vision-takes-a-step-forward.aspx&send=false&layout=button_count&width=100&show_faces=true&font=segoe+ui&colorscheme=light&action=like&height=21\" frameborder=\"0\" scrolling=\"no\"><\/iframe><\/li>\n<p><!--          TWITTER SHARING CODE           --><\/p>\n<li class=\"post-twitter\"><a class=\"twitter-share-button\" href=\"https:\/\/x.com\/share\" data-via=\"msftresearch\" data-count=\"horizontal\">Tweet<\/a><br \/>\n<script type=\"text\/javascript\" language=\"JavaScript\" src=\"https:\/\/msdnshared.blob.core.windows.net\/media\/TNBlogsFS\/prod.evol.blogs.technet.com\/CommunityServer.Blogs.Components.WeblogFiles\/00\/00\/00\/90\/35\/js\/6811.tweet-this.js\" original-url=\"http:\/\/blogs.technet.com\/cfs-file.ashx\/__key\/communityserver-blogs-components-weblogfiles\/00-00-00-90-35-js\/6811.tweet_2D00_this.js\"><\/script>\n<\/li>\n<\/ul>\n<\/div>\n<p><!--    END SOCIAL SHARING CONTENT    --> <!-- ENTER AUTHOR AND BLOG CONTENT BELOW --><\/p>\n<p class=\"posted-by\">Posted by <span class=\"author\">Rob Knies<\/span><\/p>\n<p><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/msdnshared.blob.core.windows.net\/media\/TNBlogsFS\/prod.evol.blogs.technet.com\/CommunityServer.Blogs.Components.WeblogFiles\/00\/00\/00\/90\/35\/2526.mit-mobile-summit.PNG\" original-url=\"http:\/\/blogs.technet.com\/cfs-file.ashx\/__key\/communityserver-blogs-components-weblogfiles\/00-00-00-90-35\/2526.mit_2D00_mobile_2D00_summit.PNG\"><img decoding=\"async\" style=\"margin: 10px; border: 0px currentColor; float: left;\" title=\"Mobile Summit logo\" src=\"https:\/\/msdnshared.blob.core.windows.net\/media\/TNBlogsFS\/prod.evol.blogs.technet.com\/CommunityServer.Blogs.Components.WeblogFiles\/00\/00\/00\/90\/35\/2526.mit-mobile-summit.PNG\" original-url=\"http:\/\/blogs.technet.com\/resized-image.ashx\/__size\/550x0\/__key\/communityserver-blogs-components-weblogfiles\/00-00-00-90-35\/2526.mit_2D00_mobile_2D00_summit.PNG\" alt=\"Mobile Summit logo\" width=\"300\" \/><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n<p>You might not be aware of the term &ldquo;continuous mobile vision,&rdquo; but I&rsquo;ll bet there&rsquo;s a good chance you are aware of one of the scenarios it could enable.<\/p>\n<p>Remember the concept, bandied about in recent years, of technology that can remind you of a person&rsquo;s name once her or his face has been detected? Yeah, that one. I&rsquo;m sure that most of us could make use of it once in a while.<\/p>\n<p>The problem, though, is that image sensing takes lots of energy. That&rsquo;s because modern image sensors lack energy proportionality. They&rsquo;re power-hungry. That&rsquo;s fine when a high-resolution, high-frame-rate image is desired. But even when you&rsquo;re not seeking images of that quality, today&rsquo;s image sensors still consume a lot of power.<\/p>\n<p>Someday soon, though, things could change for the better. That&rsquo;s the premise behind the paper <em><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"<em><a title=\"<em>Energy Characterization and Optimization of Image Sensing Toward Continuous Mobile Vision<\/em>\" href=\"http:\/\/www.ruf.rice.edu\/~mobile\/publications\/likamwa2013mobisys1.pdf\" target=\"_blank\">Energy Characterization and Optimization of Image Sensing Toward Continuous Mobile Vision<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/em>&#8221; href=&#8221;http:\/\/research.microsoft.com\/apps\/pubs\/default.aspx?id=194140&#8243; target=&#8221;_blank&#8221;>Energy Characterization and Optimization of Image Sensing Toward Continuous Mobile Vision<\/a><\/em>, which has been accepted for presentation during <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"MobiSys 2013\" href=\"http:\/\/www.sigmobile.org\/mobisys\/2013\/index.php\" target=\"_blank\">MobiSys 2013<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, the 11th International Conference on Mobile Systems, Applications and Services, being held June 25-28 in Taipei, Taiwan.<\/p>\n<p>The paper was written by Robert LiKamWa and Lin Zhong of Rice University and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"Microsoft Research Redmond\" href=\"http:\/\/research.microsoft.com\/en-us\/labs\/redmond\/default.aspx\" target=\"_blank\">Microsoft Research Redmond<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, along with <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"Bodhi Priyantha\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/bodhip\/\" target=\"_blank\">Bodhi Priyantha<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"Matthai Philipose\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/matthaip\/default.aspx\" target=\"_blank\">Matthai Philipose<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"Paramvir (Victor) Bahl\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/bahl\/\" target=\"_blank\">Paramvir (Victor) Bahl<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> of Microsoft Research Redmond.<\/p>\n<p>Bahl, a principal researcher and manager of the <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"Mobility and Networking Research\" href=\"http:\/\/research.microsoft.com\/en-us\/groups\/mnr\/\" target=\"_blank\">Mobility and Networking Research<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> group, spoke June 11 during the MIT Technology Review <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"Mobile Summit 2013\" href=\"http:\/\/www.technologyreview.com\/summit\/mobile\/\" target=\"_blank\">Mobile Summit 2013<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, held June 10-11 in San Francisco, discussing the technology behind the MobiSys paper, and he underscored the new area being explored by him and his colleagues.<\/p>\n<p>&ldquo;Our industry has been focused on building small, higher-resolution image sensors for decades,&rdquo; he said. &ldquo;We believe that we are the first to look at systems-level optimizations for improving their energy consumption. We believe the techniques we have designed are fundamental, in that they will stand the test of time.&rdquo;<\/p>\n<p>The vision behind this work was delivered a year ago, during the Mobile Cloud Computing and Services workshop, in a paper titled <em><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"<em>Cloud-Powered Sight for All: Showing the Cloud What You See<\/em>&#8221; href=&#8221;http:\/\/research.microsoft.com\/en-us\/um\/people\/bahl\/Papers\/Pdf\/mcs12_vision.pdf&#8221; target=&#8221;_blank&#8221;>Cloud-Powered Sight for All: Showing the Cloud What You See<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/em>, written by Bahl, Philipose, and Zhong. Now, the researchers are taking the next step.<\/p>\n<p>Current image sensors are optimized for taking photographs and videos, but if paired with a processor, the sensors could do much more, such as reading barcodes or translating text.<\/p>\n<p>The next step would be to enable continuous mobile vision, in which the device in which the sensor is located always seeks ways to serve its user. That could involve remembering faces at a party, but it could also recall food and drink consumption or help the user navigate through a confusing environment&mdash;scenarios in which high resolution and high frame rates are not paramount. If, by doing so, such sensors drain less battery power, then that should increase their usefulness over a longer period of time, which is a required part of the continuous mobile vision.<\/p>\n<p>The researchers have invoked a couple of mechanisms for improving the energy efficiency in image sensors. The first optimizes clock scaling, inspired by clock-scaling techniques for processors. In the case of image sensors, reduced resolution and frame rates are analytically determined.<\/p>\n<p>The second technique toward energy proportionality is to use an aggressive standby mode between frames, when the frame rate and resolution are sufficiently low. The energy wasted while the device is in idle mode can be reduced significantly.<\/p>\n<p>In combination, for continuous image registration on video, the paper reports, these mechanisms can achieve a 36-percent power reduction by choosing an optimal clock frequency and a 95-percent power reduction by using aggressive standby. Further power reductions are available via the use of architectural modifications of image sensors, the researchers suggest, and computer-vision benchmarks demonstrate application-quality and energy-efficiency tradeoffs that can be attained.<\/p>\n<p>It all amounts to a novel, promising step toward the dream of continuous mobile vision.<\/p>\n<p>&ldquo;Researchers and engineers have been working on a similar vision for a long time,&rdquo; Bahl said. &ldquo;A case in point is the ongoing work in robotics and the early research pursued by Dan Siewiorek at Carnegie Mellon University and Steve Mann at MIT. Still, we haven&rsquo;t yet succeeded in scaling up the technology to make it mainstream.<\/p>\n<p>&ldquo;The research we are pursuing is to take on perennial problems such as battery, computing, latency, connectivity, and bandwidth in the context of this vision. Slowly, but surely, we are making headway, and I am confident that, together, we will build enough technology to make continuous mobile vision a reality for everyone.&rdquo;<\/p>\n<p>A video of Bahl&#8217;s Mobile Summit talk is available by clicking on the Day 2 tab of the <em>MIT Technology Review<\/em> <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" title=\"video website\" href=\"http:\/\/www.technologyreview.com\/summit\/mobile\/video\/\" target=\"_blank\">video website<span class=\"sr-only\"> (opens in new tab)<\/span><\/a> for the event.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Tweet Posted by Rob Knies You might not be aware of the term &ldquo;continuous mobile vision,&rdquo; but I&rsquo;ll bet there&rsquo;s a good chance you are aware of one of the scenarios it could enable. Remember the concept, bandied about in recent years, of technology that can remind you of a person&rsquo;s name once her or [&hellip;]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[200291,200733,194915,200939,200949,201101,201183,201469,201947,202443,202611,196463,196519,202857,202865,202879,202887,203203,197023,203567,204037,197694],"research-area":[],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-315","post","type-post","status-publish","format-standard","hentry","category-research-blog","tag-aggressive-standby","tag-bodhi-priyantha","tag-carnegie-mellon-university","tag-clock-scaling","tag-cloud-powered-sight-for-all-showing-the-cloud-what-you-see","tag-continuous-mobile-vision","tag-dan-siewiorek","tag-energy-characterization-and-optimization-of-image-sensing-toward-continuous-mobile-vision","tag-image-sensors","tag-lin-zhong","tag-matthai-philipose","tag-microsoft-research-redmond","tag-mit","tag-mit-technology-review-mobile-summit-2013","tag-mobile-cloud-computing-and-services","tag-mobility-and-networking-research","tag-mobisys-2013","tag-paramvir-bahl","tag-rice-university","tag-robert-likamwa","tag-steve-mann","tag-victor-bahl","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[],"related-projects":[],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"June 12, 2013","formattedExcerpt":"Tweet Posted by Rob Knies You might not be aware of the term &ldquo;continuous mobile vision,&rdquo; but I&rsquo;ll bet there&rsquo;s a good chance you are aware of one of the scenarios it could enable.Remember the concept, bandied about in recent years, of technology that can&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/315","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/comments?post=315"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/315\/revisions"}],"predecessor-version":[{"id":261537,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/posts\/315\/revisions\/261537"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=315"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/categories?post=315"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/tags?post=315"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=315"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=315"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=315"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=315"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=315"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=315"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=315"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=315"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}