{"id":1002690,"date":"2024-01-29T10:10:01","date_gmt":"2024-01-29T18:10:01","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&p=1002690"},"modified":"2024-01-29T10:10:01","modified_gmt":"2024-01-29T18:10:01","slug":"watching-the-air-rise-learning-based-single-frame-schlieren-detection","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/watching-the-air-rise-learning-based-single-frame-schlieren-detection\/","title":{"rendered":"Watching the Air Rise: Learning-Based Single-Frame Schlieren Detection"},"content":{"rendered":"

Detecting air flows caused by phenomena such <\/span>as heat convection is valuable in multiple scenarios, including <\/span>leak identification and locating thermal updrafts for extending <\/span>UAVs\u2019 flight duration. Unfortunately, these flows\u2019 heat signature <\/span>is often too subtle to be seen by a thermal camera. While <\/span>convection also leads to fluctuations in air density and hence <\/span>causes so-called<\/span> schlieren<\/span> \u2013 intensity and color variations in <\/span>images<\/span> \u2013<\/span> existing<\/span> techniques<\/span> such<\/span> as<\/span> Background-oriented <\/span>schlieren (BOS) allow detecting them only against a known <\/span>background and from a static camera, making these approaches <\/span>unsuitable for moving vehicles. In this work we demonstrate <\/span>the feasibility of visualizing air movement by predicting the <\/span>corresponding<\/span> schlieren-induced<\/span> optical<\/span> flow<\/span> from<\/span> a<\/span> single <\/span>greyscale<\/span> image<\/span> captured<\/span> by<\/span> a<\/span> moving<\/span> camera<\/span> against<\/span> an <\/span>unfamiliar background. We first record and label a set of optical <\/span>flows in an indoor setup using standard BOS techniques. We <\/span>then train a convolutional neural network (CNN) by applying <\/span>the previously collected optical flow distortions to a dataset <\/span>containing a mixture of real and synthetically generated images <\/span>to predict the two-dimensional optical flow from a single image. <\/span>Finally, we evaluate our approach on the task of extracting the <\/span>optical flow caused by schlieren from both a static and moving <\/span>camera on previously unseen flow patterns and background <\/span>images.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Detecting air flows caused by phenomena such as heat convection is valuable in multiple scenarios, including leak identification and locating thermal updrafts for extending UAVs\u2019 flight duration. Unfortunately, these flows\u2019 heat signature is often too subtle to be seen by a thermal camera. While convection also leads to fluctuations in air density and hence causes […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"ICRA 2024","msr_doi":"","msr_arxiv_id":"","msr_s2_paper_id":"","msr_mag_id":"","msr_pubmed_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_original_fields_of_study":"","msr_release_tracker_id":"","msr_s2_match_type":"","msr_citation_count_updated":"","msr_published_date":"2024-5-17","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_s2_pdf_url":"","msr_year":0,"msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_match_confidence":0,"msr_microsoftintellectualproperty":true,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13556],"msr-publication-type":[193716],"msr-publisher":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[259330,246685,249835],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1002690","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-field-of-study-drone","msr-field-of-study-machine-learning","msr-field-of-study-robotics"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2024-5-17","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/01\/Watching-the-Air-Rise.pdf","id":"1002702","title":"watching-the-air-rise","label_id":"243109","label":0}],"msr_related_uploader":"","msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[{"id":1002702,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2024\/01\/Watching-the-Air-Rise.pdf"}],"msr-author-ordering":[{"type":"text","value":"Florian Achermann","user_id":0,"rest_url":false},{"type":"text","value":"Julian Andreas Haug","user_id":0,"rest_url":false},{"type":"text","value":"Tobias Zumsteg","user_id":0,"rest_url":false},{"type":"text","value":"Nicholas Lawrance","user_id":0,"rest_url":false},{"type":"text","value":"Jen Jen Chung","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Andrey Kolobov","user_id":30910,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Andrey Kolobov"},{"type":"text","value":"Roland Siegwart","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[199565],"msr_event":[],"msr_group":[],"msr_project":[502862],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":502862,"post_title":"Project Frigatebird: AI for Autonomous Soaring","post_name":"project-frigatebird-ai-for-autonomous-soaring","post_type":"msr-project","post_date":"2018-08-27 23:27:11","post_modified":"2024-04-04 10:26:39","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-frigatebird-ai-for-autonomous-soaring\/","post_excerpt":"Autonomous Soaring: an Open-World Challenge for AI Techniques for automatic decision making under uncertainty have been making great strides in their ability to learn complex policies from streams of observations. However, this progress is happening mostly in --- and has a bias towards --- settings with abundant data or readily available high-fidelity simulators, such as games. Learning algorithms in these environments enjoy luxuries unavailable to AI agents in the open world, including resettable training episodes,…","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/502862"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1002690","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1002690\/revisions"}],"predecessor-version":[{"id":1002705,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1002690\/revisions\/1002705"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1002690"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=1002690"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1002690"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=1002690"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=1002690"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=1002690"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1002690"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1002690"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=1002690"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=1002690"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=1002690"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1002690"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1002690"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}