{"id":558237,"date":"2018-12-26T23:56:35","date_gmt":"2018-12-27T07:56:35","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=558237"},"modified":"2020-11-19T19:39:41","modified_gmt":"2020-11-20T03:39:41","slug":"deep-learning-and-representation-learning","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/deep-learning-and-representation-learning\/","title":{"rendered":"Deep Learning and Representation Learning"},"content":{"rendered":"<p>We are working on deep learning. We focus on developing new learning strategies and more efficient algorithms, designing better neural network structures, and improving representation learning.<\/p>\n<p><strong>Efficient Deep Learning<\/strong><br \/>\nXiang Li, Tao Qin, Jian Yang, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"\/\/arxiv.org\/abs\/1610.09893\"\">LightRNN: Memory and Computation-Efficient Recurrent Neural Networks<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>NIPS<\/strong> 2016. [<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"\/\/github.com\/Microsoft\/CNTK\/tree\/master\/Examples\/Text\/LightRNN\"\">Code@GitHub<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>]<br \/>\nFei Gao, Lijun Wu, Li Zhao, Tao Qin, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/aclweb.org\/anthology\/N18-1073\">Efficient Sequence Learning with Group Recurrent Networks<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>NAACL<\/strong> 2018.<br \/>\nZhuohan Li, Di He, Fei Tian, Wei Chen, Tao Qin, Liwei Wang, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/1806.02988\">Towards Binary-Valued Gates for Robust LSTM Training<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>ICML<\/strong> 2018. [<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/zhuohan123\/g2-lstm\">code<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>] [<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/mp.weixin.qq.com\/s\/8BPZ_M8EGk3KxkSleYWSNw\">Chinese article<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>]<\/p>\n<p><strong>Improving Representations<\/strong><br \/>\nJun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, Tieyan Liu\uff0c<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/openreview.net\/pdf?id=SkEYojRqtm\">Representation Degeneration Problem in Training Natural Language Generation Models<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>ICLR<\/strong> 2019.<br \/>\nChengyue Gong, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/pdf\/1809.06858\">FRAGE: Frequency-Agnostic Word Representation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>NIPS <\/strong>2018. [<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/github.com\/ChengyueGongR\/Frequency-Agnostic\">code<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>]<\/p>\n<p><strong>Advanced Learning Strategies<\/strong><br \/>\nDi He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"\/\/arxiv.org\/pdf\/1611.00179\"\">Dual Learning for Machine Translation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>NIPS<\/strong> 2016.<br \/>\nYingce Xia, Tao Qin, Wei Chen, Jiang Bian, Nenghai Yu and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"\/\/arxiv.org\/pdf\/1707.00415\"\">Dual Supervised Learning<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <b>ICML <\/b>2017<b>.<\/b><br \/>\nLijun Wu, Li Zhao, Tao Qin, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"\/\/www.ijcai.org\/proceedings\/2017\/0432.pdf\"\">Sequence Prediction with Unlabeled Data by Reward Function Learning<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>IJCAI<\/strong> 2017.<br \/>\nYingce Xia, Jiang Bian, Tao Qin, Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"\/\/www.ijcai.org\/proceedings\/2017\/0434.pdf\"\">Dual Inference for Machine Learning<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>IJCAI<\/strong> 2017.<br \/>\nYingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/papers.nips.cc\/paper\/6775-deliberation-networks-sequence-generation-beyond-one-pass-decoding.pdf\">Deliberation Networks: Sequence Generation Beyond One-Pass Decoding<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>NIPS<\/strong> 2017.<br \/>\nDi He, Hanqing Lu, Yingce Xia, Tao Qin, Liwei Wang, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/papers.nips.cc\/paper\/6622-decoding-with-value-networks-for-neural-machine-translation.pdf\">Decoding with Value Networks for Neural Machine Translation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>NIPS<\/strong> 2017.<br \/>\nYingce Xia, Xu Tan, Fei Tian, Tao Qin, Nenghai Yu, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/proceedings.mlr.press\/v80\/xia18a\/xia18a.pdf\">Model-Level Dual Learning<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>ICML<\/strong> 2018.<br \/>\nYiren Wang, Yingce Xia, Tianyu He, Fei Tian, Tao Qin, ChengXiang Zhai, Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/openreview.net\/pdf?id=HyGhN2A5tm\">Multi-Agent Dual Learning<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>ICLR<\/strong> 2019.<br \/>\nChengyue Gong, Xu Tan, Di He, and Tao Qin, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/1812.04784\">Sentence-wise Smooth Regularization for Sequence to Sequence Learning<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>AAAI<\/strong> 2019.<\/p>\n<p><strong>New Network Structures<\/strong><br \/>\nKaitao Song, Xu Tan, Di He, Jianfeng Lu, Tao Qin, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/1806.04856\">Double Path Networks for Sequence to Sequence Learning<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>COLING<\/strong> 2018.<br \/>\nJianxin Lin, Yingce Xia, Tao Qin, Zhibo Chen, and Tie-Yan Liu, <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/pdf\/1805.00251\">Conditional Image-to-Image Translation<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, <strong>CVPR<\/strong> 2018.<br \/>\nChang Xu, Tao Qin, Yalong Bai, Gang Wang and Tie-Yan Liu, Convolutional Neural Networks for Posed and Spontaneous Expression Recognition, <strong>ICME<\/strong> 2017.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We are working on deep learning. We focus on developing new learning strategies and more efficient algorithms, designing better neural network structures, and improving representation learning. Efficient Deep Learning Xiang Li, Tao Qin, Jian Yang, and Tie-Yan Liu, Code@GitHub] Fei Gao, Lijun Wu, Li Zhao, Tao Qin, and Tie-Yan Liu, Efficient Sequence Learning with Group [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-558237","msr-project","type-msr-project","status-publish","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[],"related-downloads":[],"related-videos":[1132209],"related-groups":[705946],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Li Zhao","user_id":36152,"people_section":"Section name 1","alias":"lizo"}],"msr_research_lab":[199560],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/558237","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/558237\/revisions"}],"predecessor-version":[{"id":558243,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/558237\/revisions\/558243"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=558237"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=558237"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=558237"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=558237"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=558237"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}