{"id":191779,"date":"2015-01-21T00:00:00","date_gmt":"2015-01-21T15:15:02","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/msr-research-item\/datahub-a-hosted-platform-for-organizing-managing-sharing-collaborating-and-processing-data-e-store-fine-grained-elastic-partitioning-for-distributed-transaction-processing-systems\/"},"modified":"2016-07-15T15:21:29","modified_gmt":"2016-07-15T22:21:29","slug":"datahub-a-hosted-platform-for-organizing-managing-sharing-collaborating-and-processing-data-e-store-fine-grained-elastic-partitioning-for-distributed-transaction-processing-systems","status":"publish","type":"msr-video","link":"https:\/\/www.microsoft.com\/en-us\/research\/video\/datahub-a-hosted-platform-for-organizing-managing-sharing-collaborating-and-processing-data-e-store-fine-grained-elastic-partitioning-for-distributed-transaction-processing-systems\/","title":{"rendered":"DataHub: A Hosted Platform for Organizing, Managing, Sharing, Collaborating, and Processing Data; E-Store: Fine-Grained Elastic Partitioning for Distributed Transaction Processing Systems"},"content":{"rendered":"
\n

Title: DataHub: A Hosted Platform for Organizing, Managing, Sharing, Collaborating, and Processing Data<\/p>\n

In this talk, I will describe DataHub \u2013 a hosted data collaboration platform we are building at MIT. DataHub is a unified data management and collaboration platform. The DataHub platform is (a) a flexible data-store (files, relational databases, extendible to other data-storage backends) with sharing\/collaboration, security, and other data management capabilities, managed on behalf of different users\/groups, and (b) an app ecosystem which hosts apps for various data-processing activities such as ingestion, curation, integration, discovery, query, analytics, visualization, machine learning, etc. A new application can be written and published to the DataHub App Center using our SDK (thrift-based APIs \u2013 can be compiled into any of the 20+ thrift-supported languages). The DataHub users can use any of the apps from the App Center for processing their data as it fits their need. Essentially, DataHub is a unified, managed, collaborative platform for making data-processing easy. I will also discuss some useful data-processing apps we are building for the DataHub platform: a) Distill: a general purpose, example-based data cleaning\/extraction tool for converting semi-structured text into a structured table, b) DViz: a simple drag\/drop interface for creating visualizations, and c) DataHub Notebook: an IPython extension that enables sophisticated data science directly inside DataHub.<\/p>\n

Title -> E-Store: Fine-Grained Elastic Partitioning for Distributed Transaction Processing Systems<\/p>\n

On-line transaction processing (OLTP) database management systems (DBMSs) often serve time-varying workloads due to daily, weekly or seasonal fluctuations in demand, or because of rapid growth in demand due to a company\u2019s business success. In addition, many OLTP workloads are heavily skewed to \u201chot\u201d tuples or ranges of tuples. For example, the majority of NYSE volume involves only 40 stocks. To deal with such fluctuations, an OLTP DBMS needs to be elastic; that is, it must be able to expand and contract resources in response to load fluctuations and dynamically balance load as hot tuples vary over time. In this talk I will present E-Store, an elastic partitioning framework for distributed OLTP DBMSs. It automatically scales resources in response to demand spikes, periodic events, and gradual changes in an application\u2019s workload. E-Store addresses localized bottlenecks through a two-tier data placement strategy: cold data is distributed in large chunks, while smaller ranges of hot tuples are assigned explicitly to individual nodes. This is in contrast to traditional single-tier hash and range partitioning strategies. Our experimental evaluation of E-Store shows the viability of our approach and its efficacy under variations in load across a cluster of machines. Compared to single-tier approaches, E-Store improves throughput by up to 130% while reducing latency by 80%.<\/p>\n<\/div>\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Title: DataHub: A Hosted Platform for Organizing, Managing, Sharing, Collaborating, and Processing Data In this talk, I will describe DataHub \u2013 a hosted data collaboration platform we are building at MIT. DataHub is a unified data management and collaboration platform. The DataHub platform is (a) a flexible data-store (files, relational databases, extendible to other data-storage […]<\/p>\n","protected":false},"featured_media":198789,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_hide_image_in_river":0,"footnotes":""},"research-area":[],"msr-video-type":[206954],"msr-locale":[268875],"msr-post-option":[],"msr-session-type":[],"msr-impact-theme":[],"msr-pillar":[],"msr-episode":[],"msr-research-theme":[],"class_list":["post-191779","msr-video","type-msr-video","status-publish","has-post-thumbnail","hentry","msr-video-type-microsoft-research-talks","msr-locale-en_us"],"msr_download_urls":"","msr_external_url":"https:\/\/youtu.be\/6-KAgAZDD8M","msr_secondary_video_url":"","msr_video_file":"","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/191779","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-video"}],"version-history":[{"count":0,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video\/191779\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/198789"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=191779"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=191779"},{"taxonomy":"msr-video-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-video-type?post=191779"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=191779"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=191779"},{"taxonomy":"msr-session-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-session-type?post=191779"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=191779"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=191779"},{"taxonomy":"msr-episode","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-episode?post=191779"},{"taxonomy":"msr-research-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-theme?post=191779"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}