{"id":17555,"date":"2016-11-16T07:35:58","date_gmt":"2016-11-16T15:35:58","guid":{"rendered":"https:\/\/blogs.technet.microsoft.com\/dataplatforminsider\/?p=17555"},"modified":"2024-01-22T22:50:37","modified_gmt":"2024-01-23T06:50:37","slug":"announcing-the-next-generation-of-databases-and-data-lakes-from-microsoft","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/sql-server\/blog\/2016\/11\/16\/announcing-the-next-generation-of-databases-and-data-lakes-from-microsoft\/","title":{"rendered":"Announcing the Next Generation of Databases and Data Lakes from Microsoft"},"content":{"rendered":"

This post was authored by Joseph Sirosh, Corporate Vice President of the Microsoft Data Group.<\/em><\/p>\n

\"Microsoft<\/a><\/p>\n

For the past two years, we\u2019ve unveiled several of our cutting-edge technologies and innovative solutions at Connect(); which will be livestreaming<\/a> globally from New York City starting November 16. This year, I am thrilled to announce the next generation of SQL Server and Azure Data Lake, and several new capabilities to help developers build intelligent applications.<\/p>\n

1. Next release of SQL Server with Support for Linux and Docker (Preview)<\/h3>\n

I am excited to announce the public preview of the next release of SQL Server<\/a> which brings the power of SQL Server to both Windows \u2013 and for the first time ever – Linux. Now you can also develop applications with SQL Server on<\/strong> Linux, Docker, or macOS (via Docker)<\/strong> and then deploy to Linux, Windows, Docker, on-premises, or in the cloud.\u00a0 This represents a major step in our journey to making SQL Server the\u00a0platform of choice across operating systems, development languages, data types, on-premises and the cloud.\u00a0 All major features of the relational database engine, including advanced features such as in-memory OLTP, in-memory columnstores, Transparent Data Encryption, Always Encrypted, and Row-Level Security now come to Linux. Getting\u00a0started is easier than ever. You\u2019ll find native Linux installations (more info\u00a0here<\/a>) with familiar RPM and APT packages for Red Hat Enterprise Linux, Ubuntu Linux, and SUSE Linux Enterprise Server. The public preview on Windows and Linux will be available on\u00a0Azure Virtual Machines and as images available on Docker Hub, offering a quick and easy installation within minutes.\u00a0 The Windows download is available on the Technet Eval Center<\/a>.<\/p>\n

We have also added significant improvements into R Services inside SQL Server<\/a><\/span>, such as a very powerful set of machine learning functions that are used by our own product teams across Microsoft. This brings new machine learning and deep neural network functionality with increased speed, performance and scale, especially for handling a large corpus of text data and high-dimensional categorical data. We have just recently showcased SQL Server running more than one million R predictions per second<\/a><\/span> and encourage you all to try out R examples and machine learning templates for SQL Server on GitHub<\/a><\/span>.<\/p>\n

The choice of application development stack with the next release of SQL Server is absolutely amazing \u2013 it includes .NET, Java, PHP, Node.JS, etc. on Windows, Linux and Mac (via Docker). Native application development experience for Linux and Mac developers has been a key focus for this release. Get started with the next release of SQL Server on Linux, macOS (via Docker) and Windows with our developer tutorials<\/a> that show you how to install and use the next release of SQL Server on macOS, Docker, Windows, RHEL and Ubuntu and quickly build an app in a programming language of your choice.<\/p>\n

\"SQL<\/a><\/p>\n

2. SQL Server 2016 SP1<\/h3>\n

We are announcing SQL Server 2016 SP1<\/a> which is a unique service pack – for the first time we introduce consistent programming model <\/strong><\/span>across SQL Server editions. With this model, programs written to exploit powerful SQL features such as in-memory OLTP, in-memory columnstore analytics, and partitioning will work across Enterprise, Standard and Express editions<\/span>. Developers will find it easier than ever to take advantage of innovations such as in memory databases and advanced analytics \u2013 you can use these advanced features in the Standard Edition and then step up to Enterprise for Mission Critical performance, scale and availability \u2013 without having to re-write your application.<\/p>\n

Our software partners are excited about the flexibility that this change gives them to adopt advanced features while supporting multiple editions of SQL Server.<\/p>\n

\u201cWith SQL Server 2016 SP1, we can run the same code entirely on both platforms and customers who need Enterprise scale buy Enterprise, and customers who don\u2019t need that can buy Standard and run just fine. From a programming point of view, it\u2019s easier for us and easier for them,\u201d said Nick Craver, Architecture Lead at Stack Overflow.<\/p><\/blockquote>\n

To be even more productive with SQL Server, you can now take advantage of improved developer experiences<\/strong> on Windows, Mac and Linux for Node.js, Java, PHP, Python, Ruby, .NET core and C\/C++. Our JDBC Connector is now published and available as 100% open source which gives developers more access to information and flexibility on how to contribute and work with the JDBC driver. Additionally, we\u2019ve made updates to ODBC for PHP driver and launched a new ODBC for Linux connector, making it much easier for developers to work with Microsoft SQL-based technologies. To make it more seamless for all developers Microsoft VSCode<\/a><\/span> users can also now connect to SQL Server, including SQL Server on Linux, Azure SQL Database and Azure SQL Data Warehouse.\u00a0 In addition, we\u2019ve released updates to SQL Server Management Studio, SQL Server Data Tools, and Command line tools which now support SQL Server on Linux.<\/p>\n

\"Tools\"<\/a><\/p>\n

3. Azure Data Lake Analytics and Store GA<\/h3>\n

Today, I am excited to announce the general availability of Azure Data Lake Analytics<\/b><\/a><\/span> and Azure Data Lake Store<\/b><\/a><\/span>.<\/p>\n

Azure Data Lake Analytics<\/b><\/a> is a cloud analytics service that allows you to develop and run massively parallel data transformations and processing programs in U-SQL, R, Python and .Net over petabytes of data with just a few lines of code. There is no infrastructure to manage, and you can process data on demand allowing you to scale in seconds, and only pay for the resources used. U-SQL<\/a> is a simple, expressive, and super-extensible language that combines the power of C# with the simplicity of SQL. Developers can write their code either in Visual Studio or Visual Studio Code and the execution environment gives you debugging and optimization recommendations to improve performance and reduce cost.<\/p>\n

Azure Data Lake Store<\/b><\/a> is a cloud analytics data lake for enterprises that is secure, massively scalable and built to the open HDFS standard. You can store trillions of files, and single files can be greater than a petabyte in size. It provides massive throughput optimized to run big analytic jobs. It has data encryption in motion and at rest, single sign-on (SSO), multi-factor authentication and management of identities built-in through Azure Active Directory, and fine-grained POSIX-based ACLS for role-based access controls.<\/p>\n

\"Azure<\/a><\/p>\n

Furthermore, we\u2019ve incorporated the technology that sits behind the Microsoft Cognitive Services<\/a><\/span> inside U-SQL directly. Now you can process any amount of unstructured data, e.g., text, images, and extract emotions, age, and all sorts of other cognitive features using Azure Data Lake and perform query by content. You can join emotions from image content with any other type of data you have and do incredibly powerful analytics and intelligence over it. This is what I call Big Cognition<\/strong>. It\u2019s not just extracting one piece of cognitive information at a time, not just about understanding an emotion or whether there\u2019s an object in an image, but rather it\u2019s about joining all the extracted cognitive data with other types of data, so you can do some really powerful analytics with it. We have demonstrated this capability at Microsoft Ignite<\/a><\/span> and PASS Summit<\/a>, by showing a Big Cognition demo in which we used U-SQL inside Azure Data Lake Analytics to process a million images<\/a><\/span> and understand what\u2019s inside those images. You can watch this demo<\/a> (starting at minute 38) and try it yourself using a sample project on GitHub<\/a><\/span>.<\/p>\n

4. DocumentDB Emulator<\/h3>\n

We live on a Planet of the Apps<\/em>, and the best back-end system to build modern intelligent mobile or web apps is Azure DocumentDB<\/a><\/span> – planet-scale, globally distributed managed NoSQL service, with 99.99% availability and guarantees for low latency and consistency, all of which is backed by an enterprise grade security and SLA.<\/p>\n

Today I am happy to announce a public preview of DocumentDB Emulator<\/b><\/a> which provides a local development experience for the Azure DocumentDB<\/a><\/span>. Using the DocumentDB Emulator, you can develop and test your application locally without an internet connection, without creating an Azure subscription, and without incurring any costs. This has long been the most requested feature<\/a> on the user voice site, so we are thrilled to roll this out to everyone.<\/p>\n

Furthermore, we\u2019ve added .NET Core<\/a> support in DocumentDB. The .Net Core is a lightweight and modular platform to create applications and services that run on Linux, Mac and Windows. With DocumentDB support for .Net Core, developers can now use .Net Core to build cross platform applications and services that use DocumentDB API.<\/p>\n

\"Planet<\/a><\/p>\n

5. Other Announcements<\/h3>\n