{"id":50747,"date":"2024-06-26T08:00:00","date_gmt":"2024-06-26T15:00:00","guid":{"rendered":""},"modified":"2024-09-12T08:21:27","modified_gmt":"2024-09-12T15:21:27","slug":"getting-started-with-delivering-generative-ai-capabilities-in-sql-server-and-azure-sql","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/sql-server\/blog\/2024\/06\/26\/getting-started-with-delivering-generative-ai-capabilities-in-sql-server-and-azure-sql\/","title":{"rendered":"Getting started with delivering generative AI capabilities in SQL Server and Azure SQL"},"content":{"rendered":"\n

AI is transforming everything we do, including how we interact with data. Data is the fuel for AI. Microsoft SQL Server and Azure SQL<\/a> is the data platform to power today\u2019s modern applications with security, performance, and availability, but also have capabilities and support scenarios required in the era of AI.<\/p>\n\n\n\n

Azure SQL and SQL Server support building new generative AI experiences that become supercharged when combined with your data. In addition, SQL brings AI assistance to a new level with copilot experiences for both self-help and natural language to SQL capabilities.<\/p>\n\n\n\n

In this blog post, I\u2019ll share how you can get started with these new AI experiences<\/a>\u2014Azure SQL and SQL Server. First, check out our latest story on Microsoft Mechanics:<\/p>\n\n\n

\"Use<\/a><\/figure>\n\n\n\n

<\/a><\/p>\n\n\n\n

Responsible AI<\/h2>\n\n\n\n

Many conversations about AI starts with a statement on responsible AI<\/a>. Microsoft has established a set of policies, research, engineering efforts, and principles to ensure AI technologies are adopted, implemented, and used in a responsible manner.<\/p>\n\n\n\n

These principles include fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. Your data is your data. One promise for Microsoft is that private data of any user, including prompts and responses, are never used to fine tune a model that Microsoft hosts or implements.<\/p>\n\n\n\n

Generative AI applications with your data<\/h2>\n\n\n\n

One of the motivations for generative AI applications is to become more productive, creative, and efficient through the generation of content in all forms: text, audio, and video. Many of today\u2019s examples for generative AI applications involve the user of a natural language prompt<\/em> and the interaction with a language model<\/em>. Many of you have probably at some point used an application like ChatGPT<\/a> or Microsoft Copilot<\/a> which are great examples of generative AI applications.<\/p>\n\n\n\n

Get smarter with your data<\/h2>\n\n\n\n

While these are great applications, they don’t know about your data. The combination of a generative AI application with your data, for example, stored in a database, can be quite powerful. Generative AI provides methods for smarter searching<\/em> on your data. A common application pattern is to use language models with a prompt application to \u201cchat with your data.\u201d Using the concept of vector embeddings<\/em>, language models allow you to get more precision on questions about your data. In addition, responses to questions are more tailored to your users and searches can often be faster because language models allow you to use the power of natural language. Generative AI applications with your data provide unique intelligence in an interactive manner, including conversations. Language models are trained to provide more context on your search, often giving you more (hence generated) content than you might normally get using common searching techniques within a database engine with a language like SQL.<\/p>\n\n\n\n

Prompt engineering, Retrieval Augmented Generation fundamentals, and hybrid search<\/h2>\n\n\n\n

As you investigate how you can take advantage of generative AI with language models, there are two important concepts to understand:<\/p>\n\n\n\n

Prompt engineering<\/em><\/strong> is the discipline of using high quality and descriptive prompts when interacting with a language model. The concept is simple. The better the prompt, the likelihood of a better response from the model. For example, let\u2019s say you use Microsoft Copilot and type in a prompt like \u201cWhat are the best steak restaurants in Fort Worth, Texas?<\/em>\u201d You will get a good list of steak houses in Fort Worth, Texas based on a search by Copilot of rankings across a broad set of searches. But what if you are on a bit of a tight budget? Instead of looking at the results from the prompt and trying to figure out what prices you can afford you could instead ask \u201cI’m on a tight budget but want to eat at a good steakhouse in Fort Worth, Texas.<\/em>\u201d Now your results are more tailored for what you really want. And since you are interacting with a language model, it understands the phrase \u201ctight budget\u201d means you need choices that are good but affordable.<\/p>\n\n\n\n

While this technique can be great if you are interacting with a model that is trained to help you search the internet, what about your own data? One prompt engineering technique to get smarter with your data is called Retrieval Augmented Generation<\/em><\/strong> (RAG). The concept of RAG is to search for information from a source of data and use those results to augment<\/em> the prompt to the model. For Azure SQL and SQL Server, this could mean using standard SQL techniques to search for data using Transact-SQL (T-SQL), taking these results, and sending them along with the original prompt to the language model. This technique is simple and can be an effective way to get smarter with your data, and this can work with almost any type of data you search, not just SQL databases.<\/p>\n\n\n\n

For Azure SQL and SQL Server, a more sophisticated technique is called hybrid search<\/em><\/strong>. With hybrid search, you can use the power of vector <\/em><\/strong>search combined with the query capabilities of your SQL data. Vector embeddings are numerical representations of data that capture semantic meaning and similarities. The key to embeddings with language models is that the model can generate embeddings based on data like text. This means you can take text data inside your SQL database and use a model to generate embeddings and then store these embeddings in your database. Now anytime you want to search for data inside the database, you can send a prompt to a language model which will generate embeddings for the prompt. And then you can use vector search<\/em><\/strong> techniques to compare the embeddings from the prompt with the embeddings stored in your database. You can then combine the vector search with other techniques you would normally use in T-SQL to find data in your database: a hybrid search.<\/p>\n\n\n\n

There are methods today to use hybrid search completely inside the engine using T-SQL and outside the engine using Microsoft Azure AI Services<\/a> or frameworks like LangChain or Semantic Kernel.<\/p>\n\n\n\n

Get started quickly with Azure AI Services<\/h2>\n\n\n\n

One approach to get started quickly with no code required is to index your SQL database using Azure AI Search and then use Azure OpenAI Service<\/a> to build a simple prompt app and \u201cchat with your data\u201d using a hybrid search technique.<\/p>\n\n\n\n

You can use Azure AI Search<\/a> to build an index based on a table in your SQL Server or Azure SQL database. When you build the index, you can apply a skillset<\/em><\/strong> to generate embeddings based on your data and store the result in the index. Now you can use Azure OpenAI with a prompt application to perform hybrid searches on your data. One example prompt application to perform simple testing is to use Azure AI Studio<\/a>. In addition, as you change your SQL data, the index is automatically updated including the embeddings. The figure below shows the basic flow:<\/p>\n\n\n

\"Use<\/figure>\n\n\n\n

You can see this in action from the latest Microsoft Mechanics video or download a deck<\/a> with demo recordings. One of the interesting aspects of this example is the method of changing the system message to direct the language model to respond in a unique way using the same data<\/em>. This is also a great example of prompt engineering.<\/p>\n\n\n\n

Learn more about Azure SQL in Azure AI Search<\/a>.<\/p>\n\n\n\n

Use hybrid search inside the engine with T-SQL<\/h2>\n\n\n\n

Let\u2019s say instead of using a separate index, you would like to build generative AI capabilities for your application all inside the engine using T-SQL. You can do this in a very powerful way for Azure SQL Database<\/a> today using a combination of vector embeddings, vector search, and other T-SQL search methods. This is a true hybrid search because you are using all the power of the SQL query processor together with a vector search. An example my colleague Davide Mauri has developed uses these techniques to help him find the best restaurant for one of this favorite Italian foods, focaccia bread.<\/p>\n\n\n\n

Davide built an application that stores reviews from restaurants in the form of vector embeddings using Azure OpenAI Service with Azure SQL Database Representational State Transfer (REST) API inside the engine. With this in place, he can take any prompt to search for the best focaccia bread and use the same technique to generate embeddings for the prompt. Then, he can use a new T-SQL vector_distance<\/strong> function to perform a similarity search. The true power of SQL is possible because Davide built queries to combine this vector search with other criteria from spatial types, the new JSON data type, and the new Regular Expression (RegEx) T-SQL capabilities.<\/p>\n\n\n\n

You can see a diagram of how these techniques are combined together below:<\/p>\n\n\n\n

\"Hybrid<\/figure>\n\n\n\n

You can see this demo in action in our Microsoft Mechanics video or download a deck with demo recordings. You can learn more about the new JSON data type (preview)<\/a>. You can also sign-up to preview the new vector search capabilities<\/a> and RegEx in Azure SQL Database<\/a>.<\/p>\n\n\n\n

Building generative AI applications using frameworks<\/h2>\n\n\n\n

There are other methods to build generative AI applications with Azure SQL and SQL Server using frameworks such as:<\/p>\n\n\n\n