PROSE

PROSE

The PROSE research and engineering team, led by Sumit Gulwani, develops APIs for program synthesis (programming by examples and natural language) and incorporates them into real products.

Programming by examples (PBE) is a new frontier in AI that enables users to create scripts from input-output examples. PBE can provide a 10-100x productivity increase for developers in some task domains. 99% of computer users are non-programmers and PBE can enable them to create small scripts to automate repetitive tasks. PBE is revolutionizing data wrangling. Data scientists spend up to 80% time transforming data into a form suitable for machine learning (ML).

Integrated into Microsoft products

PROSE is integrated into Microsoft Excel​ and Visual Studio​, as well as these products and tools:

Azure Data Factory (ADF)

Azure Data Studio (ADS)

Azure Monitor​

PowerBI​

Power Query​

Powershell​

Data Migration Assistant (DMA)

SQL Server Migration Assistant​ (SSMA)

SQL Server Management Studio (SSMS)

Microsoft Program Synthesis using Examples (PROSE)

Given a domain-specific language (DSL) and input-output examples for the desired program’s behavior, PROSE synthesizes a ranked set of DSL programs that are consistent with the examples.

Download NuGet package

NuGet package for automatically synthesizing programs from examples.

Tutorial

A step-by-step walkthrough of the process of building a DSL in PROSE and enabling program synthesis for it.

API and samples

If you want to apply an existing PROSE DSL, check out its documentation and our samples.


Release notes

Read the release notes for the SDK.

GitHub

PROSE is not open source, but we do publish on GitHub sample code using the SDK.  For questions, email us or open an issue on GitHub.

FAQ

Get answers to frequently asked questions.

Customer quotes about PROSE

Flash Fill (Excel)

​​​​“Perhaps the closest thing to magic Excel users will ever see in a workbook is Flash Fill.”

​— Susan Harkins, TechRepublic
http://www.techrepublic.com/blog/10-things/10-best-new-features-in-excel-2013/​

What customers say when PROSE technology is taken away from them (on missing Flash Fill feature in Excel for MAC):
“This is one of the main reason I purchased Office rather than sticking with free programs, only to find out that it is not supported on Mac versions of Excel. Absolutely unbelievable and disappointing.”

“Excel without flash fill will be in shambles“

“I thought Excel for Mac has the same functionalities as the Windows version. Apparently not. Disappointed in Microsoft!!! We want flash fill!!”

“You NEED to Add Flash Fill for Mac ASAP or I will be forced to stop using Microsoft Services completely. This is using to much of my time.”

“please add this for Mac! the company just gave us Macs & now I’m mi​​ss​in​g one of the best features in Excel :(“

“PLEASE add the flash fill feature for MAC! It is an amazing tool, I can’t believe you don’t already feature it on your Mac versions“

“Flash Fill is an extremely powerful and useful feature. Porting this to the Mac would greatly expand it’s usefulness, and lock in Excel as a key tool for anyone analyzing data, regardless of what OS they are running.”

Table Extraction from Text Files

​Custom Fields (Azure Log Analytics)​

“I’ve been blown away by what customers are able to do with it. Once customer had a requirement where they needed to use alerts in OMS via custom fields over SCOM data to extract AD information. The information was complicated and laid out in way where extraction via Regex would have been difficult and the customer did not want to learn Regex. Custom fields enabled them to extract the data easily so they could build alerts and dashboards based on these new fields.

Custom fields is very useful and is used in all my demos. Once I dug into it I found so many use cases! All of our verbose alert descriptions are great now because now I can pull the data out easily and put it into visualizations. The accuracy of the field extractions has improved so much since the preview I haven’t had any issues in a long time.”

— Shawn Tierney, MVP, Infront Consulting

ConvertFrom-String (PowerShell)

“PowerShell v5 ConvertFrom-String query by example templates are brilliant”

— Doug Finke, PowerShell MVP

Blog posts:

“Just…amazing. The number of transformations it can run through for this is incredible. Can’t wait to do a video on this new feature.”

— Reid Havens, Power BI MVP

“Not certain how you managed to get me excited about TEXT files but you did it!”

— Charles Sterling

​​Table Extraction from WebPages (PowerQuery)​​

Voted top 5 out of 150 new features in Power Query in 2018.

“For anyone that has futilely struggled expanding their way through the DOM of a web page in Query Editor, “Web by Example” is the best thing since sliced virtual network architectures over 5G.”

— Greg Deckler​

“Indeed is a great feature. I’ve been doing it with Python and beautifulsoup. having that feature in PowerBI is great! :)”

“I got so excited about this the day of release! I’ve managed to get so many obscure things working!”

“That is absolutely fantastic. I use the web connector a lot, it is one of my favorite connectors”

“A very exciting improvement to getting data from the web”

“This update is really cool.”

YouTube videos created by users:

Table Extraction from PDF (PowerQuery)

Top requested feature on Power BI Ideas (3000+ votes)

“PowerBIs built in data from PDF function. It works very well is has been reliable throughout my testing.”
​

— dantm on Reddit

YouTube videos:

BluePencil (IntelliCode Suggestions in Visual Studio)​

“this is blowing my mind”

— Cyrus Najmabadi, Microsoft software engineer

“​I’ve just tried #IntelliCode refactor in @VisualStudio and it’s amazing. What a time saver.”​

— Cezary Piatek on twitte​r

“I was showing it to my teammate last Friday. It’s pretty cool”

— Chee Seong Ong, Microsoft software engineer​​

“Sometimes it’s really, really clever. It does things that are really clever. It is kind of like, “woah. I just realised what you’re doing.”

— Mark ​Rendle, The .NET Core Podcast​​

I also have to admit that I was skeptical at first, but found myself doing repetitive work and just wanting to finish it so I gave it a try. It was pretty magical and now I would use it again with much less hesitation.

— Travis Liskey, Microsoft software engineer​​

Huh?! When did @VisualStudio IntelliCode start to suggest Quick Actions based on recent edits? That’s pretty neat

— Mark Seemann on twitter​​

Blog posts:

Data.Diff

Find differences in distribution of numeric, string, and categorical data.

Detection

Libraries for detecting data, encoding and file types.

Extraction.Json

Extraction.Json automatically extracts tabular data from Json files. It supports extracting Newline Delimited Json and truncated Json.

Extraction.Pdf

Extract tabular data from PDFs.

Extraction.Spreadsheets

Extract tabular data from Excel files.

Extraction.Text

Extraction.Text extracts data from semi-structured text files using examples. The Usage page and the Sample project illustrate the API usage.

Extraction.Web

Extract tabular data from webpages.

Matching.Text

Matching.Text automatically identifies different formats and patterns in string data.

Read.FlatFile

Automatically infer properties of CSV and fixed-width files (skipped lines, delimiters, etc.) to parse the files.

Split.Text

Split.Text is a system for splitting data in plain text format, where there may be multiple fields that need to be separated into different columns.

Transformation.Json

Transformation.Json transforms the structure of Json using input/output examples.

Transformation.Text

Transformation.Text is a system that performs string transformations using examples allowing for many tasks involving strings to be performed automatically.

Transformation.Tree

Transform ASTs using examples.