{"id":1031460,"date":"2024-08-16T12:50:17","date_gmt":"2024-08-16T19:50:17","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&p=1031460"},"modified":"2025-01-08T09:02:34","modified_gmt":"2025-01-08T17:02:34","slug":"guidance-control-lm-output","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/guidance-control-lm-output\/","title":{"rendered":"guidance | control LM output"},"content":{"rendered":"
\n\t
\n\t\t
\n\t\t\t\"background\t\t<\/div>\n\t\t\n\t\t
\n\t\t\t\n\t\t\t
\n\t\t\t\t\n\t\t\t\t
\n\t\t\t\t\t\n\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n

Guidance<\/h1>\n\n\n\n

Control LM outputs. Reduce latency and cost.<\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n

Get the LM output you need with a single prompt<\/h2>\n\n\n\n

Guidance <\/strong>is a proven open-source Python library (opens in new tab)<\/span><\/a> for controlling outputs of any language model (LM). In only one API call, developers express in Python the precise programmatic constraints the model must follow for structured output in JSON, Python, HTML, SQL, whatever the use case requires. <\/p>\n\n\n\n

The result: 100% guaranteed output structure\u2014with 30\u201350% reduction in latency and costs.<\/h5>\n\n\n\n
\n
Get started with {guidance}<\/a><\/div>\n<\/div>\n\n\n\n

Steering the model token by token <\/h3>\n\n\n\n

Guidance works with most open-source LMs that can be hosted locally. Fundamentally different from conventional prompting techniques, Guidance enforces constraints by steering the model token by token in the inference layer to deliver accurate outputs. No need for expensive retries or fine-tuning. The Guidance advantage includes:<\/p>\n\n\n\n

<\/div>\n\n\n\n
\n
\n

Cost savings<\/h4>\n\n\n\n

Save significantly on runtime, while accelerating inference. In contrast to prompt chaining, Guidance programs are a single API call. A Guidance program batches\u2014instead of generating\u2014any additional text that is added by the user as execution unrolls. See example (opens in new tab)<\/span><\/a>.<\/p>\n<\/div>\n\n\n\n

\n

Flexibility<\/h4>\n\n\n\n

Get structured LM output in any specified format. Guidance is uniquely flexible compared with alternative technologies, enabling developers to constrain outputs to JSON, Python, HTML, SQL, whatever is required. Enforce other constraints with select (opens in new tab)<\/span><\/a> (i.e., a set of options), regular expressions (opens in new tab)<\/span><\/a>, and context-free grammars (opens in new tab)<\/span><\/a>.<\/p>\n<\/div>\n\n\n\n

\n

Elegant workflow<\/h4>\n\n\n\n

Write constraints in pure Python and Guidance enforces syntax for a smooth developer workflow. The Guidance interface and library functionality are designed to reduce developer pain. Call and deploy tools easily (opens in new tab)<\/span><\/a>. Access rich templates with f-strings and prebuilt components (e.g., substrings). More … (opens in new tab)<\/span><\/a><\/p>\n<\/div>\n<\/div>\n\n\n\n