{"id":2652,"date":"2023-08-11T08:00:24","date_gmt":"2023-08-11T16:00:24","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/"},"modified":"2025-06-30T06:10:57","modified_gmt":"2025-06-30T13:10:57","slug":"automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution","status":"publish","type":"copilot","link":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/","title":{"rendered":"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution"},"content":{"rendered":"\n<p>When enterprise customers deploy large or complex chatbots using Power Virtual Agents, they need to ensure that the bots behave as intended.<\/p>\n\n\n\n<p>This involves testing user utterances to confirm that the bot understands them correctly and triggers the appropriate topics. It also entails verifying that the full conversation leads to the intended outcome, such as resolution or escalation.\u00a0Customers might also want to ensure that their chatbots meet quality requirements before deploying to production, which can be done automatically using Continuous Integration\/Continuous Delivery (CI\/CD) approaches like <a href=\"https:\/\/learn.microsoft.com\/en-us\/power-platform\/alm\/devops-build-tools\" target=\"_blank\" rel=\"noreferrer noopener\">Azure DevOps<\/a>.<\/p>\n\n\n\n<p>The Power Virtual Agents teams have recently released new code samples to help automate chatbot tests, including testing utterances, validating topic triggering, intent candidates (multiple topics matched), playback of full conversation transcripts, and integration into a CI\/CD pipeline.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"what-can-be-tested-with-the-pva-test-framework-sample-solution\">What can be tested with the PVA Test Framework sample solution?<\/h2>\n\n\n\n<p>The <a href=\"https:\/\/aka.ms\/PVASamples\/PVATestFramework\" target=\"_blank\" rel=\"noreferrer noopener\">PVA Test Framework<\/a> is a sample solution available on GitHub that showcases how to execute tests against a Power Virtual Agents chatbot through APIs, using the Direct Line channel. It verifies that the bot performs as expected in various scenarios, such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Testing the Natural Language Understanding model (topic triggering)<\/li>\n\n\n\n<li>Validating the multiple topics matched options (&#8220;did you mean&#8230;&#8221;)<\/li>\n\n\n\n<li>Performing scale load testing<\/li>\n\n\n\n<li>Testing full end-to-end conversations<\/li>\n\n\n\n<li>Testing adaptive cards<\/li>\n\n\n\n<li>Including a CI\/CD test step in the deployment pipeline to prevent deployment if tests fail<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"how-does-the-pva-test-framework-sample-solution-work\">How does the PVA Test Framework sample solution work?<\/h2>\n\n\n\n<p>The\u00a0<a href=\"https:\/\/aka.ms\/PVASamples\/PVATestFramework\" target=\"_blank\" rel=\"noreferrer noopener\">PVA Test Framework<\/a> tool is a console application that can run tests against the Power Virtual Agents Direct Line API. It uses JSON as the format to play conversations back and verify outcomes, and it enables you to connect to Dataverse to download past conversation transcripts to run them again.<\/p>\n\n\n<figure class=\"wp-block-image is-resized has-custom-border\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2023\/05\/architecture-diagram-1024x593.webp\" alt=\"diagram\" class=\"wp-image-2655 webp-format\" style=\"border-radius:0px;width:520px;height:auto\" data-orig-src=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2023\/05\/architecture-diagram.webp\"><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"how-to-use-the-pva-test-framework-tool\">How to use the PVA Test Framework tool<\/h2>\n\n\n\n<p>There are different ways you can use the&nbsp;PVA Test Framework sample solution. The easiest one is to run it locally with the Command Prompt in Windows.<\/p>\n\n\n\n<p>You first need to <a href=\"https:\/\/github.com\/microsoft\/PowerVirtualAgentsSamples\/tree\/master\/PVATestFramework\/Tool\" target=\"_blank\" rel=\"noreferrer noopener\">download<\/a> or <a href=\"https:\/\/github.com\/microsoft\/PowerVirtualAgentsSamples\/blob\/master\/PVATestFramework\/PUBLISH.MD\" target=\"_blank\" rel=\"noreferrer noopener\">build<\/a> the&nbsp;<strong>PVATestFramework.Console.exe<\/strong> file.<br>It then works as a console application, with command lines.<\/p>\n\n\n\n<p>To run tests, you need conversation files in a .JSON format. You can either easily create these files from scratch, by writing your test cases in a .CHAT file format and converting them to .JSON, or you can&nbsp;<a href=\"https:\/\/github.com\/microsoft\/PowerVirtualAgentsSamples\/blob\/master\/PVATestFramework\/TOOL_USAGE.MD#how-to-download-a-file-from-dataverse\" target=\"_blank\" rel=\"noreferrer noopener\">download conversation transcripts from Dataverse<\/a>.<\/p>\n\n\n\n<p>For Natural Language Understanding testing, where you test many user utterances and verify that the bot answers as expected, it&#8217;s usually easier to start from your own set of&nbsp;user and bot utterances in a .CHAT format.<\/p>\n\n\n\n<p>This command transforms a .CHAT file into a .JSON file:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\n.\\PVATestFramework.exe convertChatFile --path test-set-sample.chat --outputFile test-set-sample.json\n<\/pre><\/div>\n\n\n<p>Here is what conversations look like a .CHAT format:<\/p>\n\n\n\n<p>Each test conversation is separated by the <strong>&lt;EOC&gt;<\/strong> tag (end of conversation).<\/p>\n\n\n\n<p>Messages following the user: tag are sent by the tool to the bot, and the messages received back from the bot are compared\u00a0with the expected response, documented after the\u00a0bot:\u00a0tag, to determine if the test passes or fails.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\nuser: Good morning\nbot: Hello, how can I help you today?\n<EOC>\nuser: When are you closed\nbot: I'm happy to help with store hours.\n<EOC>\nuser: Find me your nearest location\nbot: I'd be glad to help find a store near you.\n<EOC>\nuser: Where can I find your address?\nbot: I'd be glad to help find a store near you.\n<EOC>\nuser: I'd like to order a product\nbot: I am happy to help you place your order.\n<EOC>\nuser: what are you store hours and locations?\nsuggested: Lesson 2 - A simple topic with a condition and variable|Lesson 1 - A simple topic||None of these\nbot: To clarify, did you mean:\nuser: Lesson 1 - A simple topic\nbot: I'm happy to help with store hours.\n<EOC>\nuser: What is weather today?\nbot: I'm sorry, I'm not sure how to help with that. Can you try rephrasing?\n<EOC>\nuser: What is weather today?\nbot: I'm sorry, I'm not sure how to help with that. Can you try rephrasing?\n<EOC>\n<\/pre><\/div>\n\n\n<p>Once converted to .JSON, you can run tests by using the below sample command. You can obtain\u00a0<a href=\"https:\/\/learn.microsoft.com\/power-virtual-agents\/publication-connect-bot-to-custom-application#retrieve-your-power-virtual-agents-bot-parameters\" target=\"_blank\" rel=\"noreferrer noopener\">the chatbot token endpoint<\/a> from the Mobile app channel in Power Virtual Agents.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\n.\\PVATestFramework.exe test --path test-set-sample.json --tokenEndpoint https:\/\/c53bf00279234d1cb5ae3265093d59.e1.environment.api.powerplatform.com\/powervirtualagents\/botsbyschema\/cr507_testChatbot\/directline\/token?api-version=2022-03-01-preview --log --verbose\n<\/pre><\/div>\n\n\n<figure class=\"wp-block-image has-custom-border\"><img loading=\"lazy\" decoding=\"async\" width=\"1114\" height=\"624\" src=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2023\/05\/2023-06-01-11-34-25-835-hejammes-vm_4_-_20.199.19.1643389_-_Remote_Deskt.gif\" alt=\"code gif\" class=\"wp-image-2732\" style=\"border-radius:0px\" \/><\/figure>\n\n\n\n<p>Test outcomes are then available in a CSV format that you can then use for further analysis.<\/p>\n\n\n\n<p>Example analysis of the outputs of the PVA Test Framework:<\/p>\n\n\n<figure class=\"wp-block-image is-resized has-custom-border\"><img decoding=\"async\" src=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2023\/06\/2023-06-01-16-50-04-158.webp\" alt=\"table\" class=\"wp-image-2733 webp-format\" style=\"border-radius:0px;width:494px;height:auto\" data-orig-src=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2023\/06\/2023-06-01-16-50-04-158.webp\"><\/figure>\n\n\n\n<p>You can learn more about the PVA Test Framework sample solution&#8217;s different settings and deployment options in the <a href=\"https:\/\/github.com\/microsoft\/PowerVirtualAgentsSamples\/blob\/master\/PVATestFramework\/README.md\">GitHub documentation<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"frequently-asked-questions\">Frequently Asked Questions<\/h2>\n\n\n\n<h6 class=\"wp-block-heading\" id=\"what-is-the-cost-of-using-the-pva-test-framework-sample-solution\"><em><strong>What is the cost of using the PVA Test Framework sample solution?<\/strong><\/em><\/h6>\n\n\n\n<p>Conversations generated using the PVA Test Framework sample solution would typically generate billed sessions. See:&nbsp;<a href=\"https:\/\/powervirtualagents.microsoft.com\/pricing\/\" target=\"_blank\" rel=\"noreferrer noopener\">Power Virtual Agents pricing<\/a>.<\/p>\n\n\n\n<h6 class=\"wp-block-heading\" id=\"how-is-the-pva-test-framework-supported-by-microsoft\"><strong><em>How is the PVA Test Framework supported by Microsoft?<\/em><\/strong><\/h6>\n\n\n\n<p>Although the underlying features and components used to build the PVA Test Framework are fully supported, the code itself represents a sample implementation of these features.<\/p>\n\n\n\n<p>Our customers and community can use and customize these features to meet their requirements. Any issue with the PVA Test Framework sample solution should be raised on the Power Virtual Agents Samples&nbsp;<a href=\"https:\/\/aka.ms\/PVASamples\/\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub repository<\/a>&nbsp;and not through Microsoft Support.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The PVA Test Framework for Power Virtual Agents is a sample\u00a0solution published on GitHub that demonstrates how to run tests against a chatbot using the Direct Line channel, and validate that the bot works as expected .<\/p>\n","protected":false},"author":70,"featured_media":1919,"template":"","cs-content-type":[933],"cs-topic":[939,940,941],"coauthors":[684],"class_list":["post-2652","copilot","type-copilot","status-publish","has-post-thumbnail","hentry","cs-content-type-tips-and-guides","cs-topic-agent-governance","cs-topic-agentic-ai","cs-topic-extensibility"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution | Microsoft Copilot Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution | Microsoft Copilot Blog\" \/>\n<meta property=\"og:description\" content=\"The PVA Test Framework for Power Virtual Agents is a sample\u00a0solution published on GitHub that demonstrates how to run tests against a chatbot using the Direct Line channel, and validate that the bot works as expected .\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Copilot Blog\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-30T13:10:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"534\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Henry Jammes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/\"},\"author\":[{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/author\/henry-jammes\/\",\"@type\":\"Person\",\"@name\":\"Henry Jammes\"}],\"headline\":\"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution\",\"datePublished\":\"2023-08-11T16:00:24+00:00\",\"dateModified\":\"2025-06-30T13:10:57+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/\"},\"wordCount\":706,\"publisher\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp\",\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/\",\"name\":\"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution | Microsoft Copilot Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp\",\"datePublished\":\"2023-08-11T16:00:24+00:00\",\"dateModified\":\"2025-06-30T13:10:57+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp\",\"width\":1024,\"height\":534,\"caption\":\"Placeholder\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Copilot Studio\",\"item\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#website\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/\",\"name\":\"Microsoft Copilot Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#organization\",\"name\":\"Microsoft Copilot Blog\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/05\/cropped-microsoft_logo_element.webp\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/05\/cropped-microsoft_logo_element.webp\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Copilot Blog\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#\/schema\/person\/86838a807c26de6c2b449ff238171cc4\",\"name\":\"Henry Jammes\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/08\/Henry-Jammes-PP-2022-2-96x96.jpega34837d73f2677c6be105c24b22706fb\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/08\/Henry-Jammes-PP-2022-2-96x96.jpeg\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/08\/Henry-Jammes-PP-2022-2-96x96.jpeg\",\"caption\":\"Henry Jammes\"},\"description\":\"Henry Jammes is a leader in the Microsoft Copilot Studio customer advisory team, part of the Power CAT team within Microsoft Engineering. With deep expertise in generative AI, conversational AI, and agentic AI, Henry and his team collaborate closely with top-tier enterprise customers as well as Microsoft\u2019s product leadership and program managers. Passionate about connecting technology to real-world customer outcomes, his team actively channels customer feedback into product enhancements, creates practical guidance, delivers impactful samples, and contributes broadly through community engagement and public speaking to accelerate customer success.\",\"sameAs\":[\"https:\/\/www.linkedin.com\/in\/henryjammes\/\"],\"url\":\"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/author\/hejammes\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution | Microsoft Copilot Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/","og_locale":"en_US","og_type":"article","og_title":"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution | Microsoft Copilot Blog","og_description":"The PVA Test Framework for Power Virtual Agents is a sample\u00a0solution published on GitHub that demonstrates how to run tests against a chatbot using the Direct Line channel, and validate that the bot works as expected .","og_url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/","og_site_name":"Microsoft Copilot Blog","article_modified_time":"2025-06-30T13:10:57+00:00","og_image":[{"width":1024,"height":534,"url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes","Written by":"Henry Jammes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#article","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/"},"author":[{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/author\/henry-jammes\/","@type":"Person","@name":"Henry Jammes"}],"headline":"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution","datePublished":"2023-08-11T16:00:24+00:00","dateModified":"2025-06-30T13:10:57+00:00","mainEntityOfPage":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/"},"wordCount":706,"publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#organization"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage"},"thumbnailUrl":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp","inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/","url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/","name":"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution | Microsoft Copilot Blog","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage"},"thumbnailUrl":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp","datePublished":"2023-08-11T16:00:24+00:00","dateModified":"2025-06-30T13:10:57+00:00","breadcrumb":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#primaryimage","url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp","contentUrl":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2021\/10\/Environment-lifecycle-mangement-1024x534-1.webp","width":1024,"height":534,"caption":"Placeholder"},{"@type":"BreadcrumbList","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/automate-testing-of-your-power-virtual-agents-chatbots-with-the-pva-test-framework-sample-solution\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/"},{"@type":"ListItem","position":2,"name":"Copilot Studio","item":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/copilot-studio\/"},{"@type":"ListItem","position":3,"name":"Automate testing of your Power Virtual Agents chatbots with the PVA Test Framework sample solution"}]},{"@type":"WebSite","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#website","url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/","name":"Microsoft Copilot Blog","description":"","publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#organization","name":"Microsoft Copilot Blog","url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/05\/cropped-microsoft_logo_element.webp","contentUrl":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/05\/cropped-microsoft_logo_element.webp","width":512,"height":512,"caption":"Microsoft Copilot Blog"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/#\/schema\/person\/86838a807c26de6c2b449ff238171cc4","name":"Henry Jammes","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/08\/Henry-Jammes-PP-2022-2-96x96.jpega34837d73f2677c6be105c24b22706fb","url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/08\/Henry-Jammes-PP-2022-2-96x96.jpeg","contentUrl":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-content\/uploads\/2024\/08\/Henry-Jammes-PP-2022-2-96x96.jpeg","caption":"Henry Jammes"},"description":"Henry Jammes is a leader in the Microsoft Copilot Studio customer advisory team, part of the Power CAT team within Microsoft Engineering. With deep expertise in generative AI, conversational AI, and agentic AI, Henry and his team collaborate closely with top-tier enterprise customers as well as Microsoft\u2019s product leadership and program managers. Passionate about connecting technology to real-world customer outcomes, his team actively channels customer feedback into product enhancements, creates practical guidance, delivers impactful samples, and contributes broadly through community engagement and public speaking to accelerate customer success.","sameAs":["https:\/\/www.linkedin.com\/in\/henryjammes\/"],"url":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/author\/hejammes\/"}]}},"msxcm_display_generated_audio":false,"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/copilot\/2652","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/copilot"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/types\/copilot"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/users\/70"}],"version-history":[{"count":1,"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/copilot\/2652\/revisions"}],"predecessor-version":[{"id":5535,"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/copilot\/2652\/revisions\/5535"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/media\/1919"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/media?parent=2652"}],"wp:term":[{"taxonomy":"cs-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/cs-content-type?post=2652"},{"taxonomy":"cs-topic","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/cs-topic?post=2652"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/microsoft-copilot\/blog\/wp-json\/wp\/v2\/coauthors?post=2652"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}