{"id":109184,"date":"2017-12-28T10:54:39","date_gmt":"2017-12-28T18:54:39","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/power-platform\/blog\/power-apps\/cognitive-services-with-powerapps-using-custom-connectors\/"},"modified":"2025-06-11T08:10:26","modified_gmt":"2025-06-11T15:10:26","slug":"cognitive-services-with-powerapps-using-custom-connectors","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/power-platform\/blog\/power-apps\/cognitive-services-with-powerapps-using-custom-connectors\/","title":{"rendered":"Cognitive Services with PowerApps using Custom Connectors"},"content":{"rendered":"

Microsoft Cognitive Services<\/a> are a set of APIs, SDKs and services available to developers to make their applications more intelligent, engaging and discoverable. This blog is an attempt to share an approach for PowerApps makers to use these Cognitive Services using custom connectors<\/a>.<\/p>\n

There is an out-of-the-box Computer Vision API connector<\/a>, however the API uses dynamic types, which are not supported in PowerApps yet. There is also an out-of-the-box Face API connector<\/a>, however it isn\u2019t much useful in enterprise scenarios yet, as it\u2019s an API Key based connector and when a PowerApps app created using this connector is shared with other users, each user has to bring in their own API keys<\/a> for using the app.<\/p>\n

\u00a0<\/h3>\n

About this blog<\/h3>\n

In the approach described in this blog, the cognitive services APIs are called from an Azure API App<\/a>, using the client libraries which are just a C# client wrapper for the APIs. We shall be using the Microsoft.ProjectOxford.Face<\/a> and Microsoft.ProjectOxford.Vision<\/a> Nuget packages in our API app. I have shared the complete source code for the API App<\/a> for download as well as the PowerApps app file<\/a> to use & test the APIs as well. I have also combined the code for uploading an image to Azure Blob storage<\/a> in this new API app \u2013 ImageUtilities<\/strong>, so that one can use all these functions together in an app if needed.<\/p>\n

First we shall create the necessary Azure Services (App Service, Blob Storage, Computer Vision API & Face API), then publish the API App to Azure.<\/p>\n

Then, we shall register a new custom connector in PowerApps using the swagger definition from the API App.\u00a0<\/p>\n

Finally, we shall create the PowerApps app using this new custom connector.<\/p>\n

In PowerApps, when a maker shares an app created using a custom connector, the connector is automatically shared and hence other users can use the app (without needing to type in an API key). In the blog below, we shall also protect our API app endpoint with AAD and also register an app in Azure AD for the custom connector, so that other users can use their own AAD credentials to authenticate and get access to the APIs.<\/p>\n

\u00a0<\/p>\n

Let\u2019s get started.<\/h3>\n

\u00a0<\/p>\n

Here are the steps :<\/p>\n

    \n
  1. Create the API App project<\/a><\/li>\n
  2. Publish the API to Azure<\/a><\/li>\n
  3. Protect with the API with Azure AD authentication<\/a> (optional but highly recommended)<\/li>\n
  4. Register another app in Azure AD for the custom connector<\/a> (optional but highly recommended)<\/li>\n
  5. Register a new custom connector in PowerApps<\/a><\/li>\n
  6. Create a PowerApps App to use the connector<\/a><\/li>\n
  7. Save, Publish and Test the App<\/a><\/li>\n<\/ol>\n

    \u00a0<\/p>\n

    Create the API App project<\/a><\/h3>\n

    I have chosen to use an API App for the custom connector versus Azure Functions primarily as API apps can generate the Swagger document<\/a> for use with PowerApps using the Swashbuckle<\/a> Nuget package. This auto generates the definitions for all the reference classes like AnalysisResult<\/a> and others used by the APIs in the swagger document. Since I have used the Swashbuckle feature to include XML Comments<\/a> from my assembly to generate Summary and Description tags in swagger, it becomes quite handy for both the API registration and also for providing intelli-sense when authoring the functions from the PowerApps app.<\/p>\n

    \u00a0<\/p>\n

    You will need the following Azure Services setup before you can complete this step:<\/p>\n