This post delves into creating a Power Automate flow that connects PVA to Azure OpenAI's ChatGPT and to an Azure Cognitive Search Instance (which is connected to your data source, such as your Azure Storage Account).
Prerequisites:
1. Azure OpenAI deployment
2. Power Virtual Agent app deployed
3. Azure Cognitive Search instance with an index (formed by connecting to your desired data source and indexing).
Steps
1. Go back to where you made your MS Teams chatbot, or go to https://web.powerva.microsoft.com/ and click into your chatbot.
2. Now we will configure the flow of the PVA. Go Topics on the left, then to the Systems tab and click 'Fallback' to view what your PVA does when the response cannot be determined with a generic response (which will be almost every case where a user is attempting to write a prompt on their enterprise documents). You should initially see something like the below.
3. Select the existing steps after the initial Trigger and delete them so you are left with just the below.
4. Click the (+) sign, select 'Call an Action' and then 'Create a Flow'. This opens Power Automate:
5. Click 'Add an input' and add a Text input to the trigger. Name it 'request'. This represents the initial user input request that users enter via the Teams chat.
6. Select (+) to add a new action after the input request. Search for 'Initialize variable' to add the Initialize Variable action. Paste the JSON code that you copied from that OAI Chat session (see Creating an MS Teams Enterprise Chatbot). Assuming enterprise data is going to be used, the JSON will also need to be modified, as shown below.
Note: the dataSources array contains details that can be retrieved from your Azure Cognitive Search instance, as follows. Go to your Azure Portal and into your Azure Cognitive Search instance.
endpoint: from the contents sidebar, go to Overview and copy the URL shown toward the right side.
key: Under Settings, go to Keys and select API keys. Copy the Primary admin key.
indexName: in the sidebar, under Search management, go to Indexes and copy the name of the index you want your chatbot to refer to. Note that an index can be created by one or more indexers, and therefore one or more different data sources.
{
"dataSources": [
{
"type": "AzureCognitiveSearch",
"parameters": {
"endpoint": "INSERT YOUR AZURE COGNITIVE SEARCH URL",
"key": "INSERT YOUR AZURE COGNITIVE SEARCH ADMIN KEY",
"indexName": "INSERT THE NAME OF YOUR INDEX"
}
}
],
"messages": [
{
"role": "system",
"content": "You are an AI assistant that helps people find information and interact with the data stored within the data source.
###
Answer as concisely as possible.
###
You should not add information from outside of the documents you are linked to.
###
If you cannot answer a question with only the information from the linked data, reply with (without quotes): 'I do not have access to that information. Please try a different question or check your linked documents.'
###"
},
{
"role": "user",
"content": @{triggerBody()['text']}
}
],
"temperature": 0,
"top_p": 1,
"frequency_penalty": 0,
"presence_penalty": 0,
"max_tokens": 800,
"stop": null
}
Now your first 2 steps should look something like this (change the system context setting message to what works best for you):
Note: in the role: 'user' section, add the dynamic item (that you named 'request') from the
previous step. In the above code snippet, this is represented by @{triggerBody()['text']}. In the images above/below, we see the dynamic icon for it. Once added, it should look something like this:
7. Create a subsequent HTTP action (search HTTP). Enter the following items as shown in the snippet below.
Method: POST
URI: Use the URL from Azure OAI that you copied previously, but importantly adding 'extensions/' after the name of your bot, and also changing the version (near the end of the string) to '=2023-06-01' – as shown in the image below.
Headers - key-value pairs:
'content-type' & 'application/json'
'api-key' & (the API key you copied previously) – consider using a dynamic value from Azure Key vault, for security purposes. Not covered in this post.
Body: 'OAI_Request' dynamic parameter.
8. Add a subsequent Initialize variable action (as done previously), as follows.
9. Add a Condition action where the value is the status code from the HTTP body, and we are testing whether it is equal to 200 (meaning a connection is made), as shown below.
10. If the status check condition is 'no', meaning connection cannot be made, we can add an action called 'Set Variable' to produce a response in this case – below is an example.
11. If the condition is yes, then a connection is made and we can add a Parse JSON action to parse the JSON of the HTTP response body, per below.
The schema to copy in is below.
Note: if the below schema is causing errors, then you can also test the Power Automate flow, observe the output from the HTTP Body, and generate the schema from that.
{
"type": "object",
"properties": {
"id": {
"type": "string"
},
"model": {
"type": "string"
},
"created": {
"type": "integer"
},
"object": {
"type": "string"
},
"choices": {
"type": "array",
"items": {
"type": "object",
"properties": {
"index": {
"type": "integer"
},
"messages": {
"type": "array",
"items": {
"type": "object",
"properties": {
"index": {
"type": "integer"
},
"role": {
"type": "string"
},
"content": {
"type": "string"
},
"end_turn": {
"type": "boolean"
}
},
"required": [
"index",
"role",
"content",
"end_turn"
]
}
}
},
"required": [
"index",
"messages"
]
}
}
}
}12. Add a subsequent 'Set Variable' action with the response from ChatGPT. The name will be OAI_response and the value will be an expression, which will pull the text from the previously parsed JSON. Instead of dynamic content, click 'Expression' and paste the following in.
body('Parse_JSON')['choices'][0]['messages'][1]['content']
13. Click the [+ New Step] button to add a new Power Virtual Agent action: 'Return value(s) to Power Virtual Agents'. Name the title 'response' and select the dynamic content 'OAI_response' for the body.
14. The summarised flow should look as follows.
15. Go back to the Power Virtual Agents window, go into Topics and into fallback, so that we can link the Power Automate Input and Output to PVA, per below.
Ensure the items after 'On Unknown Intent' are cleared, as mentioned previously.
Add a 'Call an Action' and select the Power Automate flow you just made:
Set the request(String) = Activity.Text.
Set the output to be response(string) = response(string).
Note: there can be a delay on the saved settings in the Power Automate flow showing here. If the output 'response – string' does not show, wait 1-2 hours and try again.
16. Add the last action, which sends the response to MS Teams in an Adaptive Card format:
Click (+) > 'send a message' > 'add' > 'adaptive card':
17. In the editor, click 'edit formula' (NOT 'edit json') and paste in the below snippet, which determines the aesthetic of how the response shows in the Teams chat. Click save.
{
'type': "AdaptiveCard",
'$schema': "http://adaptivecards.io/schemas/adaptive-card.json",
'version': "1.3",
'body': [
{
'type': "TextBlock",
'text': "Answer generated by Azure OpenAI using ChatGPT model ",
'wrap': true,
'size': "Medium",
'weight': "Bolder"
},
{
'type': "Container",
'items': [
{
'type': "TextBlock",
'text': Text(Topic.response),
'wrap': true
}
],
'style': "emphasis"
},
{
'type': "TextBlock",
'text': "NOTE: This service is in preview version and model does not have context of previous messages.",
'wrap': true,
'color': "Accent",
'size': "Small"
}
]
}We can now publish and deploy to users.
18. Go to 'Publish' in the left menu and click Publish, which updates the PVA with any changes made from this page.
Note: Saved changes made to the Power Automate flow itself (outside of PVA) will have changes reflected in the Teams chatbot automatically, without having to re-publish via PVA.
19. Go to Settings > Channels > Microsoft Teams > Availability Options, per below.
20. Click 'Show to my teammates' or 'show to everyone in org' depending on your use case.
Note: 'Show to everyone in org' requires admin approval in the MS Teams administrator settings for organisational apps.
Congratulations. You should now have a chatbot in MS Teams, powered by ChatGPT and Azure, that can interact with your enterprise documents.
The goal has been to mimic the Chat Playground in oai.azure.com in how it can interact with your data, and with the parameters/context you set. Setting this up as a Teams chatbot gives people in your organisation the ability to utilise ChatGPT with your enterprise documents.
You may have noticed that in the Chat Playground, ChatGPT produces references at the end of each response, referring to your enterprise documents, and the location within those documents (e.g., a section of a PDF). Currently, your Teams Chatbot does not do this. The data is still being received from ChatGPT, but more extensive parsing is required to display the references in MS Teams. Depending on your use case, the effort to implement this may or may not be worth it. The article in the link below shows you how to do this.
Looks good David!