How to Augment Voice Calls with Twilio Intelligence in C# Twilio published a C# tutorial showing developers how to combine Conversation Intelligence, Conversation Orchestrator, and Conversation Memory to generate a short summary of each customer call, an analysis of the caller's sentiment, and an assessment of whether the agent followed configured guidelines, then persist that data to a SQLite database. The tutorial requires a free Twilio account, a voice-capable Twilio phone number, the Owl Air phone agent from a prior Conversation Relay tutorial, .NET 9 or newer, an OpenAI API key, the SQLite Command Line Shell, Git, and ngrok, and adds a POST webhook route that Twilio calls after customer calls end. How to Augment Voice Calls with Twilio Intelligence in C Time to read: Written by How to Augment Voice Calls with Twilio Intelligence in C introduction You know how to build a voice AI agent from scratch https://www.twilio.com/en-us/blog/developers/tutorials/product/ai-phone-agent-twilio-conversation-relay-php , one that supports speech recognition, text-to-speech, turn detection, and real-time audio streaming — all at low latency. But, what if you could also analyze customer conversations in real time and collect useful data insights for future calls? With Twilio Conversation Intelligence https://www.twilio.com/docs/conversations/intelligence and Conversation Orchestrator https://www.twilio.com/docs/conversations/orchestrator , backed by Conversation Memory https://www.twilio.com/docs/conversations/memory , you can Specifically, in this tutorial you're going to learn how to use these three technologies together to retrieve a short summary of each customer call, and an analysis of the caller's sentiment. What's more, you'll also see whether your agent followed the guidelines you set for it. All of this information will then be persisted to a SQLite database, so that you can make use of it later. Architecture architecture As this tutorial adds three new technologies to the previous application, here's a quick overview of how the new functionality works. You will add a new route which receives a POST webhook request from Twilio after customer calls end. The request body will be a JSON string that contains, among other things, a short summary of the call, an assessment of the caller's sentiment, and how the agent adhered to a series of criteria. That information will be extracted from the request and then persisted to the application's SQLite database. You're not going to do more with the received information. But, there are links at the end of the tutorial showing how you could continue building on the changes made in this tutorial, should you want to. Prerequisites prerequisites To follow along with the tutorial, you will need the following: - A free Twilio account — Sign up for an account here https://login.twilio.com/u/signup - A voice-capable Twilio phone number - The Owl Air phone agent https://www.twilio.com/en-us/blog/developers/tutorials/product/ai-phone-agent-twilio-conversation-relay-csharp from the previous tutorial, or any Conversation Relay-based agent built with .NET - .NET 9 https://dotnet.microsoft.com/en-us/download or newer - An OpenAI API key https://platform.openai.com/api-keys - The SQLite Command Line Shell https://sqlite.org/cli , or your preferred database admin tool which has SQLite support - Git https://git-scm.com/ - ngrok https://ngrok.com/ or a similar tool, to expose your local server to Twilio Build the app build-the-app Step 1: Set up Conversation Orchestrator and Conversation Memory toc-heading-76a487b7-c91e-47b6-9625-5a8aae0a31a6 Before you can set up Conversation Intelligence, which does most of the work, you need to create a Memory Store and Conversation Configuration. To do that, sign in to the Twilio Console https://1console.twilio.com/ , and go to Products & Services Conversation Orchestrator Conversation configurations https://1console.twilio.com/go?to=/account/ account /us1/conversation-orchestrator/configurations . There, click Create a Conversation configuration . On the Name Configuration step, enter a name and description, then click Next . On the Messaging and chat traffic step, click Next . On the Voice traffic step, scroll down and enable the Set up automatic capture checkbox. From the Voice phone numbers list, select your Twilio phone number, then click Next . Now, on the Configure lifecycle step click Next . After that, on the Enable Conversation Memory step, create a memory store, by clicking Create new memory store , entering a name In the Memory store name field, and clicking Save . From the Memory store list, select the memory store you just created, leave Turn on observations and summaries enabled, and click Next . Finally, On the Summary step, review your settings and click Create Conversation configuration . Copy the conversation configuration ID for use later. Step 2: Set up Conversation Intelligence toc-heading-fd34c8f5-cd81-44f5-934f-d11dce039c62 Before you can complete this step, you need to make the application publicly accessible on the internet as you'll need the ngrok URL later in this section. Run the command, below, to create a connection to the app on port 5000. Now it is time to configure your Conversation Intelligence. To do that, go to Products & Services Conversation Intelligence Intelligence configurations. https://1console.twilio.com/go?to=/account/ account /us1/conversation-intelligence/configurations There, click Create Intelligence configuration . Add a name, description, and, in the Attach Conversation configurations section, select the name of the Conversation configuration that you created in the previous step, and click Submit . With that done, in Intelligence configurations, click Create rule next to the Intelligence configuration which you just created. Then, in the Add language operators section, enable Sentiment , Summary , and Script-Adherence, and click Next . Now, in the Script-Adherence section, at the bottom of the Set Parameters step, add the following text into the script field and click Next . Now, on the Trigger and action step, choose At conversation end in the Trigger section. Then, in the Action section, paste your ngrok URL plus "/intelligence-results" in the Webhook action field. Click Next . In the Add context step, scroll down to the Conversation Memory section and enable Enable Conversation Memory for this rule and click Next . In the Summary step, click Create rule . Step 3: Update the existing project structure toc-heading-617043fb-4c53-4481-976e-ad887336f55e Add some new directories to your project by running the following command: This data/database directory will store the application's SQLite database and a SQL file defining the database's schema. The other folders, which may or may not already exist in your base project, will help structure the .NET application. Step 4: Set up the application's database toc-heading-da19ded6-b611-4d58-9226-b1759f05d00e Create a new file named dump.sql in the data/database directory and paste the following SQL into that file. The instructions: - Enable SQLite's WAL Write-Ahead Logging mode https://sqlite.org/wal which, among other benefits, significantly improves performance - Enable foreign key support - Define three tables: - intelligence results: stores the core information about the conversation - operators: stores the information extracted by Conversation Intelligence, such as the call summary and intent - operator script adherence categories: stores the script adherence information, linking it to the relevant record in operators It's not the most sophisticated schema, but it can store the information in a maintainable way. Now, use SQLite's Command-Line Shell or your preferred database management tool to provision the database with the following command. Step 5: Create the Conversation Intelligence route toc-heading-22518d45-7a3d-41d6-a860-02839ec9dd9a Add the required packages add-the-required-packages The application needs a few NuGet packages to run: Dapper , a lightweight object mapper that simplifies working with the SQLite database, and Microsoft.Data.Sqlite , the official SQLite driver for .NET. You will also need to be sure you are running the latest version of the Twilio package. Install these dependencies by running the following commands from the project root: Create the operator result records create-the-operator-result-records Now, create the small immutable data types that will hold the pieces of information extracted from the webhook. Each of the three operator types — Summary, Sentiment, and ScriptAdherence — implements a common ITypeOperator interface, so that the database repository can treat them uniformly. ScriptAdherence additionally holds a list of Category records, one per script-adherence category that Conversation Intelligence evaluates. In a final build these would all be different files for extensibility. But for the purpose of a tutorial, because these are small, related, immutable data types, you can keep all five in a single file. In the Models folder of your existing project, create a file named OperatorResults.cs , and paste the code below into it. Create the database service create-the-database-service It's time to create the database service that the webhook controller will use to persist the retrieved webhook data into the application's SQLite database. In the Services folder, create a file named DatabaseService.cs , and paste the code below into the file. This code opens a connection, starts a transaction, and inserts into intelligence results . For each operator, the switch expression maps the runtime type Summary, Sentiment , or ScriptAdherence to the short string stored in the database "summary", "sentiment", or "script-adherence" . Dapper's ExecuteScalar