Build a Schema-First AI Call QA Pipeline in Python A developer published a Python sample in the Telnyx Code Examples repository that builds a schema-first AI call QA pipeline, converting completed conversations into typed records for sentiment, compliance, and escalation. The pipeline defines three versioned JSON-schema insight templates (sentiment-v1, compliance-v1, escalation-v1), bundles them into a qa-scorecard-v1 insight group attached to an AI Assistant via insight_settings, and validates results against constrained enums and required fields. The sample also performs a read-after-write membership check, exiting if any expected template is missing from the group. Free-text call summaries are useful for people, but awkward for software. A dashboard cannot reliably group a paragraph by sentiment, count missing disclosures, or identify why callers escalated without another parsing step. This tutorial builds a schema-first AI call QA pipeline in Python. Each completed conversation becomes a set of typed records for sentiment, compliance, and escalation. Those records can move directly into a scorecard because the expected fields, enums, and required values are declared before analysis begins. The complete sample is available in the Telnyx Code Examples repository https://github.com/team-telnyx/telnyx-code-examples/tree/main/insights-pipeline . The example has four layers: JSON-schema insight templates | v Insight group: qa-scorecard-v1 | v AI Assistant insight settings | v Conversation insights - validation - QA scorecard It creates three versioned insight templates: sentiment-v1 returns positivity, frustration, overall sentiment, and key emotions. compliance-v1 returns an identity-verification result and a constrained list of compliance flags. escalation-v1 returns a controlled escalation reason and whether the agent handled it well. The templates are assigned to one insight group. That group is attached to an AI Assistant once, so future assistant conversations can be analyzed against all three contracts. Without a contract, an analysis model might describe the same issue as "caller asked for a manager," "requested escalation," or "wanted a supervisor." Those phrases may be equivalent to a reader, but they create inconsistent analytics. The escalation template constrains the result instead: ESCALATION SCHEMA = { "type": "object", "properties": { "escalation reason": { "type": "string", "enum": "caller request", "compliance violation", "poor call quality", "system error", "other", , }, "agent performed well": {"type": "boolean"}, }, "required": "escalation reason", "agent performed well" , "additionalProperties": False, } The sample sends json schema as a JSON string, which is the canonical wire format used by create templates.py : payload = { "name": "escalation-v1", "instructions": "Determine whether the caller escalated and why.", "json schema": json.dumps ESCALATION SCHEMA , } requests.post f"{API BASE}/ai/conversations/insights", headers=headers, json=payload, timeout=30, The API also accepts the schema as an object. See the structured insights documentation https://developers.telnyx.com/docs/inference/ai-insights/structured-insights for the concept and request shape. create group.py finds or creates qa-scorecard-v1 , assigns the three templates, and then reads the group back. The script exits if any expected template is missing from the returned membership. This read-after-write check is important. A successful request only proves that the API accepted it; reading the resource back proves that the configuration the assistant will use is present. for name in TEMPLATE NAMES: assign headers, group id, templates name membership = read membership headers, group id assigned names = {item.get "name" for item in membership} missing = name for name in TEMPLATE NAMES if name not in assigned names if missing: sys.exit f"Assignment verification failed - missing {missing}" Telnyx insight groups https://developers.telnyx.com/docs/inference/ai-insights/insight-groups let you apply a reusable bundle of analysis templates to an assistant instead of managing each template separately. The assistant is connected to the group through insight settings : payload = { "insight settings": { "insight group id": group id, } } requests.post f"{API BASE}/ai/assistants/{assistant id}", headers=headers, json=payload, timeout=30, wire assistant.py follows that update with a GET request and verifies that the returned insight group id matches the expected group. If ASSISTANT ID is not set, the script first creates an assistant using the platform-default model and voice. The sample supports two ways to discover completed analysis: call.conversation.insights generated webhook provides a completion signal for real-time workflows. GET /ai/conversations/{conversation id}/conversations-insights remains the polling source of truth and works even when the webhook receiver is not running. The Flask receiver verifies Telnyx's Ed25519 signature before accepting an event. TELNYX PUBLIC KEY is required, and there is no unverified fallback path. Missing or invalid signature data returns 400 . Polling also makes the lifecycle explicit: | Status | Pipeline behavior | |---|---| | pending | Count the record as pending. | | in progress | Keep waiting; analysis is not complete. | | completed | Validate the result against its declared schema. | | failed | Report the failed record and stop the live-call workflow with a nonzero exit. | That distinction prevents a slow record from being mislabeled as bad data. scorecard.py maps each returned template ID back to its template name, parses the result, and validates required fields and enum values. Only valid completed records contribute to the scorecard. The output aggregates: A completed result that violates its schema is printed with its conversation ID and excluded from the totals. You can test this behavior locally with --strict , which injects a synthetic sentiment record missing positivity score : python scorecard.py --strict This gives the pipeline two separate failure categories: records that are not ready yet and records that are complete but structurally invalid. Clone the repository and install the Python dependencies: git clone https://github.com/team-telnyx/telnyx-code-examples.git cd telnyx-code-examples/insights-pipeline python -m venv .venv source .venv/bin/activate pip install -r requirements.txt cp .env.example .env Configure .env with your Telnyx API key and placeholders for the assistant and destination numbers: TELNYX API KEY=YOUR TELNYX API KEY ASSISTANT PHONE NUMBER=