LlamaIndex Observability & Monitoring with OpenTelemetry

SigNoz has released a new guide for enabling observability and monitoring in Python-based LlamaIndex applications using OpenTelemetry, allowing developers to trace AI-specific operations such as document ingestion, retrieval, querying, and text generation. The integration, which leverages the OpenInference standard, provides end-to-end visibility into RAG workflows by capturing detailed spans on request durations, model outputs, and retrieval scores, helping teams identify latency bottlenecks and improve production reliability.

Overview This guide walks you through enabling observability and monitoring for your Python-based LlamaIndex https://www.llamaindex.ai/ application and streaming telemetry data to SigNoz Cloud using OpenTelemetry. By the end of this setup, you'll be able to monitor AI-specific operations such as document ingestion, document retrieval, user querying, text generation, and user feedback within LlamaIndex, with detailed spans capturing request durations, node and query inputs, model outputs, retrieval scores, metadata, and intermediate steps throughout the pipeline. Instrumenting your RAG workflows with telemetry enables full observability across the retrieval and generation pipeline. This is especially valuable when building production-grade developer-facing tools, where insight into model behavior, latency bottlenecks, and retrieval accuracy is essential. With SigNoz, you can trace each user question end-to-end, from prompt to response, and continuously improve performance and reliability. To get started, check out our example LlamaIndex RAG Q&A bot, complete with OpenTelemetry-based monitoring via OpenInference . View the full repository here https://github.com/SigNoz/llamaindex-rag-opentelemetry-demo . Prerequisites - A Python application using Python 3.8+ - LlamaIndex integrated into your app, with document ingestion and query interfaces set up - Basic understanding of RAG Retrieval-Augmented Generation workflows - SigNoz setup choose one : SigNoz Cloud account https://signoz.io/teams/ with an active ingestion key- Self-hosted SigNoz instance pip installed for managing Python packages- Internet access to send telemetry data to SigNoz Cloud Optional but recommended A Python virtual environment to isolate dependencies Instrument your LlamaIndex application To capture detailed telemetry from LlamaIndex without modifying your core application logic, we use OpenInference https://arize.com/docs/ax/learn/tracing-concepts/what-is-openinference , a community-driven standard that provides pre-built instrumentation for popular AI frameworks like LlamaIndex, built on top of OpenTelemetry. This allows you to trace your LlamaIndex application with minimal configuration. Check out detailed instructions on how to set up OpenInference instrumentation in your LlamaIndex application over here https://pypi.org/project/openinference-instrumentation-llama-index/ . No-code auto-instrumentation is recommended for quick setup with minimal code changes. It's ideal when you want to get observability up and running without modifying your application code and are leveraging standard instrumentor libraries. Step 1: Install the necessary packages in your Python environment. pip install \ opentelemetry-distro \ opentelemetry-exporter-otlp \ opentelemetry-instrumentation-httpx \ opentelemetry-instrumentation-system-metrics \ llama-index \ openinference-instrumentation-llama-index Step 2: Add Automatic Instrumentation opentelemetry-bootstrap --action=install Step 3: Configure logging level To ensure logs are properly captured and exported, configure the root logger to emit logs at the INFO level or higher: python import logging logging.getLogger .setLevel logging.INFO This sets the minimum log level for the root logger to INFO, which ensures that logger.info calls and higher severity logs WARNING, ERROR, CRITICAL are captured by the OpenTelemetry logging auto-instrumentation and sent to SigNoz. Step 4: Run an example python from llama index.llms.openai import OpenAI llm = OpenAI model="gpt-4o" response = llm.complete "Hello, world " print response 📌 Note: Ensure that the OPENAI API KEY environment variable is properly defined with your API key before running the code. Step 5: Run your application with auto-instrumentation OTEL RESOURCE ATTRIBUTES="service.name=<service name " \ OTEL EXPORTER OTLP ENDPOINT="https://ingest.<region .signoz.cloud:443" \ OTEL EXPORTER OTLP HEADERS="signoz-ingestion-key=<your-ingestion-key " \ OTEL EXPORTER OTLP PROTOCOL=grpc \ OTEL TRACES EXPORTER=otlp \ OTEL METRICS EXPORTER=otlp \ OTEL LOGS EXPORTER=otlp \ OTEL PYTHON LOG CORRELATION=true \ OTEL PYTHON LOGGING AUTO INSTRUMENTATION ENABLED=true \ opentelemetry-instrument <your run command is the name of your service <service name <region : Your SigNoz Cloud region https://signoz.io/docs/ingestion/signoz-cloud/overview/ endpoint <your-ingestion-key : Your SigNoz ingestion key https://signoz.io/docs/ingestion/signoz-cloud/keys/ - Replace <your run command with the actual command you would use to run your application. For example: python main.py Using self-hosted SigNoz? Most steps are identical. To adapt this guide, update the endpoint and remove the ingestion key header as shown in Cloud → Self-Hosted https://signoz.io/docs/ingestion/cloud-vs-self-hosted/ cloud-to-self-hosted . Code-based instrumentation gives you fine-grained control over your telemetry configuration. Use this approach when you need to customize resource attributes, sampling strategies, or integrate with existing observability infrastructure. Step 1: Install OpenInference and OpenTelemetry related packages pip install openinference-instrumentation-llama-index \ opentelemetry-exporter-otlp \ opentelemetry-sdk \ llama-index Step 2: Import the necessary modules in your Python application python from opentelemetry.exporter.otlp.proto.http.trace exporter import OTLPSpanExporter from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.sdk.resources import Resource from openinference.instrumentation.llama index import LlamaIndexInstrumentor Step 3: Set up the OpenTelemetry Tracer Provider to send traces directly to SigNoz Cloud resource = Resource.create {"service.name": "<service name "} provider = TracerProvider resource=resource span exporter = OTLPSpanExporter endpoint="https://ingest.<region .signoz.cloud:443/v1/traces", headers={"signoz-ingestion-key": "<your-ingestion-key "}, provider.add span processor BatchSpanProcessor span exporter is the name of your service <service name : Your SigNoz Cloud <region region : Your SigNoz <your-ingestion-key ingestion key Using self-hosted SigNoz? Most steps are identical. To adapt this guide, update the endpoint and remove the ingestion key header as shown in Cloud → Self-Hosted https://signoz.io/docs/ingestion/cloud-vs-self-hosted/ cloud-to-self-hosted . Step 4: Instrument LlamaIndex using OpenInference and the configured Tracer Provider Use the LlamaIndexInstrumentor from OpenInference to automatically trace LlamaIndex operations with your OpenTelemetry setup: LlamaIndexInstrumentor .instrument tracer provider=provider 📌 Important: Place this code at the start of your application logic — before any LlamaIndex functions are called or used — to ensure telemetry is correctly captured. Step 5: Run an example python from llama index.llms.openai import OpenAI llm = OpenAI model="gpt-4o" response = llm.complete "Hello, world " print response 📌 Note: Ensure that the OPENAI API KEY environment variable is properly defined with your API key before running the code. Your LlamaIndex commands should now automatically emit traces, spans, and attributes. Finally, you should be able to view this data in Signoz Cloud under the traces tab: When you click on a trace ID in SigNoz, you'll see a detailed view of the trace, including all associated spans, along with their events and attributes. Additional Resources - Read Observing LlamaIndex Apps with OpenTelemetry + SigNoz https://signoz.io/blog/opentelemetry-llamaindex/ - Set up alerts https://signoz.io/docs/alerts/ for high latency or error rates - Learn more about querying traces https://signoz.io/docs/userguide/traces/