{"slug": "instrumenting-an-ai-powered-github-analyzer-with-opentelemetry-and-signoz", "title": "Instrumenting an AI-Powered GitHub Analyzer with OpenTelemetry and SigNoz", "summary": "A developer built GitIntel, an AI-powered GitHub repository analyzer using Gemini 2.5 Flash, and instrumented it with OpenTelemetry and SigNoz to gain end-to-end observability. The tool analyzes code across eight engineering dimensions, and the instrumentation revealed insights into latency, token usage, and execution flow that application logs alone could not provide.", "body_md": "This article is my submission for the [Agents of SigNoz Hackathon](https://www.wemakedevs.org/hackathons/signoz): Blog Track, where participants instrument real applications with OpenTelemetry and SigNoz to make AI systems observable.\n\nGitIntel is an AI-powered GitHub repository analyzer that uses `Gemini 2.5 Flash`\n\nto evaluate repositories and generate a comprehensive developer assessment. Given a GitHub username and selected repositories (atmost 5), it analyzes code across eight engineering dimensions: architecture, security, testing, documentation, complexity, and engineering practices to build both repository-level insights and an overall developer profile.\n\nThe idea was straightforward: Combine GitHub data with an LLM to understand not just what the code does, but what it says about the developer behind it.\n\nAs GitIntel evolved, every analysis became a multi-step workflow involving GitHub API requests, asynchronous file fetching, prompt construction, multiple Gemini API calls, response aggregation, and report generation.\n\nFrom the user's perspective, it was just one **Analyze** button.\n\nFrom the application's perspective, it was a distributed pipeline.\n\nThat mismatch quickly became a problem.\n\nSome repository analyses consumed nearly 40,000 Gemini tokens, while others finished with fewer than 4,000. End-to-end latency ranged from 8 secs to 52 secs, even for repositories that appeared similar.\n\nMy application logs already gave me useful information. I could see total token usage, overall analysis duration, GitHub rate-limit status, and when requests succeeded or failed.\n\nBut they couldn't answer many other questions like:\n\nI had the outcomes.\n\nI didn't have the execution story.\n\nTerminal logs told me what happened. They couldn't show me how the entire pipeline behaved or where time was actually being spent. Every optimization still involved a degree of guesswork.\n\nThat's when I decided to instrument GitIntel with **OpenTelemetry** and use **SigNoz** as the observability backend.\n\nTo make GitIntel observable, I instrumented the application with OpenTelemetry and connected it to a self-hosted SigNoz instance.\n\nInstead of treating the application as a black box, I could now trace every request end-to-end from GitHub API calls to Gemini interactions, while correlating traces, metrics, and logs in a single place. The result was complete visibility into latency, token usage, failures, retries, and the overall execution flow.\n\nIn this article, I'll share the setup, the instrumentation process, the challenges I encountered, the insights observability uncovered, and how you can reproduce the same environment on Ubuntu.\n\n**Live Demo:** [GitIntel](https://gitintel-2kh2.onrender.com)\n\n**Source Code:**\n\nAI-powered GitHub profile analyzer. Enter a username, select up to 5 repos, and get a deep code assessment powered by Gemini 2.5 Flash — with full observability via OpenTelemetry and SigNoz.\n\n**Live demo:** [gitintel-2kh2.onrender.com](https://gitintel-2kh2.onrender.com)\n\nBuilt for the\n\n[Agents of SigNoz Hackathon]— blog track.\n\n`gemini.tokens.total`\n\n, `gemini.tokens.prompt`\n\n, `gemini.tokens.completion`\n\n…| Component | Stack |\n|---|---|\nBackend |\nPython 3.12, FastAPI, Uvicorn |\nAI |\nGemini 2.5 Flash (`google-genai` ) |\nData Fetching |\nGitHub REST API, `aiohttp` (10 concurrent workers) |\nObservability |\nOpenTelemetry SDK + Self-hosted SigNoz |\nFrontend |\nHTML, CSS, Vanilla JavaScript |\nPlatform |\nUbuntu 24.04 LTS, Docker & Docker Compose |\n\nBefore getting started, make sure you have the following ready:\n\n`repo`\n\n`read:user`\n\n`gemini-2.5-flash`\n\nBefore continuing, verify that your environment is set up correctly:\n\n```\npython3 --version    # should be 3.12.x\ndocker --version     # 20+ is fine\ndocker compose version\n```\n\nIf `docker compose version`\n\nfails, you're most likely using the legacy **docker-compose (v1)**. Install Docker Compose v2 with:\n\n```\nsudo apt-get install docker-compose-plugin\n```\n\n1) Start by forking the repository on GitHub, then clone your fork locally:\n\n```\n   git clone https://github.com/YOUR_USERNAME/Github-Analyzer\n   cd Github-Analyzer\n```\n\n2) Create a virtual environment and install the project dependencies:\n\n```\n   python3 -m venv venv\n   source venv/bin/activate\n   pip install -r requirements.txt\n```\n\n3) Next, create your environment file:\n\n```\n   cp .env.example .env\n   nano .env\n```\n\n4) Update it with your GitHub token, Gemini API key, and OpenTelemetry configuration:\n\n```\n   GITHUB_TOKEN=ghp_yourtokenhere\n   GEMINI_API_KEY=AIzaSy_yourkeyhere\n   OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\n   OTEL_SERVICE_NAME=gitintel\n```\n\n`OTEL_EXPORTER_OTLP_ENDPOINT`\n\npoints to the SigNoz OpenTelemetry Collector, which we'll set up in Part 2. Don't worry if it isn't running yet—the application works perfectly fine without it.\n\n5) Now start the application:\n\n```\n   uvicorn main:app --reload\n```\n\n6) Open ** http://localhost:8000**, enter any GitHub username, select a few repositories, and click\n\nYou'll get a complete developer assessment: repository scores, developer profile, strengths, weaknesses, and the generated report.\n\nAt this point, the application is fully functional.\n\nBut there's one problem:-\n\nYou still have no idea what happened behind the scenes. The terminal logs show you total token counts and overall duration, but not which batch cost what, not per-step timing, not which API call took 30 of those 40 secs, and not why one repo took 5x longer than another.\n\nThe results are there. The why isn't.\n\nThe application gives you the results, but none of the answers.\n\nThat's exactly what **Part 2** solves.\n\nOne of the reasons I chose SigNoz is that it's Open Source, OpenTelemetry-native, and can be self-hosted in just a few mins. That meant I could keep my entire observability stack running locally while developing GitIntel, without relying on any third-party SaaS.\n\nThe repo already includes a complete Docker Compose configuration under `pours/deployment/compose.yaml`\n\n. Spinning it up launches everything GitIntel needs to collect, process, store, and visualize telemetry:\n\n`4317 (gRPC)`\n\nand `4318 (HTTP)`\n\n, receiving telemetry exported from GitIntel.Start the entire stack with:\n\n```\ncd pours/deployment\ndocker compose up -d\n\n# Verify everything is healthy\ndocker compose ps\n```\n\nThe first startup usually takes around 60 secs, as ClickHouse needs a little time to initialize.\n\nOnce everything is ready, open ** http://localhost:8080**.\n\nAt this point, the SigNoz dashboard will load, but it will be empty. That's expected because GitIntel hasn't exported any telemetry yet.\n\nNow restart GitIntel so it can connect to the OpenTelemetry Collector:\n\n```\ncd ../..\nsource venv/bin/activate\nuvicorn main:app --reload\n```\n\nWith both GitIntel and SigNoz running, analyze any GitHub repository.\n\nAs soon as the analysis begins, GitIntel starts exporting telemetry through the OpenTelemetry Collector. Within a few seconds, a new service named `gitintel`\n\nautomatically appears under **Services** in SigNoz.\n\nNo manual registration, configuration, or dashboard creation is required, the service is discovered automatically from the OpenTelemetry `service.name`\n\nresource attribute configured in the application.\n\nOpen **Services → gitintel**, and you'll immediately see live application health, including:\n\nWith SigNoz up & running, the next step is teaching GitIntel what telemetry to collect and where to send it.\n\nInstead of scattering observability code throughout the project, I centralized everything in a single file: ** telemetry.py**. One setup function initializes traces & metrics then exports them to the OpenTelemetry Collector using OTLP/gRPC on port 4317.\n\n```\n# telemetry.py\n\nOTLP_ENDPOINT = os.getenv(\"OTEL_EXPORTER_OTLP_ENDPOINT\", \"http://localhost:4317\")\nSERVICE_NAME  = os.getenv(\"OTEL_SERVICE_NAME\", \"gitintel\")\nresource = Resource.create({\"service.name\": SERVICE_NAME})\n\ndef setup_telemetry(app):\n    otlp_available = OTLP_ENDPOINT != \"http://localhost:4317\" or os.getenv(\"OTEL_ENABLED\")\n\n    tracer_provider = TracerProvider(resource=resource)\n    if otlp_available:\n        tracer_provider.add_span_processor(\n            BatchSpanProcessor(\n                OTLPSpanExporter(endpoint=OTLP_ENDPOINT, insecure=True)\n            )\n        )\n    trace.set_tracer_provider(tracer_provider)\n```\n\nA few lines here do most of the heavy lifting.\n\nThe `Resource`\n\ndefines the identity of the application by assigning it the service name `gitintel`\n\n. Every trace, metric, & log generated by the application carries this metadata, allowing SigNoz to group everything under a single service automatically.\n\nThe `BatchSpanProcessor`\n\nbatches spans before exporting them to the OpenTelemetry Collector. Instead of sending every span individually, telemetry is exported efficiently in batches, reducing overhead during repository analysis.\n\nPerhaps my favorite part is how little manual instrumentation was actually required.\n\n```\nFastAPIInstrumentor.instrument_app(app)\nRequestsInstrumentor().instrument()\n```\n\nThese two lines unlock a surprising amount of observability.\n\nOne small detail ended up being far more important than I expected:\n\n```\notlp_available = OTLP_ENDPOINT != \"http://localhost:4317\" or os.getenv(\"OTEL_ENABLED\")\n```\n\nDuring local development, GitIntel exports telemetry to the self-hosted SigNoz instance as expected.\n\nWhen deployed to Render, however, there's no OpenTelemetry Collector running. Without this guard, the application would repeatedly try to export telemetry and flood the logs with connection refused errors every few seconds.\n\nI only discovered that after deploying the project, so this check became a simple but an important one.\n\nOnce GitIntel starts with SigNoz running, open [http://localhost:8080](http://localhost:8080) → Services.\n\nBecause the service name is already defined in the OpenTelemetry `Resource`\n\n, SigNoz automatically discovers ** gitintel** and begins collecting telemetry immediately. Without writing a single dashboard query, you can already monitor:\n\nAt this point, every request through GitIntel is visible, timed, and traceable.\n\nAuto-instrumentation gives you excellent visibility into HTTP requests, but GitIntel isn't just another web application, it's a LLM-powered analysis pipeline.\n\nKnowing that an endpoint took 12 secs is useful, but it doesn't answer the questions I actually cared about.\n\nTo answer those questions, I created eight custom OpenTelemetry instruments in `telemetry.py`\n\n:\n\n```\n# telemetry.py\n\ngithub_ratelimit_gauge           = meter.create_gauge(\"github.ratelimit.remaining\")\ngemini_tokens_counter            = meter.create_counter(\"gemini.tokens.total\",       unit=\"tokens\")\ngemini_prompt_tokens_counter     = meter.create_counter(\"gemini.tokens.prompt\",      unit=\"tokens\")\ngemini_completion_tokens_counter = meter.create_counter(\"gemini.tokens.completion\",  unit=\"tokens\")\nfiles_processed_counter          = meter.create_counter(\"github.files.processed\")\nloc_processed_counter            = meter.create_counter(\"github.loc.processed\",       unit=\"lines\")\nassessment_duration_histogram    = meter.create_histogram(\"assessment.duration\",      unit=\"s\")\napi_error_counter                = meter.create_counter(\"api.errors\")\n```\n\nEach instrument captures a different aspect of GitIntel's execution, from LLM token usage and GitHub rate limits to assessment latency and processing volume. Together, they provide a much more complete picture than HTTP metrics alone.\n\nOne implementation detail turned out to be particularly valuable.\n\nEvery token counter includes the repository name as an attribute.\n\n``` python\n# gemini_client.py — _track_usage()\n\ndef _track_usage(usage, repo: str):\n    pt = getattr(usage, \"prompt_token_count\", 0) or 0\n    ct = getattr(usage, \"candidates_token_count\", 0) or 0\n\n    gemini_prompt_tokens_counter.add(pt, {\"repo\": repo})\n    gemini_completion_tokens_counter.add(ct, {\"repo\": repo})\n    gemini_tokens_counter.add(pt + ct, {\"repo\": repo})\n```\n\nThat small `{\"repo\": repo}`\n\nattribute completely changes how useful the data becomes.\n\nWithout it, I'd only know GitIntel consumed 50,000 tokens.\n\nWith it, I can immediately identify which repository consumed 38,000 of those tokens, compare repositories side by side, and focus optimization efforts where they'll have the biggest impact.\n\nIn observability, context is just as important as the metric itself.\n\nGitHub's remaining API quota is tracked in a similar way.\n\n```\n# github_client.py — \n\nremaining = resp.headers.get(\"X-RateLimit-Remaining\")\nif remaining is not None:\n    github_ratelimit_gauge.set(int(remaining))\n```\n\nGitHub conveniently includes the remaining rate limit in every API response header. Updating the gauge here means it's always current without making any additional API requests.\n\nOnce telemetry starts flowing, open **Metrics → Query Builder** in SigNoz.\n\nHere are the visualizations I found most useful:\n\n`gemini.tokens.total`\n\n`repo`\n\n`assessment.duration`\n\n`github.ratelimit.remaining`\n\nThese dashboards quickly answered questions that were impossible to answer before instrumentation:\n\n📸\n\nSS 4:Metrics Query Builder showing`gemini.tokens.total`\n\ngrouped by the`repo`\n\nattribute.📸\n\nSS 5:Time-series chart for`github.ratelimit.remaining`\n\ndecreasing during consecutive analyses.📸\n\nSS 6:Histogram (or time-series) for`assessment.duration`\n\n, highlighting p50 and p95 latency.\n\nFinally, I combined these visualizations into a custom dashboard named **GitIntel LLM Observability**.\n\nInstead of jumping between multiple pages, I now have a single view showing token usage, GitHub rate-limit headroom, request latency, and overall application health every time I open SigNoz.\n\nAuto-instrumentation is fantastic for HTTP requests, but it doesn't know anything about your application's business logic.\n\nIt can't tell you:\n\nTo answer those questions, I added **manual spans** around the critical parts of GitIntel's analysis pipeline.\n\nFor every Gemini request, `gemini_client.py`\n\ncreates a span that records both the repository being analyzed and the size of the prompt sent to the model.\n\n``` php\n# gemini_client.py — _call_gemini()\n\ndef _call_gemini(prompt: str, repo: str) -> str:\n    with tracer.start_as_current_span(\n        \"gemini.generate\",\n        attributes={\n            \"gemini.repo\": repo,\n            \"gemini.prompt_chars\": len(prompt),\n        },\n    ):\n        for attempt in range(5):\n            try:\n                response = client.models.generate_content(\n                    model=MODEL,\n                    contents=prompt,\n                )\n                _track_usage(response.usage_metadata, repo)\n                return response.text\n            except Exception as e:\n                api_error_counter.add(1, {\"service\": \"gemini\", \"repo\": repo})\n                if attempt == 4:\n                    raise\n                time.sleep(10 * (attempt + 1))\n```\n\nThe `prompt_chars`\n\nattribute became surprisingly useful. While it isn't the exact token count (that's tracked separately as a metric), it's an excellent indicator of prompt size and makes it much easier to correlate larger prompts with longer response times.\n\nI applied the same idea to GitHub.\n\nRather than simply tracing the HTTP requests, GitIntel records **repository-level metadata** during file collection.\n\n```\n# github_client.py — get_all_files()\n\nwith tracer.start_as_current_span(\n    \"github.get_all_files\",\n    attributes={\n        \"github.repo\": f\"{owner}/{repo}\",\n        \"github.branch\": branch,\n    },\n):\n    span = trace.get_current_span()\n\n    # ... fetch tree ...\n\n    span.set_attribute(\"github.file_count.total\", len(tree))\n    span.set_attribute(\"github.file_count.source\", len(source_files))\n```\n\nBy attaching these attributes directly to the span, every trace now includes useful context such as:\n\nThat context makes the trace far more useful than a simple timeline of function calls.\n\nWhen everything runs together, one repository analysis produces a distributed trace that looks like this:\n\n```\napi.analyze\n├── github.get_user              (github.username)\n├── github.get_repos\n├── github.get_all_files         (file_count.total, file_count.source)\n├── gemini.assess_repo           (github.repo, file_count)\n│   ├── gemini.generate          (prompt_chars — batch 1)\n│   └── gemini.generate          (prompt_chars — batch 2, large repos only)\n└── gemini.developer_profile\n    └── gemini.generate\n```\n\nInstead of seeing a single request that took **42 seconds**, I can now see **exactly where those 42 seconds were spent**.\n\nOpen **Traces → Filter by api.analyze** and select any request.\n\nThe trace expands into the full execution tree, showing every nested span, its duration, and its metadata.\n\nClick on any ** gemini.generate** span, and the details panel immediately shows attributes such as:\n\n`gemini.repo`\n\n`gemini.prompt_chars`\n\nThis was the moment observability finally clicked for me.\n\nThe problem was never that GitIntel was \"slow.\" The problem was that I had no visibility into *why* it was slow.\n\nWith distributed tracing, that mystery disappeared.\n\nAt this point, GitIntel was fully instrumented. The real question wasn't whether telemetry was flowing—it was whether it could answer the questions I actually cared about.\n\nTo make the differences obvious, I ran analyses on two repositories with very different characteristics:\n\nSeeing these runs side by side is where observability becomes valuable. The contrast makes latency spikes, token usage, and processing costs immediately visible instead of hidden behind a single loading spinner.\n\nThe first thing that stood out was token consumption.\n\nSome repositories finished after a single Gemini request, while larger ones required multiple batches, causing token usage to increase dramatically.\n\nInstead of guessing why one analysis was expensive, I could see exactly which repository consumed the most tokens and correlate that with the trace.\n\nBefore instrumentation, all I knew was that some analyses took 10 secs while others took nearly a minute.\n\nDistributed traces showed exactly where that time was spent.\n\nGitHub requests completed relatively quickly, while one or more `gemini.generate`\n\nspans dominated the overall request duration.\n\nRather than treating the request as a single slow operation, I could pinpoint the specific step responsible for the delay.\n\nOne assumption I had before adding telemetry was that downloading hundreds of files from GitHub would dominate the execution time.\n\nThe traces showed otherwise.\n\nRepository fetching remained relatively fast, even for larger projects. Most of the end-to-end latency came from prompt generation and LLM inference.\n\nThat insight completely changed where I would spend time optimizing GitIntel.\n\nEvery GitHub API response updates the `github.ratelimit.remaining`\n\ngauge.\n\nInstead of wondering how close I was to GitHub's limit, I could watch it decrease in real time during consecutive analyses.\n\nThis makes it much easier to decide when to slow requests, introduce caching, or back off before hitting the hourly quota.\n\nAfter creating a custom dashboard, I no longer needed to jump between traces, metrics, and logs.\n\nOne screen showed:\n\nInstead of debugging reactively, I could monitor GitIntel proactively.\n\nThree things surprised me after instrumenting GitIntel:\n\nThose are the kinds of insights I simply couldn't have gained without end-to-end observability.\n\nBefore adding observability, every failed analysis looked identical.\n\nSomething was slow.\n\nSomething failed.\n\nSomething used too many tokens.\n\nI had no way of knowing where, why, or how expensive it was.\n\nAfter instrumenting GitIntel with OpenTelemetry and SigNoz, every request became explainable.\n\nThe `prompt_chars`\n\nspan attribute immediately shows which Gemini batch caused a latency spike. The `repo`\n\nattribute on my custom metrics identifies exactly which repository consumed the most tokens. And because every log is correlated with its trace, I can see the complete execution path that led to a failure instead of staring at an isolated error message.\n\nObservability didn't just make GitIntel easier to debug.\n\nIt made the application understandable.\n\nThat's the difference between using AI APIs and owning AI infrastructure.\n\nThis wouldn't be a completely honest engineering blog if everything worked perfectly the first time.\n\nIt didn't.\n\nThese were the two issues that cost me the most time, and the fixes that finally made the setup reliable.\n\nAfter deploying GitIntel to Render, my application logs were immediately flooded with this message every few seconds:\n\n```\nWARNING opentelemetry.exporter.otlp.proto.grpc.exporter Transient error\nStatusCode.UNAVAILABLE encountered while exporting traces to localhost:4317,\nretrying in 0.82s. Failed to connect to remote host: Connection refused\n```\n\nThe reason was simple.\n\nRender wasn't running SigNoz, so nothing was listening on `localhost:4317`\n\n.\n\nOpenTelemetry kept retrying indefinitely, filling the logs with connection errors and making it difficult to spot genuine application issues.\n\nThe fix was a small guard inside `telemetry.py`\n\n:\n\n```\notlp_available = OTLP_ENDPOINT != \"http://localhost:4317\" or os.getenv(\"OTEL_ENABLED\")\n```\n\nIf the exporter endpoint is still the default localhost address and `OTEL_ENABLED`\n\nhasn't been explicitly enabled, GitIntel simply skips creating the OTLP exporters altogether.\n\nThat means:\n\nOne condition removed hundreds of unnecessary log messages.\n\nThe second issue was much more confusing.\n\nI started the entire SigNoz stack successfully.\n\nThe application launched.\n\nRequests completed normally.\n\nBut the **Services** page remained completely empty.\n\nThe problem wasn't OpenTelemetry at all.\n\nIt was startup timing.\n\nAlthough the containers report as running almost immediately after `docker compose up -d`\n\n, ClickHouse still needs roughly a minute to finish initializing before it can persist telemetry. During that window, the OpenTelemetry Collector accepts incoming spans, but the backend isn't fully ready to store them.\n\nThe solution was simply to wait until every container reported a healthy state before launching GitIntel.\n\n```\ncd pours/deployment\ndocker compose up -d\n\n# Wait until all services are healthy — takes ~60 seconds\n\ndocker compose ps\n\n# Look for (healthy) on:\n# signoz-signoz-0\n# signoz-telemetrystore-clickhouse-0-0\n# signoz-telemetrykeeper-clickhousekeeper-0\n# signoz-metastore-postgres-0\n\ncd ../..\nuvicorn main:app --reload\n```\n\nAfter starting the application, I ran a single repository analysis and waited another 15–20 secs.\n\nThat's expected. SigNoz batches telemetry before writing it to storage.\n\nIf the Services page is still empty after about half a minute, the fastest way to diagnose the issue is to inspect the Collector logs:\n\n```\ncd pours/deployment\ndocker compose logs ingester --tail=20\n```\n\nOnce the first spans appear, everything behaves normally from that point onward.\n\nThe delay only affects the initial startup while the telemetry backend finishes initializing.\n\nNeither of these bugs had anything to do with GitIntel itself.\n\nThey were observability infrastructure problems.\n\nIronically, building observability meant I first had to debug the observability stack.\n\nOnce those issues were solved, though, every repository analysis became traceable, measurable, and far easier to reason about. And that's exactly the point of instrumenting an AI application—you spend less time guessing and more time understanding what your system is actually doing.\n\nOpenTelemetry's auto-instrumentation gave me immediate visibility into GitIntel. With just a few lines of code, `FastAPIInstrumentor`\n\nand `RequestsInstrumentor`\n\nautomatically traced every FastAPI request and every outbound GitHub API call. Latency, status codes, request rate; they were all available without touching my application logic.\n\nThat was a great starting point, but it wasn't enough.\n\nAuto-instrumentation could tell me that a request took 52 secs. It couldn't tell me why.\n\nThe real value came from the telemetry I added myself. Attributes like `prompt_chars`\n\n, `repo`\n\n, and file counts transformed traces and metrics into something I could actually reason about. Instead of seeing a slow request, I could identify which Gemini batch was responsible. Instead of seeing total token usage, I could pinpoint exactly which repository consumed the most tokens. And because every log was correlated with its trace, a single click showed the complete execution path leading to a failure.\n\nThe biggest lesson I took away is this:\n\nAuto-instrumentation gives you visibility. Custom instrumentation gives you understanding.\n\nWhen I started GitIntel, I almost skipped observability altogether. I wanted to finish the project first and \"add monitoring later.\"\n\nI'm glad I didn't.\n\nThe unexpected 40,000-token analyses, the OpenTelemetry exporter spam on Render, the ClickHouse cold-start delay, and the huge latency differences between repositories would all have looked like random bugs without traces and metrics. Instead of guessing, I always had evidence pointing me in the right direction.\n\nObservability also changed how I think about metrics. A global token count wasn't nearly as useful as attaching a `repo`\n\nattribute to every measurement. A retry log became far more valuable once it was linked to the exact distributed trace that produced it. Every small piece of context made debugging dramatically easier.\n\nThe biggest shift wasn't technical, it was mental.\n\nI stopped asking, \"Why is my app behaving like this?\" and started asking, \"What does the telemetry tell me?\"\n\nThat's a much better question.\n\nBuilding GitIntel taught me that integrating an LLM is only half the problem. Understanding what the application is doing in production is the other half.\n\nWith OpenTelemetry and SigNoz, every repository analysis became observable from end to end. I could follow GitHub API requests, inspect Gemini calls, measure token usage, monitor latency, correlate logs with traces, and identify failures—all from a single platform.\n\nThat's what observability should do: replace assumptions with evidence.\n\nIf you're building applications powered by LLMs, agents, or external APIs, don't wait until production to add telemetry. Instrument them from day one. The first time something behaves unexpectedly, you'll be glad you did.\n\nBecause in the end, **you can't optimize what you don't measure, you can't debug what you can't see, and you don't truly own an AI system until you can observe it.**", "url": "https://wpnews.pro/news/instrumenting-an-ai-powered-github-analyzer-with-opentelemetry-and-signoz", "canonical_source": "https://dev.to/divyasinghdev/instrumenting-an-ai-powered-github-analyzer-with-opentelemetry-and-signoz-55l3", "published_at": "2026-07-17 12:07:31+00:00", "updated_at": "2026-07-17 12:29:31.108801+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "developer-tools", "ai-infrastructure"], "entities": ["GitIntel", "Gemini 2.5 Flash", "OpenTelemetry", "SigNoz", "GitHub", "FastAPI", "Docker"], "alternates": {"html": "https://wpnews.pro/news/instrumenting-an-ai-powered-github-analyzer-with-opentelemetry-and-signoz", "markdown": "https://wpnews.pro/news/instrumenting-an-ai-powered-github-analyzer-with-opentelemetry-and-signoz.md", "text": "https://wpnews.pro/news/instrumenting-an-ai-powered-github-analyzer-with-opentelemetry-and-signoz.txt", "jsonld": "https://wpnews.pro/news/instrumenting-an-ai-powered-github-analyzer-with-opentelemetry-and-signoz.jsonld"}}