{"slug": "the-pod-that-did-not-survive-tuesday-s-deploy", "title": "The pod that did not survive Tuesday's deploy", "summary": "Pydantic Logfire launched a new Kubernetes cluster view that surfaces pod restart loops, memory limits, and rollout failures from a single page. The feature uses standard OpenTelemetry receivers to aggregate metrics across clusters, nodes, and workloads, enabling engineers to trace a crash-looping pod to its OOMKilled event and the associated application trace in one click.", "body_md": "Your Vercel AI SDK chatbot, shipped to GKE, is throwing 500s on about 8% of requests since Tuesday's deploy. The error rate is not constant. It spikes for a few minutes, recovers, spikes again. Restart loop, probably. But which pod, on which node, after which release?\n\nWhat's new\n\n- See every layer of your Kubernetes cluster from one page: clusters, nodes, namespaces, workloads, pods, images, all sortable.\n- Find the pod that's restarting, the node that's hot, the workload that's stuck rolling out, the answers you would get from\n`kubectl top`\n\nand`kubectl describe`\n\n, in your browser. - Spot a pod in a crash loop from the cluster view. Restart counts roll up at every level so you do not have to drill all the way down to notice.\n- Click from any pod, namespace or workload straight to the traces it produced in the live view.\n\nWhy this matters\n\nOpen the Workloads tab. Sort by restart count. The chatbot Deployment shows 47 restarts in the last hour. Click in. Desired versus ready replicas tells you the rollout never finished. A memory chart for the workload is pinning the limit on one of the three pods.\n\nSwitch to the Pods tab. Two pods are green and stable. One pod is in a restart loop: OOMKilled, every six minutes. Image digest matches Tuesday's release. The other two pods are running an older digest because the rollout stalled. The new image bundles a tokenizer dependency that doubled memory usage. Mystery solved.\n\nPlenty of tools will tell you a pod is restarting. What was missing for us was the cluster view sitting next to the application traces from the workloads in that cluster, in the same product, with the same trace ID propagating across both. The pod metrics, the kill event, and the trace that triggered it should be one click away from each other. They are now.\n\nAll of this data is available to you and your closest friends: Claude, Codex, Gemini, Pydantic AI, and more via MCP and our RESTful query endpoints.\n\nHow it works\n\nThe data comes from two standard OpenTelemetry collector receivers: **kubeletstats** for pod and container metrics, and **k8scluster** for clusters, namespaces, workloads, and nodes. No DaemonSet of ours, no proprietary agent.\n\nThe drill-down from a pod to *the traces that pod produced* is what the **k8sattributes** processor enables. Run it in your application-trace pipeline and it stamps `k8s.pod.name`\n\n, `k8s.namespace.name`\n\n, `k8s.deployment.name`\n\n(and the workload-kind equivalents) onto every span the pod emits. The cluster view and the live view then share the same identifiers, and one click takes you across.\n\nTwo operational details worth knowing in advance. The `k8scluster`\n\nreceiver doesn't do leader election on itself. Run it as a single-replica Deployment, not a multi-replica one, or you'll double-count cluster metrics. Restart counts roll up at every level (cluster, namespace, workload, node) so you can spot a pod restart loop from the top without drilling all the way down.\n\nFor the full end-to-end setup (the OTel collector config, the `k8sattributes`\n\nprocessor to enrich traces with pod metadata, and unified dashboards), our earlier [Full-stack Kubernetes observability with Pydantic Logfire](/articles/kubernetes-cluster-observability-logfire) post walks through it.\n\nGetting started\n\nIf your collector already exports the kubeletstats and k8scluster receivers, the inventory populates within a minute. If not, the empty state on each tab has a **Set up** button that deep-links to the relevant page of the add-data wizard, with copy-pasteable config.\n\nIn our scenario, you roll back the deploy in two clicks and the restart loop stops. The memory request gets bumped in the manifest, the rollout finishes cleanly, and the trace from the original 500s is in the live view, with the pod restart event open in context.\n\nFull reference for the setup, including the minimum RBAC and the `k8sattributes`\n\nprocessor config: the [Kubernetes view docs](https://pydantic.dev/docs/logfire/observe/kubernetes/).\n\nNot using Logfire yet? [Get started](https://pydantic.dev/logfire). The free tier includes usage up to 10 million spans, our AI Gateway, and so much more.", "url": "https://wpnews.pro/news/the-pod-that-did-not-survive-tuesday-s-deploy", "canonical_source": "https://pydantic.dev/articles/logfire-kubernetes-view", "published_at": "2026-06-17 09:00:00+00:00", "updated_at": "2026-07-16 14:31:30.356000+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure"], "entities": ["Pydantic Logfire", "Vercel AI SDK", "GKE", "OpenTelemetry", "Claude", "Codex", "Gemini", "Pydantic AI"], "alternates": {"html": "https://wpnews.pro/news/the-pod-that-did-not-survive-tuesday-s-deploy", "markdown": "https://wpnews.pro/news/the-pod-that-did-not-survive-tuesday-s-deploy.md", "text": "https://wpnews.pro/news/the-pod-that-did-not-survive-tuesday-s-deploy.txt", "jsonld": "https://wpnews.pro/news/the-pod-that-did-not-survive-tuesday-s-deploy.jsonld"}}