{"slug": "intermittent-500s-on-inference-endpoint-requests-not-reaching-the-container", "title": "Intermittent 500s on Inference Endpoint - requests not reaching the container", "summary": "A user reports intermittent HTTP 500 errors on a custom-handler Inference Endpoint for text embeddings, with ~70% of requests failing since early morning on 2026-07-14. The failures appear to occur at the gateway/router layer before reaching the container, as container logs only show successful requests. The user has ruled out GPU-specific, endpoint-specific, and code-related issues, and seeks information about a possible known problem with the us-east-1 inference gateway.", "body_md": "I’m using a personal account. I’m running a custom-handler Inference Endpoint for text embeddings (Llama-3.1-8B). Since early this morning on 2026-07-14, I’m getting intermittent HTTP 500s (~70% of requests fail, ~30% return 200 normally).\n\nWhat I’ve ruled out:\n\nNot GPU-specific - happens on both L40S and A100.\n\nNot endpoint-specific - brand-new endpoints show the same.\n\nNot my code/deps - the same build ran hundreds of thousands of requests fine yesterday.\n\nContainer is healthy - model loads successfully, and successful requests return correct results in 30–415 ms.\n\nKey finding: the container logs only show the successful requests. The failed (500) requests don’t appear in the container logs at all — they seem to be failing at the gateway/router layer before reaching the container.\n\nThe status page shows no incident. Is there a known issue with the us-east-1 inference gateway, or something on my account causing this? Happy to share endpoint IDs privately.\n\nThanks!", "url": "https://wpnews.pro/news/intermittent-500s-on-inference-endpoint-requests-not-reaching-the-container", "canonical_source": "https://discuss.huggingface.co/t/intermittent-500s-on-inference-endpoint-requests-not-reaching-the-container/177816#post_1", "published_at": "2026-07-14 15:37:36+00:00", "updated_at": "2026-07-14 16:01:05.362147+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-products", "ai-tools"], "entities": ["Llama-3.1-8B", "L40S", "A100", "us-east-1"], "alternates": {"html": "https://wpnews.pro/news/intermittent-500s-on-inference-endpoint-requests-not-reaching-the-container", "markdown": "https://wpnews.pro/news/intermittent-500s-on-inference-endpoint-requests-not-reaching-the-container.md", "text": "https://wpnews.pro/news/intermittent-500s-on-inference-endpoint-requests-not-reaching-the-container.txt", "jsonld": "https://wpnews.pro/news/intermittent-500s-on-inference-endpoint-requests-not-reaching-the-container.jsonld"}}