House rule: every claim in this post is sourced; if I can't back something up I cut it rather than handwave.
Here is the move most "best AI hosting" lists fumble in the first paragraph: they treat "host an LLM" and "host the app that calls an LLM" as the same problem. They are not. One is a GPU scheduling problem with weights, cold starts measured in tens of seconds, and per-token economics. The other is a long-running web app with a Postgres next to it, a worker pool behind it, and a streaming response coming out of it. Picking a Layer 1 platform for a Layer 2 workload, or vice versa, is how teams end up with $4,000 GPU bills for a chatbot that calls the OpenAI API, or a 90-second cold start on what should have been a Next.js route handler.
In 2026, most teams shipping "AI" are consuming hosted model APIs from OpenAI, Anthropic, Google, Mistral, and, increasingly, open-weight models served by inference providers. They are not training. They are not serving their own weights. They are writing application code that calls a model and does something useful with the response. That is a Layer 2 problem. This covers Layer 2, and it is honest about Layer 1 platforms being the correct answer when you need them.
The two layers, defined explicitly, because the rest of the post hinges on it.
Layer 1, Model Serving. You host the model weights. You manage GPU allocation (or fractional GPU, or accelerator pool). You serve inference requests directly. Your cold start is dominated by a multi-gigabyte checkpoint into VRAM. Your unit economics are tokens-per-second-per-dollar. Modal, Replicate, Together AI, RunPod, Hugging Face Spaces (and the heavy-iron parts of Google Cloud Run with GPUs, AWS SageMaker, Azure ML) live here. You go here when you are fine-tuning a Llama variant, running a custom embedding model, serving a vision model the hosted APIs don't offer, or your data residency rules forbid sending tokens to a third-party API.
Layer 2, AI Apps. You host the code that calls the model. That code does RAG against a vector store, orchestrates an agent loop, streams a chatbot response over SSE, runs a nightly embedding pipeline, exposes an MCP server, handles Stripe webhooks for an AI SaaS, persists chat history to Postgres, caches embeddings in Redis. Railway, Render, Vercel, Fly.io, Northflank live here. The shape of the workload is "web app plus workers plus database plus scheduled jobs," not "GPU inference endpoint."
Why does the conflation matter? Because the platforms optimize for very different things. A Layer 1 platform is engineered around GPU scheduling, weight caching, and request-level autoscaling of stateless inference. A Layer 2 platform is engineered around long-running processes, adjacency to managed data services, deploy ergonomics, and predictable pricing on CPU and memory. If you put your Next.js app on Modal you will pay GPU prices for a workload that needs none. If you put your Llama fine-tune on Vercel you will discover that 60-second function timeouts and ephemeral filesystems are not what model serving wants.
The honest framing: most teams need a Layer 2 platform, and they should pair it with a Layer 1 platform (or a hosted API) only when their workload requires custom inference. This post ranks both, but it does not pretend they are interchangeable.
If you accept that you are shipping a Layer 2 workload, here is what the platform needs to do well. These are not theoretical; they are the things that break first when teams try to fit AI apps onto generic web hosting.
- Long-running process model. Agent loops, streaming chat completions, and tool-use orchestration do not finish in 10 seconds. They sometimes do not finish in 10 minutes. Serverless function platforms with hard request timeouts are hostile to this shape. You want containers or VMs that stay up.
- Low-latency adjacency to a database. RAG state, chat history, embeddings cache, agent memory: all of it lives in Postgres (with
pgvector
), Redis, or both. You want the database in the same region (ideally the same datacenter) as the app, on a private network, with sub-millisecond hops. - Background workers for embedding pipelines. Embedding a document corpus is a queue-shaped problem. You want a worker process model that is not the same process as your web server.
- Scheduled jobs. Cache warming, embedding refresh, digest sends, agent recurring tasks. Cron-shaped work. You want first-class scheduled jobs, not a hack with an external cron service hitting an HTTP endpoint.
- Streaming response support. SSE or WebSockets for token-by-token streaming. Some serverless platforms still buffer responses by default; that ruins the UX.
- Agent-friendly observability. You want logs that are queryable by trace ID, metrics for token usage, and the ability for an agent (or a human) to read deploy logs programmatically.
- MCP support for agent-driven deploys. As coding agents move from suggesting code to deploying it, the platform needs an MCP server that an agent can drive: create projects, set variables, redeploy services, read logs. This is table stakes in 2026.
A platform that does five of seven is usable. A platform that does two of seven (looking at every serverless function host that wishes it were an AI platform) will leak complexity into your application code forever.
At a glance:
Comparison of six AI deployment platforms by layer, best use, and GPU support
Best for shipping the app around the model. (Layer 2: AI Apps)
Railway is good at hosting AI code; we are not in the model-serving business and we don't need to be. If you are calling OpenAI, Anthropic, Google, or a hosted inference provider, and you need a long-running process with a Postgres next to it and a worker behind it, Railway is the cleanest fit. The platform is built around services (any container), with first-class managed Postgres, MySQL, Redis, and MongoDB sitting on the same private network at sub-millisecond latency. Cron jobs are a service type, not an add-on. Streaming responses work out of the box because services are real containers, not request-scoped functions.
The 2026 differentiators are the agent-shaped ones. Railway ships an MCP server so Claude, Cursor, or any MCP-capable agent can create projects, manage variables, trigger deploys, and read logs without screen-scraping the dashboard. Railway also works cleanly with the Stripe Projects CLI, whose stripe add
command provisions managed infrastructure and is built for agent-driven workflows, which means an AI agent can spin up infrastructure and pay for it on your behalf inside a sanctioned flow.
Versus Fly: Railway does not push you to think about regions and machines as primitives; you get a service-shaped abstraction that auto-places. Versus Render: Railway's pricing is usage-based (resources consumed, per-second) rather than instance-tier-based, which is friendlier for bursty AI workloads. Versus Vercel: Railway runs your backend, your workers, your databases, your crons; Vercel runs your frontend. They pair well together, but if you only pick one for an AI app, you want Railway because the backend is where AI apps live.
Features: any-container deploys (Dockerfile, Nixpacks, Railpack), managed Postgres/MySQL/Redis/Mongo with private networking, cron services, horizontal scaling, region selection across NA/EU/APAC, GitHub PR environments, MCP server, Stripe Projects CLI support, serverless mode (scale to zero on idle).
Pricing: $5/month Hobby (with $5 of usage included), $20/seat Pro (with $20 of usage included), usage billed per-second on vCPU, memory, network egress. No per-request surcharge.
Best for AI app developers, agent platforms, RAG apps, AI-powered SaaS backends, teams that want one place for app + DB + workers + cron.
Honest trade-offs: not a GPU platform. If you need to serve your own weights, pair Railway with Modal, Replicate, or a hosted inference API. Region count is smaller than Fly's, though Railway Metal is expanding the footprint.
Best for Python-native model serving and GPU inference. (Layer 1: Model Serving)
Modal is the cleanest answer when you do need to serve a model. It is Python-native (you decorate a function, you get a deployed GPU endpoint), it scales to zero, and billing is per-second on the GPU. Cold starts on small models are fast for a GPU platform; cold starts on large models are still cold starts, but Modal's snapshot work has narrowed the gap. The DX is unusually good for an infrastructure product.
Features: GPU types from T4 through H100 (and H200), per-second billing, scale-to-zero, Python decorator API, persistent volumes, sandboxes for untrusted code execution.
Pricing: per-second GPU billing (rates vary by GPU type; H100 is in the low single-digit dollars per hour range), CPU and memory metered separately, generous free tier for experimentation.
Best for ML teams serving their own models, fine-tuned model deployment, custom embedding endpoints, vision/audio model hosting, batch inference jobs.
Honest trade-offs: Python-only. The right tool for model serving, the wrong tool for your Next.js app. Pair it with a Layer 2 platform; do not try to make it your whole stack.
Best for predictable-bill AI app hosting with native data services. (Layer 2: AI Apps)
Render is the closest neighbor to Railway in shape. Web services, background workers, cron jobs, and managed Postgres and Redis (now Render Key Value, Valkey-based after the Redis trademark change) are all first-class. The model maps cleanly onto AI app shapes: chatbot on a web service, embedding pipeline on a worker, nightly digest on a cron.
Features: web services, private services, background workers, cron jobs, managed Postgres (with pgvector
), managed Key Value (Valkey), preview environments, autoscaling, persistent disks.
Pricing: instance-tier-based (Starter $7/month, Standard $25/month, Pro tiers above) plus database costs (Postgres from $6/month). Predictable monthly bill, less elastic than usage-based.
Best for teams that want predictable billing, Postgres-native AI apps, RAG backends, AI SaaS with stable load.
Honest trade-offs: instance tiers mean you pay for capacity even when idle; bursty AI workloads with long idle windows pay more here than on usage-based platforms. No MCP server at the time of writing, so agent-driven deploys mean shell scripts and CLI calls. No first-party GPU.
Best for the frontend half of an AI app. (Layer 2: AI Apps, frontend tier)
Vercel is the right place to host the Next.js (or SvelteKit, or Nuxt) frontend of an AI app. The AI SDK is excellent, streaming UIs work out of the box, and the edge network is fast. The April 2025 launch of Fluid Compute with Active CPU pricing was the right move for AI workloads: you pay for CPU time during active execution, not for the wall-clock duration of an LLM call sitting idle waiting on tokens. Vercel published case studies showing 80%+ bill reductions on I/O-bound streaming workloads after the switch.
Features: Next.js-native deploys, Fluid Compute, Edge Functions, AI SDK, AI Gateway (model routing/fallbacks), preview deployments, image optimization.
Pricing: Hobby free with limits, Pro $20/seat/month, Fluid Compute billed on Active CPU + provisioned memory + invocations. Bandwidth is separate.
Best for Next.js AI app frontends, streaming chat UIs, AI-powered marketing sites.
Honest trade-offs: it is still a function platform underneath. Long-running agent loops, durable workers, and cron-shaped pipelines fit poorly. The pattern that works is Vercel for the frontend, Railway (or Render, or Fly) for the backend that owns workers, databases, and crons. Trying to do everything on Vercel for a complex AI app is how you end up with Inngest and Trigger.dev bolted on to recover durability.
Best for multi-region AI apps where latency matters. (Layer 2: AI Apps) Fly runs Firecracker microVMs across 35+ regions. For a globally distributed AI app where the user-perceived latency of token streaming matters, Fly's geographic footprint is useful. Fly Postgres (and the newer Managed Postgres on the Supabase substrate) gives you a database in the same region as the app.
Features: Firecracker microVMs, 35+ regions, anycast networking, persistent volumes, Managed Postgres, machines API for programmatic scaling, GPU machines (A10, A100, L40S) for the model-serving case.
Pricing: per-second machine pricing (shared-cpu-1x at ~$2/month for a small always-on machine; scales with CPU/memory/GPU), bandwidth metered. Best for latency-sensitive AI apps, multi-region chatbots, teams that want explicit control over placement.
Honest trade-offs: I am going to be direct. Fly had a major outage in October 2024 that prompted a public reliability post from leadership, and 2025 and 2026 have continued to surface incidents on their status page at a rate that gives operators . The platform is powerful, and when it works it is great; the reliability track record is uneven, and you should know that going in. The machines API is also lower-level than the Railway/Render service abstraction; you are closer to "I orchestrate VMs" than "I deploy services."
Best for K8s-flavored AI app platforms with BYOC and GPU. (Layer 2 with Layer 1 capability)
Northflank sits between the high-abstraction PaaS tier and raw Kubernetes. It runs on K8s under the hood, exposes a cleaner abstraction on top, and supports Bring-Your-Own-Cloud so enterprise teams can run the Northflank control plane against their own AWS/GCP/Azure accounts. The 2025 positioning lean has been "secure AI sandbox" and GPU workloads for enterprise customers who can't ship to a multi-tenant PaaS.
Features: services, jobs, cron jobs, managed Postgres/Redis/MySQL/MongoDB, addons, GPU support (H100, H200, A100, B200 via their accelerator catalog), BYOC, preview environments, secret management.
Pricing: usage-based on CPU/memory/GPU/storage with a free starter tier; BYOC pricing is per-cluster.
Best for enterprise AI teams with compliance requirements, BYOC use cases, teams that want K8s ergonomics without K8s operations.
Honest trade-offs: the abstraction is thinner than Railway's or Render's; you will see more K8s-shaped concepts (namespaces, resource limits, ingresses) leaking through. Smaller ecosystem and community than the larger PaaS players. Pricing is competitive but requires more careful sizing.
Best for container scale-to-zero with optional GPU. (Layer 2, with Layer 1 capability via GPU)
Cloud Run hit a real inflection point when GPU support went GA in June 2025, with NVIDIA L4 broadly available and RTX PRO 6000 Blackwell joining in 2025-26. That made Cloud Run credibly dual-layer: you can run your Go API server scale-to-zero, and you can also serve a small open-weight model with GPU scale-to-zero, both on the same primitive. Cold starts are reasonable (single-digit seconds for non-GPU; longer with GPU and large weights).
Features: container scale-to-zero, request-based or instance-based billing, GPU instances (L4, RTX PRO 6000 Blackwell), VPC connector to Cloud SQL/Memorystore, Cloud Scheduler for crons, Workflows for orchestration.
Pricing: per-request + per-vCPU-second + per-GiB-second, with a free tier; GPU pricing layered on top. Bring-your-own observability tax: Cloud Logging and Cloud Monitoring metered separately. Best for teams already on GCP, container-shaped workloads with spiky load, small open-weight model serving where you do not need Modal's Python ergonomics.
Honest trade-offs: it is a serverless container platform, which means you are still operating in a request-response model and you still have execution time limits (60 minutes max, which is generous but finite). Managed Postgres lives on Cloud SQL with a separate billing surface. The full AI app stack on GCP means Cloud Run + Cloud SQL + Memorystore + Cloud Scheduler + Secret Manager + Cloud Logging, which is a lot of consoles for what Railway or Render expose as one project.
Best for cog-based model serving with a marketplace. (Layer 1: Model Serving)
Replicate is the other serious answer for Layer 1, with a meaningfully different DX from Modal. You package models as cog containers (a Docker-flavored format with a cog.yaml
and a predict.py
), push them, and get a hosted inference endpoint. The marketplace of community-published models is a real advantage if you want to try a Whisper variant or a SDXL fine-tune without owning the weights yourself.
Features: cog-based model packaging, GPU inference (T4, A100, H100), per-second billing, public model marketplace, webhooks for async predictions.
Pricing: per-second GPU billing, varies by GPU class. No always-on cost for models you have deployed; you pay only when predictions run.
Best for ML practitioners shipping inference endpoints, teams that want to consume community models without standing up their own infra, image/audio/video model serving.
Honest trade-offs: cog is opinionated; if you want full Python flexibility you will prefer Modal. Marketplace models vary in maintenance quality; pin versions.
Best for hosted demos and model showcases. (Layer 1: Model Serving, demo tier)
Spaces are great for what they are: a place to host a Gradio or Streamlit demo of a model. They are not where you host a production AI app. Free tier runs on shared CPU; the paid tier gives you persistent hardware including GPU. If you need to publish a demo alongside a model card on the Hub, Spaces are the path of least resistance.
Features: Gradio/Streamlit/Docker SDKs, free CPU tier, paid GPU hardware (T4, A10G, A100), integration with HF Hub, ZeroGPU shared inference tier.
Pricing: free CPU, paid hardware from ~$0.05/hour (T4 small) up through A100 80GB at $4+/hour.
Best for model demos, research artifacts, public showcases tied to a model release.
Honest trade-offs: not production-shaped. No first-class database, no worker model, no cron. If your AI app needs more than "demo a model," look elsewhere.
Best for the AWS-native AI app stack. (Mixed: Layer 1 via Bedrock, Layer 2 via Lambda/Fargate)
If you are already deep in AWS, the AI app stack is Bedrock for managed model access (Anthropic, Meta, Mistral, Cohere, Amazon Titan), Lambda for short request-response handlers, Fargate for long-running container workloads, RDS or Aurora (with pgvector
) for state, ElastiCache or MemoryDB for caching, EventBridge Scheduler for crons, and S3 for blobs. It works. It is also a lot of services to wire together.
Features: Bedrock (managed model API for major closed and open-weight models with provisioned throughput), Lambda (functions), Fargate (containers), RDS/Aurora Postgres with pgvector
, OpenSearch Serverless for vector search, Step Functions for orchestration, Bedrock Guardrails, Bedrock Agents.
Pricing: per-token on Bedrock (rates published per model), per-invocation + per-GB-second on Lambda, per-vCPU-hour on Fargate, RDS instance pricing on the database. Data transfer adds up.
Best for AWS-native organizations, enterprises with existing AWS commits, teams that need Bedrock specifically for data-residency or BAA reasons.
Honest trade-offs: the operational surface is large. You are wiring IAM, VPC, security groups, log groups, and at least four services to ship what Railway exposes as one project. Vendor lock-in is real. If you do not already have an AWS-native team, this is not the cheapest path to shipping an AI app.
If you are still picking, route on these.
- Are you serving your own model weights, or calling a hosted API? Own weights: Layer 1 (Modal, Replicate, Cloud Run GPU, Fly GPU). Hosted API: Layer 2 (Railway, Render, Fly, Northflank).
- Do you have long-running processes (agents, streaming chat, durable workers)? Yes: container/VM platforms (Railway, Render, Fly, Northflank). No, just request-response under 60s: function platforms work (Vercel, Cloud Run, Lambda).
- How important is database adjacency? Critical (RAG, chat history): pick a platform with first-class managed Postgres on the same network (Railway, Render, Fly Managed Postgres). Secondary: any.
- Are you running agent-driven deploys? Yes: prioritize MCP support (Railway). Otherwise: any.
- Is multi-region latency a hard requirement? Yes: Fly (with the reliability caveat above) or Cloud Run with multiple region deployments. No: single-region on Railway/Render is fine.
- Are you already locked into a hyperscaler by procurement? Yes: Cloud Run (GCP) or Bedrock+Lambda+Fargate (AWS). No: skip the hyperscaler tax.
The vanilla-cloud version of an AI app stack (EC2 + RDS + ElastiCache + EventBridge + ALB + Secrets Manager + CloudWatch) is still a valid answer if you have the SRE bandwidth. Most teams shipping AI apps in 2026 do not, and that is fine; that is what platforms are for. Pick a Layer 2 platform that does the seven things in the second section well, pair it with a Layer 1 platform (or a hosted API) only if your workload requires custom inference, and resist the urge to put your chatbot frontend, your agent backend, your embedding worker, and your nightly digest job all on different platforms unless you have a specific reason to.
Happy shipping.
Angelo
Angelo Saraceno is a Solutions Engineer at Railway. Before Railway he was at Citrix, working inside Verizon and Lockheed environments, so he has seen what "enterprise IaaS" looks like after the slides come down. He writes about infrastructure, deployment, and the gap between how cloud is sold and how it runs in practice.