{"slug": "tserve-open-source-inference-server-for-time-series-foundation-models", "title": "TServe – Open-source inference server for time-series foundation models", "summary": "Sktime released TServe, an open-source inference server that loads time-series foundation models including Chronos, TimesFM, Moirai, TTM, and TiRex once into memory and serves forecast requests over HTTP via a single POST /predict endpoint. The server ships over 100 checkpoints, each with its own Docker tag or pip extra for CPU or GPU, and wraps model families through sktime's common forecaster interface so switching models requires changing one field. TServe runs on user hardware rather than as a hosted API, with a Python client that accepts dict, pandas, polars, or pyarrow tables and returns predictions in the same type.", "body_md": "|  | **[Documentation](https://tserve.readthedocs.io/en/latest/)** ·**[Quick start](https://tserve.readthedocs.io/en/latest/quick-start/)** ·**[Models](https://tserve.readthedocs.io/en/latest/models/)** ·**[API](https://tserve.readthedocs.io/en/latest/reference/http/)** | \n|---|---|\n| **Project** |    | \n| **Status** |    | \n\n**Time series serving for foundation models.** TServe loads models such as Chronos, TimesFM, Moirai, TTM, and TiRex once, keeps them in memory, and answers forecast requests over HTTP.\n\nEach model family ships its own package, input format, and loading code. [sktime](https://www.sktime.net/) wraps them as forecasters with one common interface, and TServe runs those forecasters as a server behind a single request: a table of past values and a horizon. Trying another model means changing one field, not rewriting your pipeline.\n\n- **Over 100 checkpoints.** Each family has its own Docker tag or pip extra, for CPU or GPU.[Catalog](https://tserve.readthedocs.io/en/latest/models/) ·[Capabilities](https://tserve.readthedocs.io/en/latest/models/#capabilities)\n- **Loaded once, kept warm.** Weights download and load at startup, so each request pays only for inference.\n- **JSON from anywhere.**`POST /predict` works from curl or any language.[Send a prediction](https://tserve.readthedocs.io/en/latest/client/http/#send-a-prediction)\n- **Native tables in Python.** The[`Client`](https://tserve.readthedocs.io/en/latest/client/python/#connect) takes a dict, pandas, polars, or pyarrow table and returns predictions in the same type.\n- **A dashboard in the browser.**`GET /` plots a forecast from a sample series or your own CSV.[What you can do](https://tserve.readthedocs.io/en/latest/server/dashboard/#what-you-can-do)\n- **Your own sktime models.** Serve a[configured forecaster](https://tserve.readthedocs.io/en/latest/server/live-objects/) , a[saved `.zip`](https://tserve.readthedocs.io/en/latest/server/models-dir/) , or a[craft spec](https://tserve.readthedocs.io/en/latest/server/craft-specs/) next to the catalog models.\n\nTServe is a server you run on your own hardware, not a hosted API. [How it works](https://tserve.readthedocs.io/en/latest/overview/) · [Docker Hub](https://hub.docker.com/r/sktime/tserve)\n\nDocker is the short path. This image can load Chronos Bolt, Chronos T5, TTM, and TimesFM 2.x. The first start downloads the weights you name.\n\n```\ndocker run --rm -p 8000:8000 sktime/tserve:hub chronos_bolt ttm_r3\n```\n\nWhen the log prints the local URLs, the models are warm. Five days of sales, three steps ahead:\n\n```\ncurl -s http://127.0.0.1:8000/predict -H \"Content-Type: application/json\" -d '{\n  \"past\": {\n    \"timestamp\": [\"2024-01-01\", \"2024-01-02\", \"2024-01-03\", \"2024-01-04\", \"2024-01-05\"],\n    \"sales\": [120, 135, 128, 142, 138]\n  },\n  \"fh\": 3,\n  \"model\": \"chronos_bolt\"\n}'\n{\n  \"predictions\": {\n    \"timestamp\": [\"2024-01-06T00:00:00\", \"2024-01-07T00:00:00\", \"2024-01-08T00:00:00\"],\n    \"sales\": [139.96, 138.93, 138.26]\n  },\n  \"quantiles\": null,\n  \"model\": \"chronos_bolt\",\n  \"request_id\": \"…\"\n}\n```\n\nOpen [http://127.0.0.1:8000/](http://127.0.0.1:8000/), pick `chronos_bolt`, and plot the same series. The page can also take a pasted or dropped CSV. [What you can do](https://tserve.readthedocs.io/en/latest/server/dashboard/#what-you-can-do)\n\nThe same call from Python. The client posts Arrow, and `predictions` comes back as the same kind of table you sent:\n\n```\npip install \"tserve[client]\"\npython\nfrom tserve.client import Client\n\npast = {\n    \"timestamp\": [\"2024-01-01\", \"2024-01-02\", \"2024-01-03\", \"2024-01-04\", \"2024-01-05\"],\n    \"sales\": [120, 135, 128, 142, 138],\n}\n\nwith Client(\"http://127.0.0.1:8000\") as client:\n    result = client.predict(past=past, fh=3, model=\"chronos_bolt\")\n\nprint(result.predictions)\n```\n\nThe walkthrough, including `GET /models` and PowerShell: [Quick start](https://tserve.readthedocs.io/en/latest/quick-start/). A GPU host adds `--gpus all` and uses `sktime/tserve:hub-gpu`. [GPU images](https://tserve.readthedocs.io/en/latest/server/docker/#gpu-images)\n\nThe running server serves a browser console at `GET /`. The model list is whatever this process loaded. You set a horizon, optionally a prediction interval, and a series (a built-in sample, pasted CSV, or a dropped file, parsed in the browser), then the page posts `POST /predict` and plots the result. Health and runtime stats sit on the right. [What you can do](https://tserve.readthedocs.io/en/latest/server/dashboard/#what-you-can-do)\n\nBelow, [`timesfm_3`](https://tserve.readthedocs.io/en/latest/models/timesfm3/) forecasts retail sales with 90% prediction interval.\n\n117 checkpoints. The extra name is the image tag, `sktime/tserve:<tag>`, and `server` publishes as `:base`. GPU tags append `-gpu`. `base` has no GPU tag. **added** counts checkpoints that extra contributes. `full` is the total, including `naive`.\n\n`naive` always loads, so you can try the process before any download. `GET /models` lists what this process loaded, which is smaller than the catalog. [What gets loaded](https://tserve.readthedocs.io/en/latest/overview/#what-gets-loaded)\n\n| extra | families | added | example | \n|---|---|---|---|\n| [`server`](https://tserve.readthedocs.io/en/latest/models/base/) | Naive | 1 | `naive` | \n| [`hub`](https://tserve.readthedocs.io/en/latest/models/hub/) | Chronos Bolt, Chronos T5, TTM, TimesFM 2.x | 81 | `chronos_bolt` | \n| [`chronos`](https://tserve.readthedocs.io/en/latest/models/chronos/) | Chronos-2 | 3 | `chronos_2` | \n| [`kronos`](https://tserve.readthedocs.io/en/latest/models/kronos/) | Kronos, WindFM | 5 | `kronos` | \n| [`granite`](https://tserve.readthedocs.io/en/latest/models/granite/) | FlowState | 2 | `flowstate` | \n| [`moirai`](https://tserve.readthedocs.io/en/latest/models/moirai/) | Moirai 2, Moirai 1.x, Lag-Llama | 8 | `moirai_2` | \n| [`tirex`](https://tserve.readthedocs.io/en/latest/models/tirex/) | TiRex | 2 | `tirex` | \n| [`tirex2`](https://tserve.readthedocs.io/en/latest/models/tirex2/) | TiRex-2 | 4 | `tirex_2` | \n| [`toto`](https://tserve.readthedocs.io/en/latest/models/toto/) | Toto-2 | 5 | `toto_2_0_4m` | \n| [`mantis`](https://tserve.readthedocs.io/en/latest/models/mantis/) | Mantis | 3 | `mantis_8m` | \n| [`timesfm3`](https://tserve.readthedocs.io/en/latest/models/timesfm3/) | TimesFM 3 | 1 | `timesfm_3` | \n| [`t0`](https://tserve.readthedocs.io/en/latest/models/t0/) | T0 | 1 | `t0` | \n| [`tafsut`](https://tserve.readthedocs.io/en/latest/models/tafsut/) | Tafsut | 1 | `tafsut` | \n| [`full`](https://tserve.readthedocs.io/en/latest/models/full/) | all of the above | 117 | `chronos_2` | \n\n`kronos` is built on `base`. Chronos Bolt, TTM, and TimesFM 2.x load on the images that include `hub`: `chronos`, `granite`, `moirai`, `tirex`, `tirex2`, `toto`, `mantis`, `timesfm3`, `t0`, `tafsut`, and `full`. TimesFM 3, TiRex-2, T0, and Tafsut load on their own extras and on `full`. Tags, GPU variants, and how the extras stack: [Dependencies](https://tserve.readthedocs.io/en/latest/models/#dependencies).\n\nEach family page has its own start command. The catalog collects them under [Start a server](https://tserve.readthedocs.io/en/latest/models/#start-a-server). Switching images is the tag plus the example from that row:\n\n```\ndocker run --rm -p 8000:8000 sktime/tserve:moirai moirai_2\n```\n\nMultivariate series, covariates, and quantiles differ by family: [Capabilities](https://tserve.readthedocs.io/en/latest/models/#capabilities). Every checkpoint name: [All models](https://tserve.readthedocs.io/en/latest/models/#all-models). `mantis` needs more than 127 rows of `past`: [mantis](https://tserve.readthedocs.io/en/latest/models/mantis/).\n\nDocker needs no local Python. uv and pip need Python 3.12 or newer. Install the extra, or pull the tag, for the family in the table above.\n\n- **Docker.**[Pull an image](https://tserve.readthedocs.io/en/latest/server/docker/#pull-an-image) , then[run the server](https://tserve.readthedocs.io/en/latest/server/docker/#run-the-server) . Each family has a CPU tag and a`-gpu` tag.\n- **uv or pip.**[UV / Pip](https://tserve.readthedocs.io/en/latest/installation/#uv-pip) . A CPU-only build for your OS on[CPU-only install](https://tserve.readthedocs.io/en/latest/server/pip/#cpu-only-install) .\n- **A clone.**[From source](https://tserve.readthedocs.io/en/latest/server/source/) . An editable install for development and unreleased changes.\n\nA bare `tserve` loads `naive` only. Name the models you want beside it. Flags are `--host`, `--port`, and `--log-level`: [Flags](https://tserve.readthedocs.io/en/latest/reference/cli/#flags) · [Startup and exit](https://tserve.readthedocs.io/en/latest/reference/cli/#startup-and-exit).\n\n- **On the command line.**[Start](https://tserve.readthedocs.io/en/latest/server/#start) ·[Serve from the command line](https://tserve.readthedocs.io/en/latest/server/pip/#serve-from-the-command-line)\n- **From Python.**[`Server`](https://tserve.readthedocs.io/en/latest/server/pip/#serve-from-python) loads models before the port is bound. The same entry point from code:[Python entry point](https://tserve.readthedocs.io/en/latest/reference/cli/#python-entry-point) .\n- **In Docker, with a token and a weight cache.**[Hugging Face token](https://tserve.readthedocs.io/en/latest/server/docker/#hugging-face-token) ·[Keep weights between runs](https://tserve.readthedocs.io/en/latest/server/docker/#keep-weights-between-runs) ·[Choose which models to load](https://tserve.readthedocs.io/en/latest/server/docker/#choose-which-models-to-load)\n- **An estimator you already built.** Pass`(id, estimator)` . Predict requests use that id as`model` .[Live objects](https://tserve.readthedocs.io/en/latest/server/live-objects/) ·[Configured Hub estimators](https://tserve.readthedocs.io/en/latest/server/live-objects/#configured-hub-estimators)\n- **A craft spec.** A class call with constructor kwargs and no imports. On the CLI it is`id=spec` .[From the command line](https://tserve.readthedocs.io/en/latest/server/craft-specs/#from-the-command-line) ·[Rules](https://tserve.readthedocs.io/en/latest/server/craft-specs/#rules)\n- **A saved sktime `.zip`.**[Save a model](https://tserve.readthedocs.io/en/latest/server/models-dir/#save-a-model) ·[Load them](https://tserve.readthedocs.io/en/latest/server/models-dir/#load-them) · in Docker:[Models from a directory](https://tserve.readthedocs.io/en/latest/server/docker/#models-from-a-directory)\n\nStartup prints the dashboard, Swagger, and ReDoc. [What you can do](https://tserve.readthedocs.io/en/latest/server/dashboard/#what-you-can-do) · [Live OpenAPI](https://tserve.readthedocs.io/en/latest/server/dashboard/#live-openapi)\n\nJSON goes to `POST /predict`. The Python client posts Arrow to `POST /predict/bytes`. Both send the same fields. [Request fields](https://tserve.readthedocs.io/en/latest/client/data/#request-fields)\n\n`past` is one row per timestamp, `fh` is how many steps ahead, and the forecast continues from the last row. Omit `time` and the first column is time. Omit `target` and the other columns are the series, except any you also put in `future`. [Column roles](https://tserve.readthedocs.io/en/latest/client/data/#column-roles) · [Column inference](https://tserve.readthedocs.io/en/latest/client/data/#column-inference) · [Prediction horizon and model](https://tserve.readthedocs.io/en/latest/client/data/#prediction-horizon-and-model)\n\n| you want | read | \n|---|---|\n| JSON from any language | [Send a prediction](https://tserve.readthedocs.io/en/latest/client/http/#send-a-prediction) ·[Endpoints](https://tserve.readthedocs.io/en/latest/client/http/#endpoints) | \n| Row-oriented JSON, or Arrow | [Use row-oriented JSON](https://tserve.readthedocs.io/en/latest/client/http/#use-row-oriented-json) ·[Arrow endpoint](https://tserve.readthedocs.io/en/latest/client/http/#arrow-endpoint) ·[Table formats](https://tserve.readthedocs.io/en/latest/client/data/#table-formats) | \n| pandas, polars, or pyarrow | [Use native tables](https://tserve.readthedocs.io/en/latest/client/python/#use-native-tables) ·[Connect](https://tserve.readthedocs.io/en/latest/client/python/#connect) | \n| A pandas `DatetimeIndex` | [Use an indexed pandas frame](https://tserve.readthedocs.io/en/latest/client/python/#use-an-indexed-pandas-frame) ·[Time](https://tserve.readthedocs.io/en/latest/client/data/#time) | \n| Covariates or a static row | [Future and static data](https://tserve.readthedocs.io/en/latest/client/data/#future-and-static-data) ·[Request covariates](https://tserve.readthedocs.io/en/latest/client/python/#request-covariates) | \n| Quantiles | [Quantiles](https://tserve.readthedocs.io/en/latest/client/data/#quantiles) ·[HTTP](https://tserve.readthedocs.io/en/latest/client/http/#request-quantiles) ·[Python](https://tserve.readthedocs.io/en/latest/client/python/#request-quantiles) | \n| The response shape | [Response](https://tserve.readthedocs.io/en/latest/client/data/#response) | \n| Health, loaded models, latency | [Inspect the server](https://tserve.readthedocs.io/en/latest/client/http/#inspect-the-server) ·[Status routes](https://tserve.readthedocs.io/en/latest/reference/http/#status-routes) | \n\nA body the schema rejects is **422**. An unloaded model or a missing column is **400**. [Predict requests](https://tserve.readthedocs.io/en/latest/reference/errors/#predict-requests) · [Python client errors](https://tserve.readthedocs.io/en/latest/reference/errors/#python-client) · [Startup](https://tserve.readthedocs.io/en/latest/reference/errors/#startup)\n\nWhich families can take more than one target, a covariate, or a quantile: [Capabilities](https://tserve.readthedocs.io/en/latest/models/#capabilities). Panel and hierarchical input are outside this contract. [Validation and limits](https://tserve.readthedocs.io/en/latest/client/data/#validation-and-limits)\n\nThe generated reference for the same surface: [HTTP API](https://tserve.readthedocs.io/en/latest/reference/http/) · [`POST /predict`](https://tserve.readthedocs.io/en/latest/reference/http/#post-predict) · [Python API](https://tserve.readthedocs.io/en/latest/reference/api/).\n\nBSD 3-Clause. See [LICENSE](https://github.com/sktime/tserve/blob/main/LICENSE).\n\nLicense covers only the model server, not the models themselves or distributions pathways such as Hugging Face. Third party model weights, model code, or distribution pathways may create their own implications via licenses or T&C. While we try to make it easy for users to gain a transparent picture of legal implications, we do not assume any liability or guarantee correctness of metadata related to third party licenses or T&C.\n\nDevelopment setup, checks, tests, and image builds: [Development](https://tserve.readthedocs.io/en/latest/reference/development/) · [Checks](https://tserve.readthedocs.io/en/latest/reference/development/#checks) · [Tests](https://tserve.readthedocs.io/en/latest/reference/development/#tests) · [Docker images](https://tserve.readthedocs.io/en/latest/reference/development/#docker-images).", "url": "https://wpnews.pro/news/tserve-open-source-inference-server-for-time-series-foundation-models", "canonical_source": "https://github.com/sktime/tserve", "published_at": "2026-10-01 10:11:44+00:00", "updated_at": "2026-10-01 10:46:12.249330+00:00", "lang": "en", "topics": ["ai-infrastructure", "machine-learning", "ai-tools", "developer-tools", "ai-products"], "entities": ["TServe", "sktime", "Chronos", "TimesFM", "Moirai", "TTM", "TiRex", "Docker Hub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tserve-open-source-inference-server-for-time-series-foundation-models", "markdown": "https://wpnews.pro/news/tserve-open-source-inference-server-for-time-series-foundation-models.md", "text": "https://wpnews.pro/news/tserve-open-source-inference-server-for-time-series-foundation-models.txt", "jsonld": "https://wpnews.pro/news/tserve-open-source-inference-server-for-time-series-foundation-models.jsonld"}}