# Superlinked Inference Engine (SIE): Run 100+ AI Models on One Server > Source: > Published: 2026-08-24 00:00:00+00:00 # Superlinked Inference Engine (SIE): Run 100+ AI Models on One Server Superlinked Inference Engine (SIE) is an open-source server that runs embedding, reranking, extraction, and generation models locally on one GPU. ## What is the Superlinked Inference Engine? The Superlinked Inference Engine (SIE) is an open-source, Apache 2 licensed server that serves more than 100 AI models, embedding models, rerankers, extractors, and text generation models, from a single running instance. Instead of standing up a separate server for every model type your AI agent needs, you run one SIE container and call whatever model you need through the same lightweight client. It’s built to run on your own GPU hardware, and Superlinked also offers a managed cloud version of the same engine for teams that don’t want to operate GPUs themselves. ## TL;DR **SIE consolidates infrastructure** by serving embeddings, reranking, entity extraction, and text generation from one server instead of four separate deployments.- The project is **open source under Apache 2** and hosted on GitHub, where it has passed 2,800 stars and continues to add models to its catalog. - Models are **loaded on demand**, meaning the server starts instantly and only downloads and consumes GPU memory for a model the first time you actually call it. **Switching models takes one line of code**, changing the model name string in the client call, with no reinstall, no restart, and no new server setup.- The same client library handles four distinct tasks through different method calls: **encode for embeddings, score for reranking, and extract for entity extraction**, plus a separate generate call for text generation. **Text generation runs on a different backend**(deployed via Docker in the demo) than the embedding/reranking/extraction server, but uses the same model catalog and SDK.- Teams that don’t want to manage GPU infrastructure can use **Superlinked’s managed cloud**, which runs the identical engine and code, just hosted. ## Plans first. *Then code.* Remy writes the spec, manages the build, and ships the app. ## Why does this matter for people building AI agents? Anyone building an AI agent that needs more than a chat model quickly runs into a fragmentation problem. A typical agent pipeline might need an embedding model for semantic search, a reranker to sort retrieved results by relevance, an extraction model to pull structured entities out of text, and a generation model to produce the final answer. Each of those has traditionally meant its own server, its own dependencies, and its own GPU allocation. That leaves two common paths. One is self-hosting everything, which turns into a pile of separate services that all need setup and babysitting. The other is renting each capability from managed APIs, which means paying per token across multiple vendors and sending your data to several different clouds at once. SIE targets that gap directly: one engine, one API surface, running locally or in a single managed cloud account. ## How do you install and run SIE locally? The installation shown in the demo uses a standard Python virtual environment (created with `uv` , though `conda` works too) and a single install command. Once installed, one command starts the server. On launch, SIE reported finding 151 models in its registry, all without downloading anything yet. GPU memory usage stayed at zero until a model was actually requested. A readiness check endpoint confirms the server is up before you start sending requests. This “instant start, lazy load” behavior is a meaningful design choice. Instead of pre-downloading every one of the 100+ supported models (which would be impractical), SIE fetches and loads a model into GPU memory only the first time a client calls it. After that, the model stays resident for subsequent calls, and your GPU footprint reflects only the models you’re actually using. ## How does the client SDK work across different tasks? Once the server is running, a separate lightweight SDK connects to it. The pattern stays consistent across every task: create a client pointed at the local (or remote) SIE server, then call a method naming the model and the input. **Embeddings**: calling`encode` with a model like`all-MiniLM` and a string of text returns a vector, in the demo a 384-dimensional embedding, the numeric representation used for semantic search and retrieval-augmented generation.**Reranking**: calling`score` with a query and a set of candidate answers returns those candidates sorted by relevance. In the demonstration, a machine-learning-related answer correctly outranked an unrelated weather answer for a machine-learning query.**Extraction**: calling`extract` with a sentence and a list of target labels (for example, “person” and “organization”) pulls out matching entities without any fine-tuning. Given a sentence mentioning Tim Cook and Apple, the model correctly tagged Tim Cook as a person and Apple as an organization.**Generation**: a separate`generate` call, backed by a different serving engine deployed through Docker, runs a small open-source language model locally and returns generated text based on a prompt. ## One coffee. One working app. You bring the idea. Remy manages the project. Across all four tasks, the only thing that changes is the method name and arguments. There’s no separate SDK to learn for reranking versus extraction versus generation. ## Is switching models actually as easy as it looks? Yes, based on the demonstrated workflow. Swapping from a smaller embedding model like `all-MiniLM` to a larger multilingual one like `BGE-M3` required changing a single model name string in the script, with no server restart and no reinstallation. Running the script again triggered an automatic download of the new model, followed by loading it and returning embeddings in the new model’s vector space. This matters for anyone iterating on an agent’s retrieval pipeline. Testing whether a bigger or more specialized embedding model improves retrieval quality doesn’t require tearing down and rebuilding a serving stack. It’s a one-line change and a re-run. ## Is SIE worth using over managed embedding and generation APIs? The tradeoff comes down to control versus convenience. Running SIE locally means your data never leaves your own hardware, you’re not paying per-token API fees, and you can swap between 100+ models without juggling multiple vendor accounts and pricing plans. The cost is that you need a GPU and you’re responsible for the server itself, even though SIE keeps that overhead low with on-demand model loading and a single install command. For teams that want the same model catalog and API without operating GPU infrastructure, Superlinked’s managed cloud runs the identical engine and code, just hosted on their infrastructure. That gives you a middle path: the same one-server, one-API model access, without racking or renting your own GPU. For solo developers, small teams, or anyone prototyping an agent that touches embeddings, reranking, extraction, and generation, self-hosting SIE on a single GPU box removes a meaningful chunk of infrastructure complexity compared to stitching together separate serving frameworks for each model type. ## Frequently Asked Questions ### What does SIE stand for? SIE stands for Superlinked Inference Engine, an open-source model serving system built by Superlinked. ### What types of models does SIE support? It supports embedding models for semantic search, rerankers for scoring and sorting results, extraction models for pulling structured entities from text, and text generation models, all from one catalog of 100+ models. ### Do I need to download all 100+ models before using SIE? No. The server starts without downloading anything. Each model downloads and loads into GPU memory only the first time you actually call it, which keeps startup fast and GPU usage tied to what you’re actively using. ### Is SIE free to use? The core engine is open source under the Apache 2 license and free to self-host on your own hardware. Superlinked also sells a managed cloud version of the same engine for teams that prefer not to manage their own GPUs. ### Does text generation use the same server as embeddings and reranking? Not exactly. In the demonstrated setup, text generation runs through a different backend deployed via Docker, separate from the embedding, reranking, and extraction server, but it uses the same model catalog and the same client SDK.