# Google's LangExtract on a Custom OpenAI-Compatible API: Model IDs That Route to Ollama, Strict Schemas and Silent Chunks > Source: > Published: 2026-10-19 07:38:16+00:00 LangExtract is Google's Python library for pulling structured entities out of long text, with every extraction mapped back to its character position in the source. You describe the task, give one or two examples, and it chunks the document, sends each chunk to a model, parses the JSON and aligns the results. Gemini is the default, but it has an OpenAI provider built on the official OpenAI SDK, so it can run against vLLM, LiteLLM or a hosted gateway. Getting it to use that provider with a non-OpenAI model name is the first surprise. I tested langextract 1.7.1 (openai 3.35.1, Python 3.13) against a small local mock server that logged every request and could return 400, 429, 503, fenced JSON or truncated output on demand. Everything below is what went over the wire. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is `https://apiclaw.biz/v1`; replace `YOUR_MODEL_ID` with an exact ID from your provider's model list. The obvious call is: ``` python import langextract as lx result = lx.extract( text_or_documents=text, prompt_description=prompt, examples=examples, model_id="qwen3-coder-plus", api_key="your-key", model_url="https://apiclaw.biz/v1", ) ``` LangExtract routes on regex patterns over `model_id`. Only names starting with `gpt-3.5`, `gpt-4`, `gpt-5` or `o1` to `o9` go to the OpenAI provider. Names starting with `qwen`, `deepseek`, `llama`, `mistral`, `gemma`, `phi` and `gpt-oss`, among others, go to the **Ollama** provider. Here is what happened in my tests: `qwen3-coder-plus` and `gpt-oss-120b` went to the Ollama client, which POSTed to `/v1/api/generate` on my base URL and failed with `Can't find Ollama qwen3-coder-plus. Try: ollama run qwen3-coder-plus`.` plain-model`, failed with `No provider registered for model_id='plain-model'`.` gpt-4o-mini` with `model_url` worked and hit `/v1/chat/completions`. So the code above only works if your gateway's model happens to be named like an OpenAI model. ``` python import os import langextract as lx config = lx.factory.ModelConfig( model_id="YOUR_MODEL_ID", provider="OpenAILanguageModel", # "openai" also worked provider_kwargs={ "api_key": os.environ["APICLAW_API_KEY"], "base_url": "https://apiclaw.biz/v1", "temperature": 0.0, "max_output_tokens": 2000, "max_workers": 4, }, ) result = lx.extract( text_or_documents=text, prompt_description="Extract every person mentioned, with their role. Use exact text.", examples=examples, config=config, show_progress=False, ) for e in result.extractions: print(e.extraction_class, e.extraction_text, e.attributes, e.char_interval) ``` That sent `POST /v1/chat/completions` for every model name I tried. Two details: `api_key` explicitly.`OPENAI_API_KEY` from the environment when the model ID contains `gpt`. With an explicit provider and a name like `API key not provided.` even though `base_url` can come from the environment.`OPENAI_BASE_URL`. Keep `/v1` on the end either way. You'll also see a warning: `With 'config', schema constraints are still applied via examples.` That matters for the next section. Every chunk became one request with: `You are a helpful assistant that responds in JSON format.` `Q:`/` A:` pairs with the expected JSON, and the chunk text,`n: 1`,` response_format` of type `json_schema`, named `langextract_extractions`, with `strict: true`, built from your examples. By default there was **no `temperature` and no `max_tokens`**, so the server's defaults applied. Setting `temperature` and `max_output_tokens` in `provider_kwargs` sent `temperature: 0.0` and `max_tokens: 2000`. For extraction I'd always set both: a low temperature for consistency and an output cap that fits a long list of entities. Strict `json_schema` output is not universal. Some models and some OpenAI-compatible servers support only `json_object`, or nothing. My mock returned a 400 for `json_schema`, and the whole `extract()` call raised `InferenceRuntimeError: OpenAI API error: Error code: 400`. ``` result = lx.extract(..., config=config, use_schema_constraints=False) ``` With `use_schema_constraints=False` the request switched to `response_format: {"type": "json_object"}` and succeeded. The examples in the prompt still show the model the expected shape. This is the setting I'd change first. By default LangExtract suppresses parse errors per chunk. Each of these runs printed one log line and returned **zero extractions with no exception**: `Here is the result:` followed by a fenced json block). Under `json_object` mode LangExtract expects raw JSON, so the chunk was omitted: `Chunk omitted from results (parse error): Failed to parse JSON content`.` finish_reason: "length"`. The truncated JSON failed to parse and the chunk was dropped the same way. On a 40-chunk document, a few dropped chunks look like the model just didn't find anything there. Two fixes: ``` result = lx.extract( ..., config=config, use_schema_constraints=False, fence_output=True, # accept fenced json replies resolver_params={"suppress_parse_errors": False}, # fail loudly instead ) ``` `fence_output=True` parsed the fenced reply correctly. `suppress_parse_errors: False` turned the silent drop into a `ResolverParsingError`, which you can catch and retry. A reply that started with a `...` block parsed fine in my test, so reasoning models weren't a problem there. I ran a 5,320-character document with the defaults: | Setting | Requests | Max in flight | |---|---|---| | defaults ( `max_char_buffer=1000` ) | 6 | 6 | | `max_workers: 3` in`provider_kwargs` | 6 | 3 | | `extraction_passes=3` | 18 | 6 | | `max_char_buffer=4000` | 2 | 2 | Each request repeats your prompt description and examples, so with small chunks the fixed part can be a large share of every request. `max_workers` defaults to 10, which means a long document can fire 10 requests at once. On a plan with a per-minute or concurrency limit, set it lower. `extraction_passes` runs the whole document again per pass, one pass after another, so three passes triple both requests and time. A larger `max_char_buffer` cuts request count, but long chunks give the model more to track, so test recall on your own documents before raising it a lot. A server that always returned 503 got three requests (the SDK's default two retries) and then `extract()` raised `InferenceRuntimeError`. A 429 with `Retry-After: 2` was honoured: the SDK waited and the third request succeeded after about 4.6 seconds. LangExtract adds no retry loop of its own, so for big batch jobs wrap the call in your own retry, and remember that a failure in one chunk fails the whole document. `ModelConfig(provider="OpenAILanguageModel")` unless your model ID starts with `gpt-` or `o1` to `o9`.` provider_kwargs`; the env key is only read for `temperature` and `max_output_tokens` yourself.`use_schema_constraints=False` if your server rejects `suppress_parse_errors: False` (and `fence_output=True` for chatty models) so dropped chunks don't go unnoticed.`max_workers` to fit your rate limit, and count `extraction_passes` as a multiplier. *Disclosure: I build [APIClaw](https://apiclaw.biz), the OpenAI-compatible gateway in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.*