Google's LangExtract on a Custom OpenAI-Compatible API: Model IDs That Route to Ollama, Strict Schemas and Silent Chunks

A developer tested Google's LangExtract 1.7.1 against a custom OpenAI-compatible gateway and found that the library routes models by regex on model_id, sending names like qwen and gpt-oss to its Ollama provider and failing, while only gpt-3.5/gpt-4/gpt-5/o1-o9 names reach the OpenAI provider. Passing an explicit ModelConfig with provider="OpenAILanguageModel" forced all requests to POST /v1/chat/completions, and the tests showed LangExtract sends strict json_schema response_format built from examples but no temperature or max_tokens unless set in provider_kwargs.

LangExtract is Google's Python library for pulling structured entities out of long text, with every extraction mapped back to its character position in the source. You describe the task, give one or two examples, and it chunks the document, sends each chunk to a model, parses the JSON and aligns the results. Gemini is the default, but it has an OpenAI provider built on the official OpenAI SDK, so it can run against vLLM, LiteLLM or a hosted gateway. Getting it to use that provider with a non-OpenAI model name is the first surprise. I tested langextract 1.7.1 openai 3.35.1, Python 3.13 against a small local mock server that logged every request and could return 400, 429, 503, fenced JSON or truncated output on demand. Everything below is what went over the wire. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is https://apiclaw.biz/v1 ; replace YOUR MODEL ID with an exact ID from your provider's model list. The obvious call is: python import langextract as lx result = lx.extract text or documents=text, prompt description=prompt, examples=examples, model id="qwen3-coder-plus", api key="your-key", model url="https://apiclaw.biz/v1", LangExtract routes on regex patterns over model id . Only names starting with gpt-3.5 , gpt-4 , gpt-5 or o1 to o9 go to the OpenAI provider. Names starting with qwen , deepseek , llama , mistral , gemma , phi and gpt-oss , among others, go to the Ollama provider. Here is what happened in my tests: qwen3-coder-plus and gpt-oss-120b went to the Ollama client, which POSTed to /v1/api/generate on my base URL and failed with Can't find Ollama qwen3-coder-plus. Try: ollama run qwen3-coder-plus . plain-model , failed with No provider registered for model id='plain-model' . gpt-4o-mini with model url worked and hit /v1/chat/completions . So the code above only works if your gateway's model happens to be named like an OpenAI model. python import os import langextract as lx config = lx.factory.ModelConfig model id="YOUR MODEL ID", provider="OpenAILanguageModel", "openai" also worked provider kwargs={ "api key": os.environ "APICLAW API KEY" , "base url": "https://apiclaw.biz/v1", "temperature": 0.0, "max output tokens": 2000, "max workers": 4, }, result = lx.extract text or documents=text, prompt description="Extract every person mentioned, with their role. Use exact text.", examples=examples, config=config, show progress=False, for e in result.extractions: print e.extraction class, e.extraction text, e.attributes, e.char interval That sent POST /v1/chat/completions for every model name I tried. Two details: api key explicitly. OPENAI API KEY from the environment when the model ID contains gpt . With an explicit provider and a name like API key not provided. even though base url can come from the environment. OPENAI BASE URL . Keep /v1 on the end either way. You'll also see a warning: With 'config', schema constraints are still applied via examples. That matters for the next section. Every chunk became one request with: You are a helpful assistant that responds in JSON format. Q: / A: pairs with the expected JSON, and the chunk text, n: 1 , response format of type json schema , named langextract extractions , with strict: true , built from your examples. By default there was no temperature and no max tokens , so the server's defaults applied. Setting temperature and max output tokens in provider kwargs sent temperature: 0.0 and max tokens: 2000 . For extraction I'd always set both: a low temperature for consistency and an output cap that fits a long list of entities. Strict json schema output is not universal. Some models and some OpenAI-compatible servers support only json object , or nothing. My mock returned a 400 for json schema , and the whole extract call raised InferenceRuntimeError: OpenAI API error: Error code: 400 . result = lx.extract ..., config=config, use schema constraints=False With use schema constraints=False the request switched to response format: {"type": "json object"} and succeeded. The examples in the prompt still show the model the expected shape. This is the setting I'd change first. By default LangExtract suppresses parse errors per chunk. Each of these runs printed one log line and returned zero extractions with no exception : Here is the result: followed by a fenced json block . Under json object mode LangExtract expects raw JSON, so the chunk was omitted: Chunk omitted from results parse error : Failed to parse JSON content . finish reason: "length" . The truncated JSON failed to parse and the chunk was dropped the same way. On a 40-chunk document, a few dropped chunks look like the model just didn't find anything there. Two fixes: result = lx.extract ..., config=config, use schema constraints=False, fence output=True, accept fenced json replies resolver params={"suppress parse errors": False}, fail loudly instead fence output=True parsed the fenced reply correctly. suppress parse errors: False turned the silent drop into a ResolverParsingError , which you can catch and retry. A reply that started with a <think ...</think block parsed fine in my test, so reasoning models weren't a problem there. I ran a 5,320-character document with the defaults: | Setting | Requests | Max in flight | |---|---|---| | defaults max char buffer=1000 | 6 | 6 | | max workers: 3 in provider kwargs | 6 | 3 | | extraction passes=3 | 18 | 6 | | max char buffer=4000 | 2 | 2 | Each request repeats your prompt description and examples, so with small chunks the fixed part can be a large share of every request. max workers defaults to 10, which means a long document can fire 10 requests at once. On a plan with a per-minute or concurrency limit, set it lower. extraction passes runs the whole document again per pass, one pass after another, so three passes triple both requests and time. A larger max char buffer cuts request count, but long chunks give the model more to track, so test recall on your own documents before raising it a lot. A server that always returned 503 got three requests the SDK's default two retries and then extract raised InferenceRuntimeError . A 429 with Retry-After: 2 was honoured: the SDK waited and the third request succeeded after about 4.6 seconds. LangExtract adds no retry loop of its own, so for big batch jobs wrap the call in your own retry, and remember that a failure in one chunk fails the whole document. ModelConfig provider="OpenAILanguageModel" unless your model ID starts with gpt- or o1 to o9 . provider kwargs ; the env key is only read for temperature and max output tokens yourself. use schema constraints=False if your server rejects suppress parse errors: False and fence output=True for chatty models so dropped chunks don't go unnoticed. max workers to fit your rate limit, and count extraction passes as a multiplier. Disclosure: I build APIClaw https://apiclaw.biz , the OpenAI-compatible gateway in these examples flat monthly plans; crypto on the site, other payment methods through support . Everything above applies to any compatible endpoint.