{"slug": "integrate-llm-into-your-python-runtime", "title": "Integrate LLM into your Python runtime", "summary": "MiskaKan released thinair, a Python library that integrates large language models into the Python runtime by allowing developers to create probabilistic objects, where undefined attributes and methods are filled by an LLM with an associated confidence score. The library's core is a single class, Thing, which represents a value with a probability, and it requires only one inference call per typed operation, with all other operations free. This enables developers to code certain parts of their program while leaving blanks for the model to imagine, with probabilities attached to each generated value.", "body_md": "Probabilistic Python objects. Invent attributes, methods, anything out of thin air; an LLM of your choice (local or hosted) fills in the blanks, with confidence attached.\n\n**Code the certain, imagine the rest.**\n\nThe entire public surface is one class, and the whole idea fits in one line: a `Thing`\n\nis **a value with a probability**.\n\n```\ncolor = Thing(\"rusty red\", confidence=0.4)   # the value the imagined read\n                                             # in the demo handed back\n+color      # 'rusty red': the value         (free, no inference)\n~color      # 0.4: the probability           (free, no inference)\n```\n\nThe interface is Python itself. There are no prompt strings, no message arrays, and no output parsing: you write ordinary classes, attributes, and method calls, and the model's capabilities are available wherever you left a blank, always priced with a probability.\n\nEverything you write yourself (code, bare values, your own words) is certain: probability 1.0, and the model can never touch it. Everything the model fills in (a field nobody defined, a method nobody wrote) comes back as a `Thing`\n\nthat reports how sure it is. The model draws its answers from one axiom: **an object is a story, and every interaction is a continuation of it.** Everything else follows from it; see [SPEC.md](/MiskaKan/thinair/blob/main/SPEC.md) for the full spec.\n\nThree operators cover the surface, and only `@ <type>`\n\ncosts an inference call:\n\n`+thing` |\nthe value | free |\n`~thing` |\nthe probability | free |\n`thing @ int` |\nshape it as a type or schema | one call |\n`thing @ 0.9` |\nconfidence gate | free |\n\nNo class is required. Anything lifts into Thing space: a description in words, a bare value with your own doubt attached, or a whole frozen object (`blob @ Car`\n\n, below). The simplest is words alone, and your own words count as certain until inference touches them:\n\n``` python\nfrom thinair import Thing\n\nspider = Thing(\"the number of legs on a spider\")\n\n+spider                  # \"the number of legs on a spider\": your own\n                         # words back; free, no inference\n~spider                  # 1.0: your words are certain\n\nlegs = spider @ int      # collapsing happens through typing: one inference\n                         # call, and legs is now a Thing carrying 8\n+legs                    # 8: the value, a real int\n~legs                    # 0.99: the probability, a bare float\n```\n\nRequirements chain, and an unmet one drops the value but keeps the probability, so failures explain themselves:\n\n```\n+(spider @ int @ 0.8)          # 8: typed and vouched for at p >= 0.8\n\ncar = Thing(\"a rusty 1990 Toyota Hilux\")   # attributes you never defined\nguess = car.price_eur @ float @ 0.9        # are imagined on first read,\n                                           # so this is a typed, gated guess\n+guess                         # None: did not clear the bar\n~guess                         # 0.1: the probability survives to say why\nif guess: ...                  # failed Things are falsy: gate whole branches\n```\n\nThe type operand scales from primitives through schemas to real classes:\n\n```\nmovie = Thing(\"the Ridley Scott movie with the xenomorph\")\n\n+(movie @ {\"title\": str, \"year\": int})   # {'title': 'Alien', 'year': 1979},\n                                         # a schema-guaranteed dict\n\n@dataclass\nclass Record:\n    title: str\n    year: int\n\n+(movie @ Record)        # Record(title='Alien', year=1979): the model\n                         # imagines the kwargs, your constructor builds it\n```\n\nYou can assert your own doubt, and comparisons happen in Thing space: they are judgments, not byte comparisons:\n\n```\nprice = Thing(19_990, confidence=0.4)    # lift a belief into Thing space\n+(price @ 0.5)                           # None: you said so yourself\n\nThing(\"a car\") < Thing(\"a cat\")          # False (p 0.75), an imagined judgment\n+car.price < 20_000                      # take the value out first for plain,\n                                         # free Python semantics\n```\n\nSubclass `Thing`\n\nand write the parts you are sure of. Written code and bare values are the certain skeleton: they run as ordinary Python, cost nothing, and **the model can never touch them**. Everything else is imagined on demand:\n\n```\nclass Car(Thing):\n    \"\"\"A road vehicle.\"\"\"\n    wheels = 4                      # certain by definition\n\n    def horn(self):                 # real code: runs in CPython,\n        return \"beep\"               # inference is never consulted\n\ncar = Car(\"a rusty 1990 Toyota Hilux, engine coughs, \"\n          \"radio stuck on a Finnish schlager station\")\n\ncar.wheels               # 4: a bare int; no inference ran, nothing billed\ncar.horn()               # \"beep\": real code, really executed\n+car.color               # \"rusty red\": imagined where the class was silent\n~car.color               # 0.4: honestly unsure about it\n\ncar.owner = \"Miska\"                  # bare assignment: authoritative and\n                                     # locked; no plan may overwrite it\ncar.mood = Thing(\"unknown so far\")   # a slot the model may manage\n```\n\nProvenance is permission: bare values belong to the programmer, `Thing`\n\nvalues belong to the imagination.\n\nAny method you never wrote is imagined at call time, and it acts: it reads state, writes state (with confidence, journaled), and calls your real methods, which actually execute. Results are Things, so calls chain:\n\n```\nproblems = car.list_your_problems(returns=[str])\n+problems                # ['Engine is coughing', 'Radio is stuck on a\n                         #  Finnish schlager station', 'High rust level']\n\ncar.repair_engine()      # no such method: a plan is imagined and acted\n                         # out; the story now contains the repair\n\ncar.diagnose().severity_of_worst_issue()   # results are Things: chain\n                                           # imagined calls on imagined calls\n```\n\nGuard the branches that matter: inside `with Thing.require(0.9):`\n\nany resolution below 0.9 raises `Thing.LowConfidence`\n\ninstead of flowing:\n\n``` js\nwith Thing.require(0.9):\n    if car.can_drive():      # p 0.93: clears the bar, the branch is trusted\n        plan_road_trip(car)\n    car.vin_number           # p 0.02: a guess this wild now raises\n                             # Thing.LowConfidence instead of flowing onward\n```\n\nThere is no chatbot framework, and `chat`\n\nis not a built-in: nobody wrote it, so it is imagined at call time like any other missing method. The object can hold a conversation because a conversation is just more story:\n\n```\nwhile True:\n    print(car.chat(input(\"> \")))   # chat appears out of thin air too\n> Why did you break down on me this morning?\n*cough cough* ... beep. Look, I'm a 1990 Toyota Hilux. I'm rusty, I'm tired,\nand my radio is stuck on Finnish schlager. I didn't break down on purpose;\nI just... gave up. The engine was coughing all morning. Do you have any idea\nhow hard it is to start when you're this old and this red?\n```\n\nEvery turn is journaled, so the car remembers what you said. Objects can also size each other up the same way, by handing Things to a Thing:\n\n```\ncustomer = Thing(\"a retired couple with a caravan and two dogs, modest budget\")\n\npick = customer.prefers(sedan, suv, roadster,\n                        returns={\"choice\": str, \"why\": str})\npick.choice      # \"a full-size SUV: seven seats, tow hook, thirsty\"\npick.why         # \"The caravan requires a tow hook, which only the SUV\n                 #  has, and it provides necessary space for the two dogs.\"\n```\n\n`car.__getstate__()`\n\nis a JSON-able document: description, state, story, flags. No code, no weights, no client. `blob @ Car`\n\nrestores it with written methods reattached, and `pickle`\n\nworks out of the box. `__source__`\n\nrenders any object as the class it currently is:\n\n```\nprint(car.__source__)\nclass Car(Thing):\n    \"\"\"\n    A road vehicle.\n\n    a rusty 1990 Toyota Hilux, engine coughs, radio stuck on a Finnish schlager station\n    \"\"\"\n\n    wheels = 4\n\n    owner = 'Miska'  # written (p = 1.0)\n    color = 'brown'  # imagined (p = 0.10)\n    top_speed_kmh = 130  # imagined (p = 0.10)\n\n    def horn(self):\n        return \"beep\"\n```\n\n`__story__`\n\nis the other lens: the full journal of every event, answer, and imagined step, in order. Consistency and provenance come for free.\n\n```\npip install thinair\n```\n\nAvailable on [PyPI](https://pypi.org/project/thinair/). No dependencies, one file, stdlib only. Point it at any OpenAI-compatible endpoint (defaults target a local server):\n\n```\nexport THINAIR_BASE_URL=\"http://127.0.0.1:8000/v1\"   # default\nexport THINAIR_API_KEY=\"1234\"\nexport THINAIR_MODEL=\"Qwen3.6-35B-A3B-oQ8-mtp\"\nexport THINAIR_MAX_TOKENS=65536                      # total budget per completion, thinking included\n```\n\nRequests ask for the server's JSON output mode (`response_format: json_object`\n\n) and quietly fall back to freeform if the server doesn't support it. Inference climbs a two-tier ladder: questions are first answered single-shot with thinking disabled (and, where the server supports it, decoding constrained to the reply schema); only a question that stumbles (low confidence, a refusal-shaped null, unparseable output) earns the thinking tier. There the stream is supervised as it arrives and cut the moment it starts repeating itself verbatim. A reply the model kept rehearsing inside a loop is harvested as the answer, an answer cut mid-way gets a completion grant sized from the draft rather than the whole budget, and a model that only keeps thinking eventually gets a clear budget error, so runaway generation can never capture the budget.\n\nOr in code: `Thing.defaults(model=\"...\", base_url=\"...\", api_key=\"...\")`\n\n. A URL, a provider object with `complete(messages) -> text`\n\n, or a bare callable all work per instance too: `Thing(\"a car\", model=...)`\n\n.\n\nTo see what is happening underneath, wrap any block in `with Thing.debug():`\n\nand every prompt and raw completion is dumped to stderr, labeled by operation (`read`\n\n, `imagined`\n\n, `judge`\n\n, `collapse`\n\n, `condense`\n\n). `THINAIR_DEBUG=1`\n\nturns it on globally.\n\nThen:\n\n```\npython car_chat.py     # talk to a rusty Hilux; chat is imagined, not written\n```\n\nAn experiment. Every unresolved attribute costs an inference call, and answers are only as good as the model behind them.\n\nMIT", "url": "https://wpnews.pro/news/integrate-llm-into-your-python-runtime", "canonical_source": "https://github.com/MiskaKan/thinair", "published_at": "2026-08-01 12:31:52+00:00", "updated_at": "2026-08-01 12:52:37.858019+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "generative-ai"], "entities": ["MiskaKan", "thinair", "Thing", "Python"], "alternates": {"html": "https://wpnews.pro/news/integrate-llm-into-your-python-runtime", "markdown": "https://wpnews.pro/news/integrate-llm-into-your-python-runtime.md", "text": "https://wpnews.pro/news/integrate-llm-into-your-python-runtime.txt", "jsonld": "https://wpnews.pro/news/integrate-llm-into-your-python-runtime.jsonld"}}