{"slug": "hql-run-typed-queries-directly-inside-markdown", "title": "HQL – Run typed queries directly inside Markdown", "summary": "HQL (Hyper Query Language), a statically typed functional query language for data, knowledge graphs and AI workflows, has been released with documentation at hql.hypermarkdown.org. HQL combines pipelines, pattern matching, first-class Stream<T> types and algebraic types such as Option<T> with executable Markdown, letting users run HQL cells inside HyperMarkDown documents and directly in VS Code, and it also runs from the command line to process JSON, YAML, CSV and text in the role normally filled by jq. The language is positioned as AI-native, expressing retrieval, context construction and agent workflows as transparent data pipelines.", "body_md": "**HQL — Hyper Query Language — is a statically typed language for querying, transforming, connecting and computing over data, knowledge and AI workflows.**\n\nHQL combines functional expressions, data pipelines, streams, structured documents, semantic retrieval and knowledge graphs in one language.\n\n``` js\nbirds =\n    cards\n    | filter(c => c.metadata.category == \"bird\")\n    | sort(by = c => c.title)\n\nbirds\n| map(c => c.title)\n```\n\nThe same language can query a knowledge vault, transform JSON from the command line, build an ETL pipeline, traverse a graph, execute queries inside Markdown, or assemble context for an AI agent.\n\n- **Typed functional query language** — expressions, functions, pattern matching, collections and pipelines, checked before execution.\n- **Dataflow language** — compose transformations over finite data, streams and batches.\n- **Data integration and ETL** — connect files, standard streams, databases, queues and other sources and targets through one query model.\n- **AI-native language** — retrieval, context construction and agent workflows are expressed as transparent data pipelines.\n- **Executable Markdown** — run HQL cells inside HyperMarkDown documents and directly in VS Code, turning Markdown into live queries, reports, views and AI-enabled knowledge artifacts.\n- **Knowledge graph language** — retrieve, traverse, expand and combine structured knowledge and relationships.\n- **Command-line data tool** — process JSON, YAML, CSV, text and streams in the role normally filled by tools such as`jq` , while retaining types, functions and reusable programs.\n- **Streaming by design** —`Stream<T>` is a first-class type; pipelines can process values as they arrive instead of materializing an entire dataset.\n- **One language across execution surfaces** — use HQL from the CLI,`.hql` programs, executable documents and interactive editor cells.\n\nDocumentation: [hql.hypermarkdown.org](https://hql.hypermarkdown.org/)\n\nHQL is intentionally difficult to reduce to “a query language for X”.\n\nIt is better understood through several complementary views.\n\nHQL programs are expressions.\n\nValues flow through pipelines, while functions, bindings, collections and pattern matching provide the tools needed for reusable computation.\n\n``` js\nfn pages(b: Data) : Int =\n    match b.pages {\n        some(Int n) => n,\n        _ => 0\n    }\n\nlarge_books =\n    books\n    | filter(b => pages(b) > 500)\n    | sort(by = b => b.title)\n```\n\nTypes are checked before execution.\n\nThe type system includes collection types such as:\n\n```\nSeq<T>\nList<T>\nSet<T>\nMap<K, V>\n```\n\nfirst-class streams:\n\n```\nStream<T>\n```\n\nand algebraic value types such as:\n\n```\nOption<T>\n```\n\nalongside `Data` for dynamically structured values.\n\nPipelines are ordinary typed expressions rather than a separate query syntax:\n\n``` js\ncards\n| filter(c => c.metadata.status == \"open\")\n| map(c => c.title)\n| sort\n```\n\nThis makes the same language suitable for small queries and larger programs.\n\nHQL models computation as data moving through transformations.\n\n``` js\nevents\n| filter(e => e.kind == \"purchase\")\n| map(e => {\n    customer: e.customer,\n    amount: e.amount,\n})\n| collect(100)\n| process_batch\n```\n\nA pipeline can operate over finite collections or over streams that continue producing values.\n\n```\nStream<Event>\n```\n\ncan become batches:\n\n```\nevents\n| collect(100)\n```\n\nproducing:\n\n```\nStream<Seq<Event>>\n```\n\nwhile:\n\n```\nevents\n| collect\n```\n\nmaterializes a completed stream as:\n\n```\nSeq<Event>\n```\n\nand:\n\n```\nitems\n| emit\n```\n\nturns a finite collection back into a stream.\n\nBranches are part of the same model:\n\n``` js\nevents\n| tee errors =\n    filter(e => e.level == \"error\")\n| aggregate(...)\n```\n\nDataflow is therefore not an execution detail hidden underneath HQL.\n\nIt is part of how HQL programs are written.\n\nExternal data enters HQL through typed sources.\n\n```\nsource(stdin, Json)\n```\n\nFor example:\n\n``` js\ncurl https://example.org/events |\n    hql 'source(stdin, Json) | filter(e => e.active)'\n```\n\n`stdin` is a host-provided endpoint. It does not need to be configured.\n\nFiles use the same model:\n\n```\nbirds =\n    source(file(\"birds.json\"), Json)\n```\n\nas do configured sources:\n\n```\nwarehouse =\n    source(\"warehouse\")\n```\n\nA source can stand directly in a pipeline:\n\n``` js\nsource(\"warehouse\")\n| filter(r => r.created_at > cutoff)\n| map(r => r.customer_id)\n```\n\nOutput follows the complementary model.\n\nA completed value can be written to a target:\n\n```\nreport\n| target(file(\"report.yaml\"), Yaml)\n```\n\nwhile a stream can be consumed incrementally by a sink:\n\n```\nevents\n| sink(stdout, Json)\n```\n\nStandard endpoints such as `stdout` and `stderr` are host-provided just like `stdin`:\n\n```\nresult\n| target(stdout, Yaml)\n\nerrors\n| sink(stderr, Json)\n```\n\nThey do not need entries in `hql.toml`.\n\nThe distinction between whole values, streams and batches lives in the type system:\n\n```\nEvent\nSeq<Event>\nStream<Event>\nStream<Seq<Event>>\n```\n\nnot in separate I/O APIs.\n\nFor example:\n\n```\nevents\n| collect(100)\n| sink(stdout, Json)\n```\n\nstreams batches:\n\n```\nStream<Seq<Event>>\n```\n\nwhile:\n\n```\nevents\n| collect\n| target(file(\"events.json\"), Json)\n```\n\nmaterializes the complete stream and writes one finite result.\n\nFormats remain orthogonal:\n\n```\nJson\nYaml\nCsv\nText\nMarkdown\nHql\nD2\nMermaid\n```\n\nThe format describes representation.\n\nThe HQL type describes the shape of the data.\n\nSources, transformations, streams, batches and targets all use the same expression model.\n\nThat makes HQL suitable for ETL and data integration without introducing a separate workflow language around the query language.\n\n``` js\nsource(\"orders\")\n| filter(o => o.status == \"completed\")\n| map(o => {\n    customer: o.customer_id,\n    total: o.total,\n})\n| collect(500)\n| sink(\"analytics\")\n```\n\nA pipeline can combine multiple sources:\n\n```\ncustomers = source(\"customers\")\norders = source(\"orders\")\n\n...\n```\n\nor move between files and structured output:\n\n``` js\nsource(file(\"sales.csv\"), Csv)\n| filter(row => row.amount > 1000)\n| target(file(\"large-sales.json\"), Json)\n```\n\nIt can also mix live and finite processing:\n\n``` js\nsource(stdin, Json)\n| filter(e => e.level == \"error\")\n| tee archive =\n    sink(file(\"errors.jsonl\"), Json)\n| map(e => e.message)\n| sink(stderr, Text)\n```\n\nThe query is the dataflow.\n\nThere is no second orchestration syntax wrapped around it.\n\nStreams are first-class values.\n\n```\nStream<T>\n```\n\nrepresents values produced incrementally and may be bounded or unbounded.\n\nA stream can be transformed directly:\n\n``` js\nsource(stdin, Json)\n| filter(e => e.kind == \"event\")\n| map(e => e.payload)\n```\n\ncollected completely:\n\n```\nevents\n| collect\n```\n\nbatched:\n\n```\nevents\n| collect(100)\n```\n\nor created from a finite collection:\n\n```\nitems\n| emit\n```\n\nThis makes streaming a normal part of the type system rather than a separate execution mode.\n\nThe same operations compose over:\n\n```\nSeq<T>\nStream<T>\nStream<Seq<T>>\n```\n\nand preserve the distinction between finite data, individual stream elements and streaming batches.\n\nAI applications spend much of their complexity not in the model call itself, but in everything around it:\n\n- retrieving relevant information,\n- expanding context,\n- selecting evidence,\n- constructing prompts,\n- invoking agents,\n- evaluating results,\n- reacting to events,\n- recording outputs,\n- and feeding results into subsequent computation.\n\nThese are dataflow problems.\n\nHQL expresses them as dataflow.\n\n```\ncontext =\n    cards\n    | semantic(question)\n    | take(10)\n    | expand(depth = 1)\n\ncontext\n| ...\n```\n\nRetrieval is not a terminal operation.\n\nThe result remains ordinary typed data that can continue through filtering, graph traversal, transformation, aggregation and rendering.\n\n``` js\ncards\n| semantic(\"distributed execution\")\n| take(20)\n| filter(hit => hit.score > threshold)\n| expand(depth = 2)\n```\n\nThis makes HQL a natural language for AI harnesses: retrieval, context construction, prompts, agents and their outputs can be represented as inspectable programs instead of opaque framework callbacks.\n\nIn HyperMarkDown, agent definitions and instructions can themselves be documents.\n\nThat means the knowledge, the prompt and the logic assembling the prompt can all remain visible, editable and queryable.\n\nHQL can run directly inside Markdown and HyperMarkDown documents.\n\nA fenced HQL cell is both documentation and executable logic:\n\n``` js\n# Open architecture decisions\n\n``` hql\ncards\n| filter(c => c.metadata.kind == \"adr\")\n| filter(c => c.metadata.status == \"open\")\n| sort(by = c => c.title)\nThe same document can be rendered from the command line:\n\n```\nhql render architecture.hmd\n```\n\nor used interactively in VS Code through the HyperMarkDown extension:\n\n[HyperMarkDown VS Code extension](https://github.com/ewiger/hypermarkdown/tree/main/tools/hmd-vsc-ext)\n\nThe extension supports HQL cells directly in HyperMarkDown documents, making the document itself an interactive query surface.\n\nThis changes the role of Markdown.\n\nA document can become:\n\n- a live project index,\n- an architecture or ADR view,\n- a generated table or report,\n- a knowledge-base query,\n- a semantic-search surface,\n- a project dashboard,\n- an AI prompt,\n- a context-assembly document,\n- or an operational tool placed next to the prose that explains it.\n\nFor example:\n\n```\n# Services\n\nThe table below is derived from the project knowledge base.\n\n``` hql\ncards\n| filter(c => c.metadata.kind == \"service\")\n| sort(by = c => c.title)\n| map(c => {\n    service: c.title,\n    owner: c.metadata.owner,\n    status: c.metadata.status,\n})\nThe query remains visible next to the text it supports.\n\nThere is no separate dashboard definition, template language or application layer required to derive the view.\n\nThe Markdown document itself becomes the view, while HQL supplies its live data.\n\nThis gives HQL several equivalent execution surfaces:\n\n```\nCLI              hql '...'\nprogram          query.hql\ndocument         ``` hql ... ```\ninteractive doc  VS Code HQL cell\n```\n\nThe language remains the same across all of them.\n\nHyperMarkDown is therefore not merely a storage format for HQL.\n\nIt is also one of HQL's natural interactive hosts.\n\nHQL treats links and relationships as queryable structure.\n\nRetrieval can find an initial set of nodes:\n\n```\ncards\n| semantic(\"night hunting birds\")\n| take(3)\n```\n\nand graph traversal can expand from them:\n\n```\ncards\n| semantic(\"night hunting birds\")\n| take(3)\n| expand(depth = 1)\n```\n\nGraph operations compose with ordinary filtering, mapping, aggregation and rendering.\n\nThis allows HQL to move naturally between several forms of knowledge:\n\n```\ndocuments\n    ↓\nretrieval\n    ↓\nconcepts and relationships\n    ↓\ngraph traversal\n    ↓\nstructured result\n```\n\nA knowledge graph is therefore not a separate database API attached to the language.\n\nIt is another structure that participates in HQL pipelines.\n\nHQL provides retrieval as a first-class pipeline operation.\n\nLexical search:\n\n```\ncards\n| lexical(\"night hunting birds\")\n| take(3)\n```\n\nSemantic search:\n\n```\ncards\n| semantic(\"night hunting birds\")\n| take(3)\n```\n\nThe result can immediately participate in further computation:\n\n``` js\ncards\n| semantic(question)\n| take(20)\n| filter(hit => hit.score > threshold)\n| expand(depth = 2)\n```\n\nThis matters especially for retrieval-augmented AI workflows, where search is usually only the first stage of context construction.\n\nRetrieval results retain their provenance, so the result does not need to lose information about how it was obtained when it continues through a pipeline.\n\nHQL is useful without a vault or project.\n\nPretty-print JSON from standard input:\n\n```\ncat data.json |\n    hql 'source(stdin, Json)' --to json\n```\n\nFilter a JSON stream:\n\n``` js\ncat birds.jsonl |\n    hql '\n        source(stdin, Json)\n        | filter(b => b.count > 2)\n        | map(b => b.species)\n    '\n```\n\nConvert JSON to YAML:\n\n```\ncat config.json |\n    hql 'source(stdin, Json)' --to yaml\n```\n\nWork with files:\n\n``` js\nsource(file(\"birds.json\"), Json)\n| filter(b => b.count > 2)\n| target(file(\"popular-birds.yaml\"), Yaml)\n```\n\nOr move the query into a reusable program:\n\n```\nhql run report.hql\n```\n\nHQL occupies some of the same space as `jq`, but its scope is deliberately broader:\n\n```\njq-style structured data processing\n+ static types\n+ functions\n+ reusable programs\n+ streams\n+ batching\n+ multiple sources and targets\n+ ETL and dataflow\n+ knowledge graphs\n+ semantic retrieval\n+ executable Markdown\n+ AI workflows\n```\n\nThe goal is not to imitate the syntax of existing tools.\n\nIt is to provide one coherent language across these workloads.\n\nHQL treats data representations explicitly.\n\n```\nformat(result, Json)\nformat(result, Yaml)\nformat(result, Markdown)\n```\n\nPresentation types include:\n\n```\nText\nJson\nYaml\nCsv\nMarkdown\nHql\nD2\nMermaid\n```\n\nFormats describe encoding rather than execution shape.\n\nFor example:\n\n```\nitems : Seq<Item>\n\nitems\n| target(stdout, Json)\n```\n\nproduces one JSON array.\n\nBut:\n\n```\nitems\n| emit\n| sink(stdout, Json)\n```\n\nwrites items incrementally as individual JSON documents.\n\nLikewise:\n\n```\nevents\n| collect(100)\n| sink(stdout, Json)\n```\n\nwrites a stream of JSON arrays.\n\nThe difference comes from:\n\n```\nSeq<Item>\nStream<Item>\nStream<Seq<Item>>\n```\n\nnot from introducing separate formats such as a special JSON-lines data type.\n\nHyperMarkDown is a natural companion to HQL, but HQL is not limited to HyperMarkDown.\n\nA HyperMarkDown vault provides structured cards, metadata, links and knowledge relationships:\n\n``` js\ncards\n| filter(c => c.metadata.kind == \"adr\")\n| map(c => c.title)\n```\n\nThe same language can operate over other sources and can be used without a vault at all.\n\nThis distinction is intentional:\n\n**HyperMarkDown provides structured knowledge. HQL computes over it.**\n\nTogether they also provide a natural programmable-document environment: HyperMarkDown supplies the document structure, while HQL cells derive views, queries and computation directly inside those documents.\n\nSuppose a project stores architecture decisions as cards.\n\nA document can derive its architecture index:\n\n``` js\n# Architecture decisions\n\n``` hql\ncards\n| filter(c => c.metadata.kind == \"adr\")\n| sort(by = c => c.metadata.date)\n| map(c => {\n    title: c.title,\n    status: c.metadata.status,\n})\nThe same knowledge can be searched semantically:\n\n```\ncards\n| semantic(\"decisions affecting event delivery\")\n| take(10)\n```\n\nexpanded through relationships:\n\n```\ncards\n| semantic(\"event delivery\")\n| take(5)\n| expand(depth = 2)\n```\n\nand reused as context by an AI workflow.\n\nThe document, query, graph and AI context all operate over the same underlying model.\n\nA typical HQL program does not need to care whether every input came from the same kind of system.\n\n``` js\nadrs =\n    cards\n    | filter(c => c.metadata.kind == \"adr\")\n\nmetrics =\n    source(file(\"metrics.csv\"), Csv)\n\nevents =\n    source(\"deployment-events\")\n\n...\n```\n\nEach source can participate in ordinary HQL computation.\n\nThis is the larger idea behind the language:\n\n```\nfiles\nstdin\nvaults\ndatabases\nqueues\nnetwork sources\nknowledge graphs\nAI retrieval\n```\n\nall become inputs to the same typed dataflow model.\n\nHQL distinguishes pure computation from effectful interaction with external systems.\n\nConstructing an endpoint does not itself perform I/O:\n\n```\nfile(\"report.json\")\n```\n\nActual effects occur when the endpoint is used:\n\n```\nsource(file(\"report.json\"), Json)\n```\n\nor:\n\n```\nreport\n| target(file(\"report.json\"), Json)\n```\n\nThe execution host controls which effects are permitted.\n\nThis matters especially for executable documents.\n\nA host such as:\n\n```\nhql render document.hmd\n```\n\ncan allow document-local reads while refusing filesystem writes or network access.\n\nThe same language can therefore run under different capability policies without changing the program model.\n\nThe CLI exposes HQL directly without introducing a second language of command-line switches.\n\n```\nhql eval <program>     check and evaluate an expression\nhql run <file>         check and evaluate an HQL program\nhql check <file>       type-check without evaluating\nhql repl               interactive HQL session\nhql render <file>      evaluate HQL cells in a document\nhql builtins           inspect available operations\nhql config             inspect active configuration\nhql doctor             diagnose the configured environment\n```\n\nChoose the presentation format of the final result with:\n\n```\nhql run query.hql --to json\n```\n\nor:\n\n```\nhql run query.hql --to markdown\n```\n\nInput and output semantics stay in the language:\n\n```\nsource(stdin, Json)\n\nsource(file(\"input.csv\"), Csv)\n\nresult\n| target(file(\"result.yaml\"), Yaml)\n\nstream\n| sink(stdout, Json)\n```\n\nwhile:\n\n```\n--to\n```\n\nonly controls how the CLI renders the program's final result.\n\nDownload the appropriate release from:\n\nor install with Cargo:\n\n```\ncargo install hql --locked\n```\n\nInstall the current repository version:\n\n```\ncargo install --git https://github.com/ewiger/hql --locked\n```\n\nBuild from a clone:\n\n```\ncargo install --path . --locked\n```\n\nor:\n\n```\ncargo build --release\n```\n\nSee [CHANGELOG.md](https://github.com/ewiger/hql/blob/main/CHANGELOG.md) for changes between releases.\n\nThe full language documentation is at:\n\nA good way to begin is with a pipeline:\n\n``` js\ncards\n| filter(c => c.metadata.status == \"open\")\n| map(c => c.title)\n| sort\n```\n\nand then move outward into:\n\n- bindings and expressions,\n- collections and types,\n- functions and pattern matching,\n- streams and batching,\n- sources, targets and sinks,\n- retrieval,\n- knowledge graphs,\n- executable documents,\n- and AI workflows.\n\nHQL is built around a small number of strong ideas.\n\nQueries compose with ordinary language constructs instead of living in a separate statement language.\n\nPipelines are checked as programs rather than discovered dynamically one operation at a time.\n\nFinite values, streams and batches use the same pipeline model.\n\nA transformation should not fundamentally change because its input moved from a file to stdin, a vault, a database or a stream.\n\nDocuments, metadata, relationships, search results and graph nodes are all available for computation.\n\nRetrieval, context assembly, agents and their outputs belong in inspectable pipelines.\n\nHQL cells live next to the knowledge and prose they derive from and can run both from the CLI and interactively inside VS Code.\n\nThe same language model should remain valid whether computation is executed over a local value, a live stream, a configured data source or a larger execution backend.\n\nMIT — see [LICENSE](https://github.com/ewiger/hql/blob/main/LICENSE).", "url": "https://wpnews.pro/news/hql-run-typed-queries-directly-inside-markdown", "canonical_source": "https://github.com/ewiger/hql", "published_at": "2026-10-19 23:09:18+00:00", "updated_at": "2026-10-19 23:46:17.396394+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "structured-data", "ai-tools"], "entities": ["HQL", "Hyper Query Language", "HyperMarkDown", "VS Code", "hql.hypermarkdown.org"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/hql-run-typed-queries-directly-inside-markdown", "markdown": "https://wpnews.pro/news/hql-run-typed-queries-directly-inside-markdown.md", "text": "https://wpnews.pro/news/hql-run-typed-queries-directly-inside-markdown.txt", "jsonld": "https://wpnews.pro/news/hql-run-typed-queries-directly-inside-markdown.jsonld"}}