# HQL – Run typed queries directly inside Markdown > Source: > Published: 2026-10-19 23:09:18+00:00 **HQL — Hyper Query Language — is a statically typed language for querying, transforming, connecting and computing over data, knowledge and AI workflows.** HQL combines functional expressions, data pipelines, streams, structured documents, semantic retrieval and knowledge graphs in one language. ``` js birds = cards | filter(c => c.metadata.category == "bird") | sort(by = c => c.title) birds | map(c => c.title) ``` The same language can query a knowledge vault, transform JSON from the command line, build an ETL pipeline, traverse a graph, execute queries inside Markdown, or assemble context for an AI agent. - **Typed functional query language** — expressions, functions, pattern matching, collections and pipelines, checked before execution. - **Dataflow language** — compose transformations over finite data, streams and batches. - **Data integration and ETL** — connect files, standard streams, databases, queues and other sources and targets through one query model. - **AI-native language** — retrieval, context construction and agent workflows are expressed as transparent data pipelines. - **Executable Markdown** — run HQL cells inside HyperMarkDown documents and directly in VS Code, turning Markdown into live queries, reports, views and AI-enabled knowledge artifacts. - **Knowledge graph language** — retrieve, traverse, expand and combine structured knowledge and relationships. - **Command-line data tool** — process JSON, YAML, CSV, text and streams in the role normally filled by tools such as`jq` , while retaining types, functions and reusable programs. - **Streaming by design** —`Stream` is a first-class type; pipelines can process values as they arrive instead of materializing an entire dataset. - **One language across execution surfaces** — use HQL from the CLI,`.hql` programs, executable documents and interactive editor cells. Documentation: [hql.hypermarkdown.org](https://hql.hypermarkdown.org/) HQL is intentionally difficult to reduce to “a query language for X”. It is better understood through several complementary views. HQL programs are expressions. Values flow through pipelines, while functions, bindings, collections and pattern matching provide the tools needed for reusable computation. ``` js fn pages(b: Data) : Int = match b.pages { some(Int n) => n, _ => 0 } large_books = books | filter(b => pages(b) > 500) | sort(by = b => b.title) ``` Types are checked before execution. The type system includes collection types such as: ``` Seq List Set Map ``` first-class streams: ``` Stream ``` and algebraic value types such as: ``` Option ``` alongside `Data` for dynamically structured values. Pipelines are ordinary typed expressions rather than a separate query syntax: ``` js cards | filter(c => c.metadata.status == "open") | map(c => c.title) | sort ``` This makes the same language suitable for small queries and larger programs. HQL models computation as data moving through transformations. ``` js events | filter(e => e.kind == "purchase") | map(e => { customer: e.customer, amount: e.amount, }) | collect(100) | process_batch ``` A pipeline can operate over finite collections or over streams that continue producing values. ``` Stream ``` can become batches: ``` events | collect(100) ``` producing: ``` Stream> ``` while: ``` events | collect ``` materializes a completed stream as: ``` Seq ``` and: ``` items | emit ``` turns a finite collection back into a stream. Branches are part of the same model: ``` js events | tee errors = filter(e => e.level == "error") | aggregate(...) ``` Dataflow is therefore not an execution detail hidden underneath HQL. It is part of how HQL programs are written. External data enters HQL through typed sources. ``` source(stdin, Json) ``` For example: ``` js curl https://example.org/events | hql 'source(stdin, Json) | filter(e => e.active)' ``` `stdin` is a host-provided endpoint. It does not need to be configured. Files use the same model: ``` birds = source(file("birds.json"), Json) ``` as do configured sources: ``` warehouse = source("warehouse") ``` A source can stand directly in a pipeline: ``` js source("warehouse") | filter(r => r.created_at > cutoff) | map(r => r.customer_id) ``` Output follows the complementary model. A completed value can be written to a target: ``` report | target(file("report.yaml"), Yaml) ``` while a stream can be consumed incrementally by a sink: ``` events | sink(stdout, Json) ``` Standard endpoints such as `stdout` and `stderr` are host-provided just like `stdin`: ``` result | target(stdout, Yaml) errors | sink(stderr, Json) ``` They do not need entries in `hql.toml`. The distinction between whole values, streams and batches lives in the type system: ``` Event Seq Stream Stream> ``` not in separate I/O APIs. For example: ``` events | collect(100) | sink(stdout, Json) ``` streams batches: ``` Stream> ``` while: ``` events | collect | target(file("events.json"), Json) ``` materializes the complete stream and writes one finite result. Formats remain orthogonal: ``` Json Yaml Csv Text Markdown Hql D2 Mermaid ``` The format describes representation. The HQL type describes the shape of the data. Sources, transformations, streams, batches and targets all use the same expression model. That makes HQL suitable for ETL and data integration without introducing a separate workflow language around the query language. ``` js source("orders") | filter(o => o.status == "completed") | map(o => { customer: o.customer_id, total: o.total, }) | collect(500) | sink("analytics") ``` A pipeline can combine multiple sources: ``` customers = source("customers") orders = source("orders") ... ``` or move between files and structured output: ``` js source(file("sales.csv"), Csv) | filter(row => row.amount > 1000) | target(file("large-sales.json"), Json) ``` It can also mix live and finite processing: ``` js source(stdin, Json) | filter(e => e.level == "error") | tee archive = sink(file("errors.jsonl"), Json) | map(e => e.message) | sink(stderr, Text) ``` The query is the dataflow. There is no second orchestration syntax wrapped around it. Streams are first-class values. ``` Stream ``` represents values produced incrementally and may be bounded or unbounded. A stream can be transformed directly: ``` js source(stdin, Json) | filter(e => e.kind == "event") | map(e => e.payload) ``` collected completely: ``` events | collect ``` batched: ``` events | collect(100) ``` or created from a finite collection: ``` items | emit ``` This makes streaming a normal part of the type system rather than a separate execution mode. The same operations compose over: ``` Seq Stream Stream> ``` and preserve the distinction between finite data, individual stream elements and streaming batches. AI applications spend much of their complexity not in the model call itself, but in everything around it: - retrieving relevant information, - expanding context, - selecting evidence, - constructing prompts, - invoking agents, - evaluating results, - reacting to events, - recording outputs, - and feeding results into subsequent computation. These are dataflow problems. HQL expresses them as dataflow. ``` context = cards | semantic(question) | take(10) | expand(depth = 1) context | ... ``` Retrieval is not a terminal operation. The result remains ordinary typed data that can continue through filtering, graph traversal, transformation, aggregation and rendering. ``` js cards | semantic("distributed execution") | take(20) | filter(hit => hit.score > threshold) | expand(depth = 2) ``` This makes HQL a natural language for AI harnesses: retrieval, context construction, prompts, agents and their outputs can be represented as inspectable programs instead of opaque framework callbacks. In HyperMarkDown, agent definitions and instructions can themselves be documents. That means the knowledge, the prompt and the logic assembling the prompt can all remain visible, editable and queryable. HQL can run directly inside Markdown and HyperMarkDown documents. A fenced HQL cell is both documentation and executable logic: ``` js # Open architecture decisions ``` hql cards | filter(c => c.metadata.kind == "adr") | filter(c => c.metadata.status == "open") | sort(by = c => c.title) The same document can be rendered from the command line: ``` hql render architecture.hmd ``` or used interactively in VS Code through the HyperMarkDown extension: [HyperMarkDown VS Code extension](https://github.com/ewiger/hypermarkdown/tree/main/tools/hmd-vsc-ext) The extension supports HQL cells directly in HyperMarkDown documents, making the document itself an interactive query surface. This changes the role of Markdown. A document can become: - a live project index, - an architecture or ADR view, - a generated table or report, - a knowledge-base query, - a semantic-search surface, - a project dashboard, - an AI prompt, - a context-assembly document, - or an operational tool placed next to the prose that explains it. For example: ``` # Services The table below is derived from the project knowledge base. ``` hql cards | filter(c => c.metadata.kind == "service") | sort(by = c => c.title) | map(c => { service: c.title, owner: c.metadata.owner, status: c.metadata.status, }) The query remains visible next to the text it supports. There is no separate dashboard definition, template language or application layer required to derive the view. The Markdown document itself becomes the view, while HQL supplies its live data. This gives HQL several equivalent execution surfaces: ``` CLI hql '...' program query.hql document ``` hql ... ``` interactive doc VS Code HQL cell ``` The language remains the same across all of them. HyperMarkDown is therefore not merely a storage format for HQL. It is also one of HQL's natural interactive hosts. HQL treats links and relationships as queryable structure. Retrieval can find an initial set of nodes: ``` cards | semantic("night hunting birds") | take(3) ``` and graph traversal can expand from them: ``` cards | semantic("night hunting birds") | take(3) | expand(depth = 1) ``` Graph operations compose with ordinary filtering, mapping, aggregation and rendering. This allows HQL to move naturally between several forms of knowledge: ``` documents ↓ retrieval ↓ concepts and relationships ↓ graph traversal ↓ structured result ``` A knowledge graph is therefore not a separate database API attached to the language. It is another structure that participates in HQL pipelines. HQL provides retrieval as a first-class pipeline operation. Lexical search: ``` cards | lexical("night hunting birds") | take(3) ``` Semantic search: ``` cards | semantic("night hunting birds") | take(3) ``` The result can immediately participate in further computation: ``` js cards | semantic(question) | take(20) | filter(hit => hit.score > threshold) | expand(depth = 2) ``` This matters especially for retrieval-augmented AI workflows, where search is usually only the first stage of context construction. Retrieval results retain their provenance, so the result does not need to lose information about how it was obtained when it continues through a pipeline. HQL is useful without a vault or project. Pretty-print JSON from standard input: ``` cat data.json | hql 'source(stdin, Json)' --to json ``` Filter a JSON stream: ``` js cat birds.jsonl | hql ' source(stdin, Json) | filter(b => b.count > 2) | map(b => b.species) ' ``` Convert JSON to YAML: ``` cat config.json | hql 'source(stdin, Json)' --to yaml ``` Work with files: ``` js source(file("birds.json"), Json) | filter(b => b.count > 2) | target(file("popular-birds.yaml"), Yaml) ``` Or move the query into a reusable program: ``` hql run report.hql ``` HQL occupies some of the same space as `jq`, but its scope is deliberately broader: ``` jq-style structured data processing + static types + functions + reusable programs + streams + batching + multiple sources and targets + ETL and dataflow + knowledge graphs + semantic retrieval + executable Markdown + AI workflows ``` The goal is not to imitate the syntax of existing tools. It is to provide one coherent language across these workloads. HQL treats data representations explicitly. ``` format(result, Json) format(result, Yaml) format(result, Markdown) ``` Presentation types include: ``` Text Json Yaml Csv Markdown Hql D2 Mermaid ``` Formats describe encoding rather than execution shape. For example: ``` items : Seq items | target(stdout, Json) ``` produces one JSON array. But: ``` items | emit | sink(stdout, Json) ``` writes items incrementally as individual JSON documents. Likewise: ``` events | collect(100) | sink(stdout, Json) ``` writes a stream of JSON arrays. The difference comes from: ``` Seq Stream Stream> ``` not from introducing separate formats such as a special JSON-lines data type. HyperMarkDown is a natural companion to HQL, but HQL is not limited to HyperMarkDown. A HyperMarkDown vault provides structured cards, metadata, links and knowledge relationships: ``` js cards | filter(c => c.metadata.kind == "adr") | map(c => c.title) ``` The same language can operate over other sources and can be used without a vault at all. This distinction is intentional: **HyperMarkDown provides structured knowledge. HQL computes over it.** Together they also provide a natural programmable-document environment: HyperMarkDown supplies the document structure, while HQL cells derive views, queries and computation directly inside those documents. Suppose a project stores architecture decisions as cards. A document can derive its architecture index: ``` js # Architecture decisions ``` hql cards | filter(c => c.metadata.kind == "adr") | sort(by = c => c.metadata.date) | map(c => { title: c.title, status: c.metadata.status, }) The same knowledge can be searched semantically: ``` cards | semantic("decisions affecting event delivery") | take(10) ``` expanded through relationships: ``` cards | semantic("event delivery") | take(5) | expand(depth = 2) ``` and reused as context by an AI workflow. The document, query, graph and AI context all operate over the same underlying model. A typical HQL program does not need to care whether every input came from the same kind of system. ``` js adrs = cards | filter(c => c.metadata.kind == "adr") metrics = source(file("metrics.csv"), Csv) events = source("deployment-events") ... ``` Each source can participate in ordinary HQL computation. This is the larger idea behind the language: ``` files stdin vaults databases queues network sources knowledge graphs AI retrieval ``` all become inputs to the same typed dataflow model. HQL distinguishes pure computation from effectful interaction with external systems. Constructing an endpoint does not itself perform I/O: ``` file("report.json") ``` Actual effects occur when the endpoint is used: ``` source(file("report.json"), Json) ``` or: ``` report | target(file("report.json"), Json) ``` The execution host controls which effects are permitted. This matters especially for executable documents. A host such as: ``` hql render document.hmd ``` can allow document-local reads while refusing filesystem writes or network access. The same language can therefore run under different capability policies without changing the program model. The CLI exposes HQL directly without introducing a second language of command-line switches. ``` hql eval check and evaluate an expression hql run check and evaluate an HQL program hql check type-check without evaluating hql repl interactive HQL session hql render evaluate HQL cells in a document hql builtins inspect available operations hql config inspect active configuration hql doctor diagnose the configured environment ``` Choose the presentation format of the final result with: ``` hql run query.hql --to json ``` or: ``` hql run query.hql --to markdown ``` Input and output semantics stay in the language: ``` source(stdin, Json) source(file("input.csv"), Csv) result | target(file("result.yaml"), Yaml) stream | sink(stdout, Json) ``` while: ``` --to ``` only controls how the CLI renders the program's final result. Download the appropriate release from: or install with Cargo: ``` cargo install hql --locked ``` Install the current repository version: ``` cargo install --git https://github.com/ewiger/hql --locked ``` Build from a clone: ``` cargo install --path . --locked ``` or: ``` cargo build --release ``` See [CHANGELOG.md](https://github.com/ewiger/hql/blob/main/CHANGELOG.md) for changes between releases. The full language documentation is at: A good way to begin is with a pipeline: ``` js cards | filter(c => c.metadata.status == "open") | map(c => c.title) | sort ``` and then move outward into: - bindings and expressions, - collections and types, - functions and pattern matching, - streams and batching, - sources, targets and sinks, - retrieval, - knowledge graphs, - executable documents, - and AI workflows. HQL is built around a small number of strong ideas. Queries compose with ordinary language constructs instead of living in a separate statement language. Pipelines are checked as programs rather than discovered dynamically one operation at a time. Finite values, streams and batches use the same pipeline model. A transformation should not fundamentally change because its input moved from a file to stdin, a vault, a database or a stream. Documents, metadata, relationships, search results and graph nodes are all available for computation. Retrieval, context assembly, agents and their outputs belong in inspectable pipelines. HQL cells live next to the knowledge and prose they derive from and can run both from the CLI and interactively inside VS Code. The same language model should remain valid whether computation is executed over a local value, a live stream, a configured data source or a larger execution backend. MIT — see [LICENSE](https://github.com/ewiger/hql/blob/main/LICENSE).