One Interface, Seven Formats: How Solon AI Turns Files, Web Pages, and Even Database Schemas into RAG Documents

Every RAG pipeline starts the same way: you have stuff, and the model needs Document s.

The interesting question is how far that idea stretches. Solon AI answers it with a deliberately small contract — and then pushes it across seven formats, including one you probably haven't tried feeding to a retriever: your database schema.

This is a source-code tour of solon-ai-rag-s. All claims below are checked against the current source tree; where a class behaves in a way you wouldn't guess from its name, I'll point it out.

public interface Document {
    Document additionalMetadata(String key, Object value);
    Document additionalMetadata(Map<String, Object> metadata);
    List<Document> load() throws IOException;
}

That's the entire contract: two metadata methods and one load(). No provider field, no API key, no vendor. Seven Maven sub-modules implement it (solon-ai-load-markdown, -pdf, -word, -excel, -html, -ppt, -ddl), each pulling only its own parsing dependency — commonmark, PDFBox, POI, jsoup, Tika.

The base class AbstractOptionsDocument adds the options pattern with two entry points:

Markdown  = new Markdown(file)
        .options(o -> o.codeBlockAsNew(true));
// or, if you already hold an Options instance:
.options(myOptions);

A SupplierEx<InputStream> constructor appears in every , so your source can be a file, a URL, a byte array, or anything else that can produce a stream lazily.

Seven s, and no single "chunk size" knob. Instead, each picks its default unit of meaning — and the defaults disagree on purpose:

That asymmetry is the design. A paragraph is the natural retrieval unit for prose; a page is the natural unit for a PDF; a slide deck usually makes more sense as one document; a table is a complete thought. You can override the defaults (Pdf goes SINGLE, Word goes SINGLE, Ppt splits on "\n\n\n"), but the out-of-the-box behavior already encodes a per-format answer to "what is a chunk here?"

Markdown doesn't slice text with regexes. It parses the document with commonmark into an AST and walks it with a visitor:

horizontalLineAsNew, blockquoteAsNew, codeBlockAsNew. codeBlockAsNew(true), the code block starts its own document. Either way, a fenced block category=header_1..6 with a title, category=code_block with lang, or category=blockquote. One nuance worth knowing before you rely on metadata: the visitor writes title/ category onto the current document while walking. If a section has no heading text before its content, the metadata simply won't be there for that chunk. Fine for retrieval; worth remembering if you build UI on top of it.

Pdf (PDFBox) defaults to one Document per page, each stamped with page, total_pages, and a summary of "Page 3" — handy in a search UI. Switch to LoadMode.SINGLE and you get the whole file as one document, pages joined by "\n\f", with just a pages count.

Word handles both binary eras: it checks the stream with POI's FileMagic and routes .docx (OOXML) and legacy .doc (OLE2) to different readers. It defaults to paragraph mode — one Document per paragraph — with a SINGLE escape hatch.

Excel (POI + snack4) treats the first non-empty row of a sheet as the header row, then maps every following row to {column: value} and serializes batches as JSON documents. Two defaults shape its behavior:

documentMaxRows(-1) to keep one document per sheet.break s, so anything after the first blank row is silently ignored — by design, trailing blank rows shouldn't kill the parse, but data below a blank row won't be indexed. Keep that in mind with hand-edited spreadsheets. Formula cells are read as their formula text, not computed values.

Ppt doesn't parse slide XML itself. It hands the stream to Apache Tika's AutoDetectParser and gets body text back. Default is SINGLE — the whole deck as one document; PAGE mode splits on "\n\n\n" if your decks have predictable slide breaks.

This is the one that changes how you think about the pipeline. Ddl connects to a plain DataSource (no ORM, no entities) and emits one Document per table containing its DDL:

Ddl  = new Ddl(dataSource);   // MySQL config built in
.options(o -> o.loadOptions("shop", null)); // schema only: all its tables
List<Document> docs = .load();

Three granularities via loadOptions(schema, table): whole instance, one schema, one table. The default configuration is MySQL (information_schema + SHOW CREATE TABLE, system schemas excluded), but every SQL string is a template — the runs them through Solon's own expression engine (SnEL.evalTmpl), so you can rewire it for another database by replacing the template set in a DdlLoadConfig.

One detail I like: SHOW CREATE TABLE returns CREATE TABLE \ order(...) — a table name that's only meaningful inside its schema. The rewrites the header to CREATE TABLE \ shop.\ order(...) so every retrieved DDL document is self-describing, and stamps metadata("table", "order") so your filter layer can target tables directly.

The use case writes itself: point it at production (read-only!), and your AI assistant retrieves schema facts instead of hallucinating column names.

load(): One Shape Downstream Whatever the format, load() hands you List<Document> — content plus metadata plus the fluent fields (title, url, summary, id, embedding, score). From here, everything is format-agnostic: embed, store in a Repository, attach as a tool. The s are the only place in the pipeline where format-specific knowledge lives.

To be fair to your architecture review: if your corpus is already clean Markdown, you might not need seven s — Solon AI's splitter story covers embedding-time splitting separately. The s earn their keep when sources are heterogeneous (office files, web pages, live schema) or when the natural unit (page, paragraph, table) should decide the chunk, not a character count.

solon-ai-rag-s is a good example of a small contract held firmly: three methods, seven implementations, and per-format defaults that encode real opinions instead of one generic knob. The DDL alone is worth a look if you build assistants that need to talk about your database accurately.

solon-ai-rag-s.