databow is an open source command-line tool, written in Rust, for querying any database that has an ADBC driver (43 at last count). Version 0.2.0 is out today with Parquet, JSON Lines, and Arrow IPC stream output, dot commands and result truncation in the interactive shell, driver inference from ADBC URIs, a Homebrew tap, and more. Install or upgrade now.
When we introduced databow, the idea was one fast CLI for querying any database that has an ADBC driver. From the first release it could write results to CSV, JSON, and Arrow IPC files.
Parquet wasn’t on that list, and it was the format people asked for most. If your results are headed for a data lake or an analytics job downstream, you probably want Parquet, since it’s compact and nearly every data tool reads it.
So 0.2.0 adds Parquet, along with JSON Lines and Arrow IPC streams. The interactive shell improvements from our roadmap are in too: dot commands, result truncation, and a little context when you open the REPL.
What’s new in databow 0.2.0 #
Export straight to Parquet
Point --output at a .parquet file and databow writes Parquet:
$ databow --driver duckdb --uri warehouse.db \
--query "SELECT * FROM events WHERE level = 'ERROR'" \
--output errors.parquet
databow talks to your database over ADBC, so results already arrive as Apache Arrow columnar data. Parquet is also columnar, so databow converts Arrow batches to Parquet directly, without serializing row by row in between.
JSON Lines and Arrow IPC streams
0.2.0 adds two more output formats. As with the existing ones, databow picks the format from the output file’s extension.
JSON Lines writes one JSON record per line to a .jsonl file. Use it when you want to process records one at a time with line-oriented tools like jq, grep, and head:
$ databow --profile warehouse \
--query "SELECT id, name, email FROM users" \
--output users.jsonl
Since each line is a complete JSON object, jq handles records one at a time as you pipe the file through. The first command below prints the first few records, and the second counts users by email domain:
$ head -n 3 users.jsonl | jq .
$ jq -r '.email | split("@")[1]' users.jsonl | sort | uniq -c | sort -rn
Arrow IPC streams use the .arrows extension. Like Parquet, Arrow is a columnar format, and a growing number of data tools read it. Parquet uses encodings and compression to keep files small, at the cost of encoding and compressing on every write and decoding and decompressing on every read, whereas Arrow data is laid out on disk and on the network exactly as it is in memory and tools can skip all of that work. When moving bytes is cheap, as on a single machine or a fast network, Arrow can be faster than Parquet.
databow already supported the Arrow IPC file format; streams are the other flavor. A reader can jump around in a file but reads a stream in order, start to finish, and a writer can start a stream without knowing where it ends, which suits pipelines and passing data between processes:
$ databow --profile prod-db \
--query "SELECT * FROM events" \
--output events.arrows
From there, any tool that reads Arrow can pick up the result. In Python, for example, Polars reads it into a DataFrame:
import polars as pl
events = pl.read_ipc_stream("events.arrows")
PyArrow reads it into a table:
import pyarrow as pa
with open("events.arrows", "rb") as source:
events = pa.ipc.open_stream(source).read_all()
The Arrow libraries for R and a dozen other languages can read the same file, and so can ClickHouse, DataFusion, and DuckDB.
Driver inferred from the URI
If your connection URI starts with a scheme, databow now uses that scheme to pick the driver. You don’t need --driver:
$ databow --driver duckdb --uri warehouse.db
$ databow --uri duckdb://warehouse.db
The same works for other databases, like --uri postgresql://user@localhost:5432/mydb. A profile:// scheme loads a connection profile (--uri profile://warehouse and --profile warehouse do the same thing). An explicit --driver still takes precedence, and URIs without a scheme work as before.
Other than profile://, the scheme is always the driver’s short name, the same name you pass to dbc install and --driver. The rest of the URI varies by database. To find the format for yours, see the Databases page in the databow docs or the documentation for your driver at adbc-drivers.org.
Statement options
The new --statement-option flag passes a driver-specific option through to every statement databow runs. We added it for a couple of databases, CockroachDB and CrateDB, that speak the PostgreSQL wire protocol but don’t fully support COPY, which the ADBC PostgreSQL driver uses by default to fetch results. For CrateDB:
$ databow --driver postgresql --uri postgresql://crate@localhost:5432/crate \
--statement-option adbc.postgresql.use_copy=false
Info on REPL startup
When you open the interactive shell, databow now prints its version and the driver it connected to:
$ databow --driver duckdb
databow 0.2.0
Connected to duckdb v1.5.5
>
You can see at a glance that you connected to the right database with the driver version you expected. It’s also two lines you can paste into a bug report.
Dot commands
The interactive shell now accepts dot commands as well as SQL. We modeled them on the DuckDB CLI’s, and like those they start with ., fit on one line, and don’t need a semicolon. Run .help to list them:
> .help
.help Show this help
.last Show the last result in full, without truncation
.maxrows ?COUNT? Show or set the maximum number of rows to display (0 for no limit)
.maxwidth ?WIDTH? Show or set the maximum table width (0 for no limit)
.mode ?MODE? Show or set the table display style
.pager ?OPTIONS? Show or configure paging of large results
.quit, .exit Exit databow
For example, .mode switches the table display style mid-session, with the same styles as the --mode startup flag, so you don’t have to restart databow to change it:
> .mode ascii-markdown
Results that fit your terminal
Before 0.2.0, a SELECT * on a big table in the REPL meant thousands of rows scrolling past. Now databow truncates large results to fit your terminal. You get the first and last rows, a gap in the middle, and a footer with the total row count and how many rows it showed:
> .maxrows 10
> SELECT range AS id, 'user_' || range AS name, range * 1.5 AS score FROM range(1000);
┌─────┬──────────┬────────┐
│ id │ name │ score │
├─────┼──────────┼────────┤
│ 0 │ user_0 │ 0.0 │
│ 1 │ user_1 │ 1.5 │
│ 2 │ user_2 │ 3.0 │
│ 3 │ user_3 │ 4.5 │
│ 4 │ user_4 │ 6.0 │
│ · │ · │ · │
│ · │ · │ · │
│ · │ · │ · │
│ 995 │ user_995 │ 1492.5 │
│ 996 │ user_996 │ 1494.0 │
│ 997 │ user_997 │ 1495.5 │
│ 998 │ user_998 │ 1497.0 │
│ 999 │ user_999 │ 1498.5 │
└─────┴──────────┴────────┘
1000 rows (10 shown) 3 columns
Similarly, for wide results, databow shortens long values with …, and if the table still doesn’t fit, it hides middle columns behind a single … column. The defaults are 40 rows and the terminal width; change them with .maxrows and .maxwidth (0 means no limit). None of this applies outside the interactive shell. Results from --query, --file, or stdin are never truncated.
A pager for when you want it all
To see the whole result, run .last, which shows the previous result in full. Big results open in an external pager (less -SRX by default, more on Windows), so you can scroll through rows and across wide tables without flooding your scrollback.
Paging only applies to output databow hasn’t truncated: .last, or any result after you turn the limits off with .maxrows 0 or .maxwidth 0. By default databow pages that output when it has more than 50 rows or is wider than the terminal. .pager on and .pager off turn paging on or off for every result, .pager set_row_threshold COUNT changes the row threshold, and .pager COMMAND sets a different pager program. databow also reads the DATABOW_PAGER and PAGER environment variables.
More databases #
The ADBC driver ecosystem has grown a lot since the first databow release. New drivers for Apache Cassandra, Apache Druid, Apache Spark, and Presto have come out, and the list of databases we’ve confirmed work with databow has grown from 28 to 43. Here they are in alphabetical order:
Amazon Redshift, Apache Cassandra, Apache DataFusion, Apache Doris, Apache Druid, Apache Spark, CedarDB, chDB, Citus, ClickHouse, CockroachDB, CrateDB, Databricks, Dremio, DuckDB, DuckDB Quack, Exasol, GizmoSQL, Google BigQuery, GreptimeDB, InfluxDB, MariaDB, Microsoft SQL Server, MotherDuck, MySQL, Neon, OceanBase Database, Oracle Database, ParadeDB, PostgreSQL, Presto, Sail, SingleStore, Snowflake, SQLite, StarRocks, Teradata, TiDB, TimescaleDB, Trino, Vitess, Yellowbrick, and YugabyteDB.
For connection examples, see the Databases page in the databow docs. If your database isn’t listed but has an ADBC driver or supports Arrow Flight SQL, databow should work with it too.
Install or upgrade to databow 0.2.0 #
databow is a single binary. Install or upgrade it with uv, Homebrew, or Cargo:
$ uv tool install databow --upgrade
$ brew install columnar-tech/tap/databow
$ cargo install databow
Next, install the ADBC driver for your database with dbc, our command-line tool for installing and managing ADBC drivers. For DuckDB:
$ dbc install duckdb
Then start querying:
$ databow --driver duckdb
What’s next #
Next on the databow roadmap are data types shown alongside query results and more dot commands for exploring your database. We want to make it easier to go from an unfamiliar database to a useful result.
Parquet support started with requests from people using databow. Try 0.2.0 with a query you run every day, and tell us what still gets in your way.
Next steps #
- Install or upgrade to databow 0.2.0 with uv, Homebrew, or Cargo.
- Use the databow documentation to connect to your database and explore output formats and shell commands.
- Open an issue to report a bug or suggest a feature, or contribute todatabow on GitHub .