# I kept hitting jq: parse error on AI streams, so I built sjq

> Source: <https://dev.to/fanweixiao/i-kept-hitting-jq-parse-error-on-ai-streams-so-i-built-sjq-166m>
> Published: 2026-10-16 00:44:53+00:00

If you work with LLMs, you've done this: `curl -N <streaming endpoint> | jq`. And you've seen this:

```
jq: parse error: Invalid numeric literal at line 1, column 9
```

Column 9. That's where `data:` ends and the JSON begins. An SSE stream isn't JSON — it's a framing protocol: `data: {...}` lines separated by blank lines, `[DONE]` at the end. jq never stood a chance.

So I built **sjq** — jq, but for AI streaming. A small Rust CLI that parses OpenAI-compatible server-sent event streams (`/v1/chat/completions` and `/v1/responses`) and prints one short line per event, showing only what each event *added*.

```
sjq curl -s $BASE/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"gemini-4-argon","stream":true,"messages":[{"role":"user","content":"hi"}]}'
model: gemini-4-argon
role: assistant
data: 'Hi'
data: ' there'
data: ' 👋'
…
time: ttft=0.42s total=3.1s tok/s=258.2
```

Instead of 99% envelope JSON, you watch the actual content arrive — plus a timing line: TTFT, total, tokens/sec. Run through `sjq curl` and it also breaks out connect, TLS handshake, and TTFB, which is genuinely useful when you're debugging a slow provider.

It understands tool calls (name + arguments as they stream in), `reasoning_content`, refusals, logprobs, usage blocks, and multiple choices — each rendered as its own labeled line.

```
brew tap vivgrid/tap
brew trust vivgrid/tap  # Homebrew 6+ requires trusting third-party taps
brew install sjq
```

Or from source: `cargo install --git https://github.com/vivgrid/sjq` (one dependency: `serde_json`).

You can — `sed 's/^data: //' | jq` — and plenty of people do. But then you're staring at full envelope objects per chunk: ids, timestamps, `object: "chat.completion.chunk"`, `finish_reason: null` fifty times over. The signal (the tokens) drowns in the envelope. sjq inverts that: the delta is the output; the envelope gets summarized once.

MIT licensed, single binary. The repo ships 14 test fixtures covering the gnarly edges: multi-line data payloads, mid-stream errors, parallel tool calls, logprobs, CJK/emoji text.

Repo: [https://github.com/vivgrid/sjq](https://github.com/vivgrid/sjq) — feedback and weird streams welcome.