{"slug": "show-hn-bricksentry-see-what-a-data-change-does-before-it-ships-databricks", "title": "Show HN: Bricksentry – see what a data change does before it ships (Databricks)", "summary": "Brahmareddy, a Databricks Champion, released Bricksentry, an Apache-2.0 open-source change-safety tool that previews what a data change will do to Databricks tables before it ships, including changes made by AI agents editing pipelines. Bricksentry runs a local demo on a 20,000-row revenue table in DuckDB with no Databricks account required, and in that demo it blocks a pull request for a join that duplicates 40 orders and blocks a bad MERGE for 25 duplicate keys, printing and applying the RESTORE command. The tool ships as a Databricks asset-bundle job template, a check-my-table notebook, a GitHub Action for pull requests, and an MCP server that gives AI agents the same checks.", "body_md": "**The safety check for data changes in the AI era.**\n\nChange-safety for Databricks pipelines and the AI agents that edit them: know what a change does to your data before it ships.\n\n**[Try it in 30 seconds](#1-try-it-in-30-seconds-on-your-laptop)** · **[Check every change on Databricks](#2-check-every-change-to-your-tables-on-databricks)** · **[Docs: bricksentry.org](https://bricksentry.org)** · [LinkedIn](https://www.linkedin.com/company/bricksentry-org/)\n\nBuilt by Brahmareddy, Databricks Champion. Open source under Apache-2.0.\n\n*Why the name?* Bricks plus entry. Your tables and pipelines are the bricks of the lakehouse, and every change to them is an entry. bricksentry checks each entry, reports what it does, and stands guard before it lands.\n\n```\nuvx --from \"bricksentry[duckdb]\" bricksentry demo\n```\n\n(Or `pip install \"bricksentry[duckdb]\"` and then `bricksentry demo`. In a Databricks or Jupyter notebook, run `%pip install \"bricksentry[duckdb]\"` in one cell, then `from bricksentry.demo import run_demo; run_demo()` in the next; on Databricks, add `dbutils.library.restartPython()` after the install.) No Databricks account and nothing leaves your machine: it runs on a 20,000-row revenue table in local DuckDB. An AI agent previews a DELETE on a clone (production untouched), gets its pull request **blocked** for a join that duplicates 40 orders, fixes it, and is routed to the owners of a certified table downstream. Then a nightly pipeline commits a bad MERGE anyway: bricksentry finds it in the table history, skips the OPTIMIZE after it, **blocks** it for 25 duplicate keys, prints the `RESTORE` and applies it.\n\n```\ndatabricks bundle init https://github.com/bricksentry/bricksentry --template-dir templates/job-check\ncd bricksentry_check && databricks bundle deploy\ndatabricks bundle run bricksentry_demo     # optional: see a block on a demo table in about 2 minutes\n```\n\nOne command and a few prompts give you a serverless job that runs after every commit to the tables you pick: the same check as the demo's last step, on your data, with the `RESTORE` that undoes a bad change and an optional Slack alert. The optional demo job writes a known-bad MERGE to a demo table in your scratch schema and checks it, so your first run ends red with a block. [Details below](#check-every-pipeline-run-automatically-asset-bundle).\n\nPut the check inside your pipeline job, right after the task that writes the table: a `block` fails that task and the tasks downstream don't run. Or run it on pull requests with the [GitHub Action](#for-data-engineers-cli-and-ci), and give AI agents the same checks through the [MCP server](#for-ai-agents-the-mcp-server).\n\n**Check-my-table notebook:** in your workspace, go to **Workspace → Import → URL** and paste\n`https://raw.githubusercontent.com/bricksentry/bricksentry/main/examples/databricks/bricksentry_check_my_table.py`\n\nPoint the [check-my-table notebook](https://github.com/bricksentry/bricksentry/blob/main/examples/databricks/bricksentry_check_my_table.py) at a Delta table your pipeline writes to. It finds the latest data change in the table's history (a WRITE, MERGE, UPDATE or DELETE), compares it with the version before, and shows the verdict a reviewer would have seen before that change shipped: rows added, removed and changed, duplicate keys, totals that moved, what depends on the table, and the `RESTORE` command to undo it. Your table is only read; the comparison uses two shallow clones in a scratch schema, dropped at the end.\n\nIf it shows you something, please [share what it found](https://github.com/bricksentry/bricksentry/issues/new?template=it_caught_this.yml) (no table names needed). Real results decide what gets built next.\n\n```\ndatabricks bundle init https://github.com/bricksentry/bricksentry --template-dir templates/job-check\ncd bricksentry_check && databricks bundle deploy -t prod\n```\n\nAnswer a few prompts (tables to watch, key columns if they have no primary key, SQL warehouse, scratch schema, whether to add the demo job, policy, optional Slack webhook secret). You get a serverless job with a **table update trigger**: whenever a watched table changes, it checks the change, sends the alert and logs the `RESTORE` that undoes it. To stop downstream tasks when a change is bad, add the same check as a task inside your pipeline job, right after the write (the generated README shows the YAML): with `--since {{job.start_time.iso_datetime}}` it checks everything the run wrote, and a `block` verdict fails the task so the tasks after it don't run.\n\nFrom the command line, the same check is one command:\n\n```\nbricksentry check-last-change main.sales.orders                 # newest data change vs the version before\nbricksentry check-last-change main.sales.orders --version 42    # a specific version\nbricksentry check-last-change main.sales.orders --since 2026-11-11T09:00:00Z --fail-on needs_human\n```\n\nKeys come from `bricksentry.yml`, `--key`, or the table's `PRIMARY KEY` constraint. OPTIMIZE and VACUUM commits are skipped because they don't change data.\n\n**HR walkthrough notebook:** Workspace → Import → URL →\n`https://raw.githubusercontent.com/bricksentry/bricksentry/main/examples/databricks/bricksentry_hr_walkthrough.py`\n\nThe [HR walkthrough](https://github.com/bricksentry/bricksentry/blob/main/examples/databricks/bricksentry_hr_walkthrough.py) builds a 200,000-row employee table (tagged pii) feeding a certified pay summary, then shows everything bricksentry does on it: table and column blast radius, two pull requests (one risky, one broken), every rule, an agent's write previewed on a clone with dangerous SQL refused, the PR report, column-level policies, sampling that can't hide a duplicate, lineage from config, alerts and the MCP tools. It removes everything it created at the end. First run on a live workspace by Genie, an AI agent in Databricks, in November 2026.\n\n**Databricks quickstart notebook:** in your workspace, go to **Workspace → Import → URL** and paste\n`https://raw.githubusercontent.com/bricksentry/bricksentry/main/examples/databricks/bricksentry_quickstart.py`\n\nThe [quickstart notebook](https://github.com/bricksentry/bricksentry/blob/main/examples/databricks/bricksentry_quickstart.py) creates a small demo table in a schema you choose and runs the same loop on Databricks: Unity Catalog blast radius, a real `RESTORE` rollback plan, a preview on a shallow clone, a blocked pull request and the fix. Then it checks one of your own tables, read-only, and cleans up after itself.\n\nYou need a SQL warehouse, `CREATE TABLE` on the schema you pick, and `SELECT` on `system.access.table_lineage` for the blast radius. A Jupyter copy is in [`bricksentry_quickstart.ipynb`](https://github.com/bricksentry/bricksentry/blob/main/examples/databricks/bricksentry_quickstart.ipynb).\n\nBefore a change reaches production, bricksentry answers three questions, in a pull-request comment for humans and in structured JSON for agents:\n\n1. **What does it do to the data?** A key-based data diff: added, removed and changed rows, schema changes, per-column changes, numeric totals, duplicate keys, and*where* the change is concentrated (\"100% of affected rows are region = APAC\").\n2. **Who gets hurt?** The downstream blast radius from Unity Catalog lineage: tables, jobs, notebooks, Lakeflow pipelines, dashboards and queries, with owners and tags. Down to the column: which columns the change touched, whether they are tagged (for example`pii` ), and which downstream columns they feed.\n3. **Is it safe, and how do I undo it?** A deterministic policy verdict (`pass` ·`warn` ·`needs_human` ·`block` ) and the exact`RESTORE TABLE … VERSION AS OF n` rollback.\n\nFor your Databricks workspace:\n\n```\npip install \"bricksentry[databricks,mcp]\"\nbricksentry init        # writes bricksentry.yml and policy.yml\n```\n\nAI now writes much of our pipeline code, and checking what that code does to the data hasn't kept up. Code review can't see a silent 3% revenue shift or a join that fans out. The tools that do catch this mostly cover dbt, or need a paid platform. Most Databricks work is PySpark notebooks, Jobs and Lakeflow pipelines, and bricksentry is built for that work:\n\n- **Unity Catalog runtime lineage** , not SQL parsing, so PySpark notebooks and jobs are included.\n- **SQL pushdown** : all diffing runs on your SQL warehouse, so no Spark session is needed and calls stay cheap enough for an agent loop.\n- **One engine, three faces** : CLI and GitHub Action · Python API · MCP server.\n\nThe quickest way to connect an agent is the `bricksentry-mcp` package, which installs bricksentry with its MCP and Databricks extras. With [uv](https://docs.astral.sh/uv/) there is nothing to install first:\n\n```\n# Claude Code\nclaude mcp add bricksentry -- uvx bricksentry-mcp --config /full/path/to/bricksentry.yml\n{\n  \"mcpServers\": {\n    \"bricksentry\": {\n      \"command\": \"uvx\",\n      \"args\": [\"bricksentry-mcp\", \"--config\", \"/full/path/to/bricksentry.yml\"]\n    }\n  }\n}\n```\n\nThe JSON works for the Claude desktop app, Cursor and other MCP clients. Use a **full path** to `bricksentry.yml`, because clients start the server from their own working folder. If an app can't find `uvx`, use its full path (` which uvx`). To share the setup through a repo, `claude mcp add -s project ...` writes it to `.mcp.json`; a relative path then works when Claude Code starts from the repo root. No config yet? `uvx --from bricksentry bricksentry init` writes a starter one.\n\nAlready installed bricksentry with pip? `pip install \"bricksentry[mcp,databricks]\"` and run `bricksentry mcp --config /full/path/to/bricksentry.yml` instead. Add `--transport streamable-http` for remote agents.\n\nbricksentry is listed in the [official MCP Registry](https://registry.modelcontextprotocol.io/v0/servers?search=io.github.bricksentry/bricksentry) as `io.github.bricksentry/bricksentry`.\n\n| Tool | What the agent gets | \n|---|---|\n| `whats_downstream(table, depth, columns?)` | Downstream assets with owners and tags. Call it *before* editing. Pass the columns you'll change to see the downstream columns they feed and their tags. | \n| `rollback_plan(table)` | The restore SQL. Capture it *before* changing anything. | \n| `preview_write(sql, target, keys?)` | Runs one write against a **disposable shallow clone** , never production, and returns the diff plus verdict. | \n| `assess_change(tables, candidates?, pr?)` | Diffs prod vs. the candidate built by your change, plus lineage and policy. Returns a verdict and an `id` . | \n| `explain_diff(assessment_id or table)` | Plain-English summary. | \n| `safe_to_ship(assessment_id)` | `safe_to_ship` , the reasons, and an explicit`next_step` to follow. | \n\nDesigned for agents:\n\n- **The verdict never comes from an LLM.** Rules in`policy.yml` decide, so the same change always gets the same answer.\n- **Read-only by default.** Only`preview_write` writes, and only to a sandbox clone. It accepts a single INSERT / MERGE / UPDATE / DELETE / CREATE OR REPLACE whose target must be the clone, and it refuses DROP, TRUNCATE, ALTER, GRANT and multiple statements.\n- **Errors come back as `{\"error\": ...}`** , so agents can recover instead of crashing.\n- **Bounded cost:** deterministic hash sampling (`sample_pct` ), capped samples, and lineage limited by depth and lookback window. A full-table checksum guards the sample: if the sample shows no change but the tables differ, the diff re-runs on every row, so a one-row change can't slip through.\n- **Assessments are saved** as JSON (default`~/.bricksentry/assessments` ), so`explain_diff` and`safe_to_ship` work from another process or a fresh`Sentry` .\n\nThe same tools are plain Python functions in `bricksentry.agent_tools`, so you can use them from LangGraph, the OpenAI Agents SDK or the Databricks Agent Framework without MCP.\n\nTo watch the whole loop, run `bricksentry demo` (see the top of this page). `bricksentry demo --report report.md` also writes the pull-request comment a reviewer would see.\n\nRun the MCP server as a Databricks App so every agent in your workspace can reach it over HTTPS, with no tokens to manage: it runs as the app's service principal, and Unity Catalog grants decide what it can read. [`deploy/databricks-app`](https://github.com/bricksentry/bricksentry/blob/main/deploy/databricks-app) is a ready app (`app.yaml`, an Asset Bundle, the grants) that deploys with two commands:\n\n``` js\ndatabricks bundle deploy --var warehouse_id=<your-sql-warehouse-id>\ndatabricks bundle run bricksentry_mcp\n```\n\nAgents connect to `https://<app-url>/mcp`. The kit starts the server with `--read-only`, which leaves out `preview_write`, so the app never writes. On any other host, `bricksentry mcp --transport streamable-http --host 0.0.0.0 --port 8000` serves the same endpoint. Unity Catalog *MCP Services* are for servers hosted outside Databricks, so a Databricks App is the recommended way to run bricksentry inside it; see the [kit's README](https://github.com/bricksentry/bricksentry/blob/main/deploy/databricks-app/README.md).\n\n```\nbricksentry init                                        # starter bricksentry.yml + policy.yml\nbricksentry init --policy finance                       # ...or start from a policy template\nbricksentry policies                                    # list the policy templates\nbricksentry impact main.sales.orders                    # blast radius\nbricksentry diff main.sales.orders main.sales_pr_42.orders\nbricksentry preview main.sales.orders --sql \"MERGE INTO {{target}} t USING updates u ON t.order_id = u.order_id WHEN MATCHED THEN UPDATE SET *\"\nbricksentry assess --from-git origin/main --pr 42 --format markdown --fail-on needs_human\nbricksentry rollback main.sales.orders\nbricksentry check-last-change main.sales.orders             # the change a pipeline already made\n```\n\nExit codes: `0` below `--fail-on`, `1` at or above it, `2` for configuration errors.\n\n**In GitHub Actions.** Build your PR's version of the pipeline into a per-PR schema (for example with a Databricks Asset Bundle `ci` target), then add:\n\n```\n- run: pip install \"bricksentry[databricks]\"\n- run: >\n    bricksentry assess\n    --from-git origin/${{ github.base_ref }}\n    --pr ${{ github.event.pull_request.number }}\n    --comment --output bricksentry-report.md\n    --fail-on needs_human\n  env:\n    GITHUB_TOKEN: ${{ github.token }}\n    DATABRICKS_HOST: ${{ secrets.DATABRICKS_HOST }}\n    DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}\n    DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}\n    DATABRICKS_WAREHOUSE_ID: ${{ secrets.DATABRICKS_WAREHOUSE_ID }}\n```\n\nIt detects changed tables from the files in the PR and posts or updates one PR comment, like this one for the demo's fan-out bug:\n\nCheck out with `fetch-depth: 0` and give the job `pull-requests: write` permission.\n\nOr use the bundled GitHub Action, which also writes the report to the job summary:\n\n```\n- uses: bricksentry/bricksentry@v0.1.13\n  with:\n    config: bricksentry.yml\n    fail-on: needs_human\n```\n\nA complete workflow you can copy to `.github/workflows/bricksentry.yml`:\n\n```\nname: bricksentry\non: pull_request\n\npermissions:\n  contents: read\n  pull-requests: write\n\njobs:\n  check-data:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n        with:\n          fetch-depth: 0\n      # Build this PR's version of your pipeline into a per-PR schema first,\n      # for example: databricks bundle deploy -t ci && databricks bundle run -t ci\n      - uses: bricksentry/bricksentry@v0.1.13\n        with:\n          config: bricksentry.yml\n          fail-on: needs_human\n          slack-webhook: ${{ secrets.SLACK_WEBHOOK_URL }}    # optional\n          teams-webhook: ${{ secrets.TEAMS_WEBHOOK_URL }}    # optional\n        env:\n          DATABRICKS_HOST: ${{ secrets.DATABRICKS_HOST }}\n          DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}\n          DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}\n          DATABRICKS_WAREHOUSE_ID: ${{ secrets.DATABRICKS_WAREHOUSE_ID }}\n```\n\nTo make the check binding, mark this job as a required status check in your branch protection rules. Then a `needs_human` or `block` verdict stops the merge, whatever the author or an agent decides.\n\nWhen the verdict is `needs_human` or worse, bricksentry can post a short alert: the verdict, the rules that fired, who should review, and a link back to where the check ran.\n\n```\n# bricksentry.yml\nnotify:\n  on: needs_human                         # alert at this verdict or worse\n  slack_webhook: env:SLACK_WEBHOOK_URL    # Slack incoming webhook\n  teams_webhook: env:TEAMS_WEBHOOK_URL    # Teams Workflows \"webhook request\" URL\n  mentions:                               # optional: @-mention reviewers in Slack\n    finance-analytics: S0123ABCD          # a user group id (starts with S)\n    jane@acme.com: U0456EFGH              # a member id (starts with U)\n```\n\nThen check the setup with one command. It sends a clearly labelled sample alert and reads no tables:\n\n```\nbricksentry test-alert\n```\n\n- **Keep webhook URLs secret.** Put them in CI secrets or environment variables (`BRICKSENTRY_SLACK_WEBHOOK` and`BRICKSENTRY_TEAMS_WEBHOOK` work too), never in the file; bricksentry warns if it finds one there.\n- **Links back to the run.** In GitHub Actions the alert links to the PR or run. In a Databricks job it links to the job run. Anywhere else, set`BRICKSENTRY_RUN_URL` (or`notify.run_url` ).\n- **Mentions.** Owners come from Unity Catalog. Map them to Slack ids under`mentions` and the alert @-mentions them; unmapped owners are shown as text. Find an id in Slack under a person's profile or a user group's menu, \"Copy member ID\" / \"Copy group ID\".\n- **Never breaks a run.**`assess` and`preview` send alerts automatically when a webhook is set (`--no-notify` turns them off); in Python call`sentry.notify(assessment)` . If Slack or Teams is busy the alert is retried once, a failure is reported without the webhook URL, and the verdict always stands.\n\n``` python\nfrom bricksentry import Sentry\n\ns = Sentry()                                   # reads ./bricksentry.yml\na = s.assess([\"main.sales.orders\"], pr=\"42\")\nprint(a.verdict, a.notify)\nprint(a.tables[0].summary)\n\ns.preview_write(\"DELETE FROM main.sales.orders WHERE status = 'test'\", \"main.sales.orders\")\nbackend: {type: databricks, warehouse_id: abc123}      # auth: standard Databricks SDK chain\nsandbox_schema: main.bricksentry_sandbox\ncandidate_pattern: \"{catalog}.{schema}_pr_{pr}.{table}\"\nsample_pct: 10                                         # sample_guard re-checks the whole table (on by default)\nassessment_store: ~/.bricksentry/assessments           # or false to keep assessments in memory\nlineage: {type: databricks, depth: 3, lookback_days: 90, column_depth: 2}   # also: exclude, exclude_dropped\ntables:\n  main.sales.orders:\n    keys: [order_id]\n    segment_by: [region]\n    measures: [amount, quantity]   # totals worth checking (see below)\n    paths: [\"src/pipelines/orders/**\", \"resources/orders_job.yml\"]\npolicy: policy.yml            # or builtin:<template>, see \"Policy templates\" below\n```\n\n**Without Unity Catalog** (local DuckDB work, tests, or other platforms), give the lineage in the config instead. Edges map a table to what reads from it; column edges and tags are optional:\n\n```\nlineage:\n  type: static\n  edges:\n    main.sales.orders: [main.sales.daily_revenue, \"dashboard:Revenue Overview\", \"job:finance_close\"]\n  owners:\n    main.sales.daily_revenue: finance-team\n  tags:\n    main.sales.daily_revenue: {certified: \"true\"}\n  column_edges:                       # \"table.column\" -> downstream \"table.column\"s\n    main.sales.orders.amount: [main.sales.daily_revenue.revenue]\n  column_tags:                        # table -> column -> tags\n    main.sales.orders: {customer_email: {pii: \"\"}}\n```\n\nTypos are caught: an unknown key such as `sample_pc` prints a warning that names the right one (`did you mean 'sample_pct'?`) instead of being silently ignored.\n\n**Which totals are checked.** `sum_change_pct_gt` rules and summaries look at measures: by default every numeric column except keys and columns whose names look like IDs, codes, years and other date parts, coordinates, ratings, scores or rates (a 10% drop in `sum(established_year)` only says rows were removed). List `measures` for a table to choose exactly, or set `measures: all` to check every numeric column. A rule that names a column (`when: {column: rating, sum_change_pct_gt: 1}`) always checks it.\n\nA small `policy.yml` with one rule per severity. `pass` is the default: a change gets it when no rule matches.\n\n``` php\ndefault: pass                       # no rule matched -> pass\nnotify_owners_on: needs_human       # tag downstream owners from this level up\n\nrules:\n  - name: type-change               # a column changed type -> warn\n    when: {column_type_changed: true}\n    then: warn\n\n  - name: revenue-moved             # total revenue moved more than 2% -> a human signs off\n    when: {column: amount, sum_change_pct_gt: 2}\n    then: needs_human\n\n  - name: duplicate-keys            # a join fanned out and duplicated keys -> block the PR\n    when: {duplicate_keys: true}\n    then: block\n```\n\n| The change | Verdict | \n|---|---|\n| A filter fix that changes 40 rows, nothing else | `pass` | \n| `order_date` goes from`STRING` to`DATE` | `warn` | \n| A pricing fix moves `SUM(amount)` by 3.1% | `needs_human` | \n| A new join duplicates 40 `order_id` s | `block` | \n\nIf several rules match, the most severe one wins. The full starter policy is in [`examples/databricks/policy.yml`](https://github.com/bricksentry/bricksentry/blob/main/examples/databricks/policy.yml), and `bricksentry init` writes one for you.\n\nStart from a ready-made policy instead of writing one:\n\n| Template | For | \n|---|---|\n| `agent-safe` | Changes written by AI agents: anything beyond a small, clean change goes to a human | \n| `default` | Balanced: block duplicate keys, a human for row loss, dropped columns, PII and certified downstream | \n| `strict` | Production-critical pipelines: small changes need a human, failed checks block | \n| `pii` | Tables with personal data: any change to a `pii` -tagged table, or one feeding it, needs a human | \n| `certified` | Certified or gold tables that dashboards and AI rely on | \n| `finance` | Revenue and money tables: totals can't move quietly, double counting is blocked | \n\nUse one directly with `policy: builtin:finance`, or write an editable copy with `bricksentry init --policy finance`. `bricksentry policies finance` prints it. Tags come from Unity Catalog, so tag your tables `pii` or `certified` for those templates to apply.\n\n`table` (glob) · `tag` · `downstream_tag` · `downstream_kind` · `downstream_count_gt` · `removed_pct_gt` · `added_pct_gt` · `changed_pct_gt` · `row_count_change_pct_gt` · `row_count_drop_pct_gt` · `column_removed` · `column_added` · `column_type_changed` · `duplicate_keys` · `null_keys` · `sum_change_pct_gt` / `null_increase_gt` (optionally with `column`) · `changed_column` (glob) · `changed_column_tag` · `downstream_column_tag` · `any_change` · `diff_error` · `lineage_error`\n\n**Column conditions** look only at the columns the change touched (changed values, new NULLs, dropped or retyped): `changed_column_tag: pii` matches when one of them is tagged `pii` in Unity Catalog, and `downstream_column_tag: certified` when one of them feeds a column tagged `certified`, following `system.access.column_lineage` up to `lineage.column_depth` hops (default 2; `0` turns it off). The default policy asks a human when a `pii` column changes; `strict`, `agent-safe` and `certified` also flag changes that feed certified columns.\n\nAll conditions in a `when` must hold. The most severe matching rule sets the verdict. Unknown conditions are rejected at load time, so a typo can't silently disable a rule.\n\n- `CAN USE` on a SQL warehouse.\n- `SELECT` on the tables you assess and their candidates.\n- `SELECT` on`system.access.table_lineage` (and`system.access.column_lineage` for column-level impact), plus`system.information_schema` for owners, table tags and column tags. Without lineage access bricksentry still diffs, and adds a`lineage-unavailable` warning (blast radius unknown) instead of guessing.\n- `CREATE TABLE` on`sandbox_schema` , for`preview_write` and`check-last-change` .`check-last-change` also needs the table's Delta history (versions not yet removed by`VACUUM` ).\n\n- **Lineage is recorded at runtime.** Brand-new tables have no downstream history yet. Unity Catalog doesn't capture RDD operations, and UDFs hide column-level mapping; column lineage records source-to-target derivations, so a column used only in a filter or join may not appear.\n- **Keys are required.** Tables without a stable key can't be diffed row by row yet (an aggregate-profile mode is planned).\n- **Changed-table detection uses path globs** from config. It doesn't parse Asset Bundles yet.\n- **Validated on one Databricks workspace so far** (serverless, November 2026, on a 2-million-row table with 0.1.4): lineage, Delta rollback plans, previews on shallow clones, every verdict level, and the sampling guard catching a single changed row. The automated tests use DuckDB and a mocked SDK, so please[open an issue](https://github.com/bricksentry/bricksentry/issues) if something behaves differently on yours.\n\n**Top priority: validation across more workspaces.** Classic and serverless compute, AWS, Azure and GCP, and workspaces with restricted system-table access. If you run the quickstart notebook, please [share what you see](https://github.com/bricksentry/bricksentry/issues), whether it works or not.\n\nNext: dbt support, so dbt models on Databricks get the same diff, blast radius and verdict (planned for 0.2) · Asset Bundle parsing for automatic table detection · an aggregate diff for keyless tables · Lakeflow expectations in the verdict · `bricksentry watch` for scheduled drift checks · a Databricks App UI.\n\nBug reports from real pipelines help the most. See [CONTRIBUTING.md](https://github.com/bricksentry/bricksentry/blob/main/CONTRIBUTING.md), and [SECURITY.md](https://github.com/bricksentry/bricksentry/blob/main/SECURITY.md) for reporting vulnerabilities privately.\n\nApache-2.0. Created and maintained by Brahma - Databricks Champion.\n\nbricksentry is an independent open-source project. It is not affiliated with, endorsed by, or sponsored by Databricks, Inc. or Functional Software, Inc. (Sentry). Databricks, Unity Catalog and Lakeflow are trademarks of Databricks, Inc. Delta Lake is a trademark of the Linux Foundation, and Apache Spark and PySpark are trademarks of the Apache Software Foundation. All other names are the trademarks of their respective owners and are used here only to describe compatibility. Thank you.", "url": "https://wpnews.pro/news/show-hn-bricksentry-see-what-a-data-change-does-before-it-ships-databricks", "canonical_source": "https://github.com/bricksentry/bricksentry", "published_at": "2026-11-13 02:31:50+00:00", "updated_at": "2026-11-13 02:51:09.397359+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-tools", "developer-tools", "mlops"], "entities": ["Bricksentry", "Databricks", "Brahmareddy", "DuckDB", "GitHub", "Apache Software Foundation"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-bricksentry-see-what-a-data-change-does-before-it-ships-databricks", "markdown": "https://wpnews.pro/news/show-hn-bricksentry-see-what-a-data-change-does-before-it-ships-databricks.md", "text": "https://wpnews.pro/news/show-hn-bricksentry-see-what-a-data-change-does-before-it-ships-databricks.txt", "jsonld": "https://wpnews.pro/news/show-hn-bricksentry-see-what-a-data-change-does-before-it-ships-databricks.jsonld"}}