Originally published on my own site.
I make a desktop AI agent. I recorded a demo of it writing files, and the
approval dialog showed /Users/<my-username>/Downloads/…
in plain text.
Blurring it out should have been a five-minute job. I checked the result by eye three times and missed a leak every time. The fix that finally worked was
The conclusion first: you cannot verify a redaction by looking at it. The
scan needs to be as much a part of the pipeline as the blur.
The naive version: open one frame where the path is visible, measure the box,
and paint over it.
delogo=x=450:y=300:w=370:h=44:enable='between(t,99.9,102.4)'
The result had the username visible for the first one or two seconds.
macOS dialogs animate in. They keep moving until they settle.
| time (s) | dialog top | dialog left |
|---|---|---|
| 98.4 | 372 | 458 |
| 98.8 | 230 | 180 |
| 99.2 | 19 | 462 |
| 99.6 | 71 | 407 |
| 100.0 | 94 | |
| 398 | ||
| 102.4 | 94 | 398 |
A box measured at 100.0 s lands nowhere near the frame at 98.4 s — and the text
is legible the whole way in.
You can sidestep this by trimming each cut to start after the dialog settles.
That was not available here: the whole point of the video was that it is
uncut. So the box has to follow the text.
The dialog moves, but the font size does not. So take the rendered
/Users/sa
from a settled frame as a stencil, and slide it over every
frame to find the best match.
Done directly, that is 2 million positions × 5,700 pixels per frame. Python
will not finish. Correlation is a convolution, and convolution is one
multiplication in the frequency domain.
def correlate(field, kernel):
"""Convolution via FFT. The direct loop does not finish."""
fh, fw = field.shape
F = np.fft.rfft2(field)
K = np.fft.rfft2(kernel[::-1, ::-1], s=(fh, fw))
out = np.fft.irfft2(F * K, s=(fh, fw))
kh, kw = kernel.shape
return out[kh - 1:, kw - 1:] # align the origin to top-left
Score on two things: misses and over-eager matches.
def find(img, tmpl):
d = dark(img)
hit = correlate(d, tmpl) # stencil ink ∩ frame ink
load = correlate(d, np.ones_like(tmpl)) # total ink inside the window
ink = tmpl.sum()
score = hit / (load + 0.35 * ink + 1e-6)
score[hit < 0.55 * ink] = 0 # reject misses outright
y, x = np.unravel_index(int(np.argmax(score)), score.shape)
return int(x), int(y), float(score[y, x])
hit
on its own latches onto any dense block of dark pixels — a code excerpt,
for instance. Dividing by load
penalises windows carrying ink the stencil
does not account for. The 0.35 * ink
term in the denominator keeps the score
from exploding in near-empty regions.
Feed the per-frame boxes into delogo
's enable
. Merge runs at the same
position, or you end up with 500 filters and the graph stops building.
The one above. The fixed box does not land.
Tracking still leaked. The dialog appears translucent first.
My dark()
threshold was 120, so the faint text on those frames did not count
as ink and tracking never started there. Faint text is still readable text.
def dark(img):
return (img < 150).astype(np.float32)
I also extend each run 0.3 s earlier and 0.2 s later.
This is the one that mattered.
At the end of the video the agent opens the generated HTML in a browser.
The address bar showed file:///Users/<my-username>/… for 23 seconds.
I had been staring at dialogs. It never occurred to me to look there.
You do not find what you did not think to look for.
At that point I stopped trusting the redaction and started scanning the output.
TEMPLATES = [
(100.0, (452, 302, 150, 38)), # write_file … larger type
(148.7, (850, 512, 90, 28)), # open_path … smaller type
(152.0, (940, 12, 100, 30)), # address bar … different again
]
One stencil is not enough. Different dialogs are different widths and set
their text at different sizes. With a single stencil the smaller one never
matched at all and instead latched onto false peaks around 0.30.
Set the verification threshold higher than the tracking one. Measured: real
hits scored 0.64–0.74, false peaks topped out at 0.46, so 0.55 separates them
cleanly. At 0.45 the scan flagged 179 frames of browser iconography and buried
the real result.
The scan found one more leaked frame.
The leak was at 113.933… s.
I was tracking at 20 fps, so I looked at 113.90 and 113.95. The output is
30 fps, so the frame that actually ships is 113.933…. Nobody looked at it.
WINDOWS = [
(113.4, 118.2, 30), # animating — sample at the output rate
(150.8, 174.8, 5), # static — coarse is fine
]
Sample the moving stretches at the output rate and leave the static ones
coarse. Running everything at 30 means 4,000 FFTs and no result.
Final pass: 2,225 frames scanned, nothing found.
None of this is needed if the app never puts a username on screen in the first
place, which is the real fix and the one I shipped. I kept the scan anyway.
You do not find what you did not think to look for.
The app in the recording is Wisp — a
desktop agent with a 3D body that runs commands and writes files, and shows you
exactly what it is about to do before it does it.