Testing AI image watermarks: C2PA and SynthID #
C2PA and SynthID are two technologies used to identify AI-generated images. C2PA (Content Credentials) is pretty simple, as it just adds signed metadata to a file. SynthID, on the other hand, is a different beast. It adds a watermark directly into the pixels of images.
I decided to experiment with these to understand more.
Test 1: Plain rectangle
Prompt:
Let's inspect the image:
c2patool ai-generated-images/pure-white-blank-canvas1.png --info
The important part here is "One manifest". Let's extract the manifest and search for interesting fields.
It is clear that this image has C2PA metadata.
Stripping is pretty easy with magick. Let's inspect again, after stripping.
At this point, the metadata is gone.
</>javascript
{
"type": "c2pa",
"outcome": "not_detected"
}
Did we change the image, though? Let's decode these files into raw RGBA pixels and check the hash.
</>javascript
magick rectangle-original.png rgba:- | shasum -a 256
magick rectangle-stripped.png rgba:- | shasum -a 256
// Both output ef994f090d047a181138e41a0e44bdec7cc22e4982bda529eecc2240c21c3cb2
They both output the same hash, so the images are identical.
Interestingly enough, OpenAI does not seem to add SynthID in simple shapes like these. Let's generate something better.
Test 2: Apple
I used the prompt "Draw a red apple, realistic studio photo, on a plain white background":
Let's check for provenance signals.
We could either use the web's version or just call the API.
I stripped the metadata and ran the check again.
</>javascript
{
"object": "content_provenance_check",
"results": [
{
"type": "c2pa",
"generated_at": null,
"issuer": null,
"model": null,
"outcome": "not_detected",
"validation_state": "not_present"
},
{
"type": "SynthID",
"generated_at": null,
"model": null,
"outcome": "detected"
}
]
}
This is expected, as removing metadata does not remove SynthID.
What happens with the crop?
magick Pure-White-Canvas.png -gravity center -crop 512x512+0+0 +repage ai_crop_512.png
SynthID survives.
JPEG conversion?
magick apple-stripped.png -quality 75 apple-q75.jpg
SynthID survives.
Maybe a diffusion attack, then?
...
pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
model_id,
torch_dtype=torch.float16 if device == "mps" else torch.float32
)
prompt = "A high-quality, detailed, photorealistic macro photograph of a single glossy red apple with a brown stem, natural yellow streaks, isolated on a plain white background with a soft shadow"
image = pipe(
prompt=prompt,
image=init_image,
strength=0.25,
guidance_scale=7.5
).images[0]
...
That is, convert the original image to numbers (latents), add some noise, then denoise based on a prompt.
But this is cheating. I am essentially regenerating the image (from the original), not removing SynthID. This type of diffusion reconstruction will not exactly preserve the original image (texture, color, edges, lighting, etc.).
Guess I am back to analyzing pixels, and there is no better candidate for this than a nearly white image.
Test 3: Plain white background
Prompt:
How about we compare this plain white image with a plain white image (that I can create locally)?
Let's analyze the AI image for spatial patterns or uniformities.
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
def fft_spectrum(channel):
centered = channel - channel.mean(dtype=np.float64) # zero mean
h, w = centered.shape
window = np.outer(
np.hanning(h),
np.hanning(w),
)
windowed = centered * window
fft_raw = np.fft.fft2(windowed)
fft_shifted = np.fft.fftshift(fft_raw)
magnitude = np.log1p(np.abs(fft_shifted))
return magnitude
def analyze_zero_mean_residual(
image_path,
output_path="zero_mean_analysis.png",
amp_factor=50,
):
img = Image.open(image_path).convert("RGB")
ai = np.array(img, dtype=np.float64)
height, width, channels = ai.shape
print("--- Image ---")
print(f"Path: {image_path}")
print(f"Dimensions: {width} x {height}")
print(f"Shape: {ai.shape}")
print("--- Raw Pixel Statistics ---")
print(f"Overall min: {ai.min():.0f}")
print(f"Overall max: {ai.max():.0f}")
print(f"Overall mean: {ai.mean(dtype=np.float64):.6f}")
unique_values = np.unique(ai.astype(np.uint8))
print(f"Unique channel values: {len(unique_values)}")
if len(unique_values) <= 30:
print(f"Values: {unique_values}")
else:
print(
f"Value range: "
f"{unique_values[0]} ... {unique_values[-1]}"
)
mean_rgb = ai.mean(
axis=(0, 1),
dtype=np.float64,
)
print("--- Mean RGB ---")
print(f"R = {mean_rgb[0]:.9f}")
print(f"G = {mean_rgb[1]:.9f}")
print(f"B = {mean_rgb[2]:.9f}")
channel_min = ai.min(axis=(0, 1))
channel_max = ai.max(axis=(0, 1))
print("--- Channel Ranges ---")
print(
f"R: {channel_min[0]:.0f} to {channel_max[0]:.0f}"
)
print(
f"G: {channel_min[1]:.0f} to {channel_max[1]:.0f}"
)
print(
f"B: {channel_min[2]:.0f} to {channel_max[2]:.0f}"
)
if np.any(mean_rgb < channel_min) or np.any(mean_rgb > channel_max):
raise RuntimeError(
"Calculated RGB mean is outside the observed pixel range."
f"Mean: {mean_rgb}"
f"Min: {channel_min}"
f"Max: {channel_max}"
)
residual = ai - mean_rgb
residual_channel_means = residual.mean(
axis=(0, 1),
dtype=np.float64,
)
print("--- Residual ---")
print(f"Min: {residual.min():.9f}")
print(f"Max: {residual.max():.9f}")
print(
f"Overall mean: "
f"{residual.mean(dtype=np.float64):.12f}"
)
print("--- Residual Mean By Channel ---")
print(f"R = {residual_channel_means[0]:.12f}")
print(f"G = {residual_channel_means[1]:.12f}")
print(f"B = {residual_channel_means[2]:.12f}")
if not np.allclose(
residual_channel_means,
0.0,
atol=1e-10,
):
raise RuntimeError(
"Residual was not centered correctly."
f"Residual means: {residual_channel_means}"
)
channel_names = ["Red", "Green", "Blue"]
print("--- Per-channel Residual Statistics ---")
for index, name in enumerate(channel_names):
channel = residual[:, :, index]
print(
f"{name:>5} | "
f"mean={channel.mean(dtype=np.float64): .12f} "
f"std={channel.std(dtype=np.float64): .9f} "
f"min={channel.min(): .9f} "
f"max={channel.max(): .9f}"
)
rms = np.sqrt(
np.mean(
residual ** 2,
dtype=np.float64,
)
)
print(f"Overall residual RMS: {rms:.9f}")
red_fft = fft_spectrum( residual[:, :, 0] )
green_fft = fft_spectrum( residual[:, :, 1] )
blue_fft = fft_spectrum( residual[:, :, 2] )
all_fft_values = np.concatenate([ red_fft.ravel(), green_fft.ravel(), blue_fft.ravel(), ])
fft_vmin = np.percentile( all_fft_values, 1.0, )
fft_vmax = np.percentile( all_fft_values, 99.5, )
print("--- Shared FFT Display Scale ---")
print(f"vmin = {fft_vmin:.9f}")
print(f"vmax = {fft_vmax:.9f}")
residual_vis = np.clip( 128.0 + residual * amp_factor, 0, 255, ).astype(np.uint8)
flat_rgb_uint8 = np.clip( np.round(mean_rgb), 0, 255, ).astype(np.uint8)
flat_vis = np.empty_like( ai, dtype=np.uint8, )
flat_vis[:] = flat_rgb_uint8
fig, axes = plt.subplots( 2, 3, figsize=(18, 11), constrained_layout=True, )
axes[0, 0].set_title( "Original AI image" )
axes[0, 0].imshow(img)
axes[0, 0].axis("off")
axes[0, 1].set_title( "Mean-matched flat reference" f"Display RGB {tuple(flat_rgb_uint8.tolist())}" )
axes[0, 1].imshow(flat_vis)
axes[0, 1].axis("off")
axes[0, 2].set_title( f"Signed residual ×{amp_factor}" "128 = zero difference" )
axes[0, 2].imshow( residual_vis )
axes[0, 2].axis("off")
fft_image = axes[1, 0].imshow( red_fft, cmap="magma", vmin=fft_vmin, vmax=fft_vmax, ) # red
axes[1, 0].set_title( "FFT: red residual" )
axes[1, 0].axis("off")
axes[1, 1].imshow( green_fft, cmap="magma", vmin=fft_vmin, vmax=fft_vmax, ) # green
axes[1, 1].set_title( "FFT: green residual" )
axes[1, 1].axis("off")
axes[1, 2].imshow( blue_fft, cmap="magma", vmin=fft_vmin, vmax=fft_vmax, )
axes[1, 2].set_title( "FFT: blue residual" )
axes[1, 2].axis("off")
fig.colorbar( fft_image, ax=axes[1, :], fraction=0.025, pad=0.02, label="log(1 + FFT magnitude)", )
plt.savefig( output_path, dpi=200, bbox_inches="tight", )
plt.close(fig)
print(f"Saved: {output_path}")
if __name__ == "__main__":
analyze_zero_mean_residual(
image_path="ai-generated-images/pure-white-blank-canvas1.png",
output_path="zero_mean_analysis.png",
amp_factor=50,
)
Output:
There is a lot of data to unpack, but we are mostly interested in mean RGB values and the residual standard deviation. Green seems to have a lot more residual energy, but we can't conclude that it is carrying the watermark.
We can use Pillow to generate a-like image. We will get the exact value distribution per channel and generate a random image with the same histogram.
random_control = np.empty_like(source, dtype=np.uint8)
channel_names = ["Red", "Green", "Blue"]
for i, channel_name in enumerate(channel_names):
channel = source[:, :, i]
unique_vals, counts = np.unique(channel, return_counts=True)
probs = counts / counts.sum()
random_channel = rng.choice( unique_vals, size=(h, w), p=probs, ).astype(np.uint8) random_control[:, :, i] = random_channel
Image.fromarray(random_control).save(output_path)
Checking this newly generated image through the provenance API shows "SynthID not detected". So, just having the matching RGB distributions is not enough. The AI image contains spatial information that matters.
We will do one final spectral comparison here.
...
for i, channel_name in enumerate(channel_names):
ai_mag, ai_log, ai_power = fft_data( ai_residual[:, :, i] )
control_mag, control_log, control_power = fft_data( control_residual[:, :, i] )
ai_fft.append(ai_log)
control_fft.append(control_log)
ai_entropy = spectral_entropy(ai_power)
control_entropy = spectral_entropy(control_power)
ai_peak = peak_prominence(ai_power)
control_peak = peak_prominence(control_power)
corr = spectrum_correlation( ai_log, control_log, )
metrics.append({
"channel": channel_name,
"ai_entropy": ai_entropy,
"control_entropy": control_entropy,
"ai_peak": ai_peak,
"control_peak": control_peak,
"correlation": corr,
})
radial_ai.append( radial_power_spectrum(ai_power) )
radial_control.append( radial_power_spectrum(control_power) )
...
This is an interesting result.
The AI image and control are identical but very different spectrally.
- Spectral entropy: AI image: ~0.889-0.900 Control image: ~0.970
- Peak prominence: AI image: ~716-1078 Control image: ~19-25
- Spectrum correlation: Zero
Running it a few more times showed:
| Metric | Run 1 | Run 2 | Run 3 | Controls |
|---|---|---|---|---|
| Red entropy | 0.9000 | 0.9011 | 0.8895 | ~0.9704 |
| Green entropy | 0.8893 | 0.8818 | 0.8702 | ~0.9704 |
| Blue entropy | 0.8970 | 0.8896 | 0.8817 | ~0.9704 |
| Red peak | 715.7 | 554.5 | 485.2 | ~21-25 |
| Green peak | 1077.8 | 884.5 | 633.6 | ~19-21 |
| Blue peak | 777.5 | 718.3 | 811.3 | ~19-24 |
In these samples, SynthID images show much lower spectral entropy than their matched random controls.
I am now curious to see how much shaving off or limiting high-frequency spikes affects the image's appearance. We can convert each color channel into the frequency domain (via a 2D Fast Fourier Transform) and shift the zero-frequency component to the center. Then we iterate over some magnitude thresholds, cap the ones that exceed and reconstruct the image back.
import numpy as np
from PIL import Image
THRESHOLDS = [300, 500, 750, 1000, 1500, 2000, 2500, 3000]
def evaluate_threshold(arr, threshold, dc_radius=15):
h, w, c = arr.shape
reconstructed = np.zeros_like(arr)
y, x = np.ogrid[:h, :w]
cy, cx = h // 2, w // 2
dc_mask = ((x - cx) ** 2 + (y - cy) ** 2) <= dc_radius ** 2
per_channel_affected = []
max_imag_residue = 0.0
for i in range(c):
channel = arr[:, :, i]
fft_raw = np.fft.fft2(channel)
fft_shifted = np.fft.fftshift(fft_raw)
magnitude = np.abs(fft_shifted)
phase = np.angle(fft_shifted)
capped_mask = (magnitude > threshold) & (~dc_mask)
affected_count = np.sum(capped_mask)
affected_fraction = (affected_count / magnitude.size) * 100.0
per_channel_affected.append(affected_fraction)
magnitude[capped_mask] = threshold
fft_capped = magnitude * np.exp(1j * phase)
inverse = np.fft.ifft2(np.fft.ifftshift(fft_capped))
max_imag_residue = max(
max_imag_residue,
np.max(np.abs(inverse.imag))
)
reconstructed[:, :, i] = inverse.real
overall_affected_fraction = float(np.mean(per_channel_affected))
diff_float = reconstructed - arr
rmse_float = np.sqrt(np.mean(diff_float ** 2))
mae_float = np.mean(np.abs(diff_float))
max_abs_float = np.max(np.abs(diff_float))
arr_8bit = np.clip(np.round(arr), 0, 255).astype(np.uint8)
recon_8bit = np.clip(np.round(reconstructed), 0, 255).astype(np.uint8)
diff_8bit = arr_8bit.astype(np.float64) - recon_8bit.astype(np.float64)
rmse_8bit = np.sqrt(np.mean(diff_8bit ** 2))
pixel_changed_mask = np.any(arr_8bit != recon_8bit, axis=2)
pixels_changed_pct = (np.sum(pixel_changed_mask) / (h * w)) * 100.0
subpixels_changed_pct = (
np.sum(arr_8bit != recon_8bit) / arr_8bit.size
) * 100.0
return {
"threshold": threshold,
"affected_overall_pct": overall_affected_fraction,
"affected_red_pct": per_channel_affected[0],
"affected_green_pct": per_channel_affected[1],
"affected_blue_pct": per_channel_affected[2],
"rmse_float": rmse_float,
"mae_float": mae_float,
"max_abs_float": max_abs_float,
"rmse_8bit": rmse_8bit,
"pixels_changed_pct": pixels_changed_pct,
"subpixels_changed_pct": subpixels_changed_pct,
"max_imag_residue": max_imag_residue,
}
def run_sweep(input_path, dc_radius=15):
print("--- Running Threshold Sweep ---")
print(f"Image: {input_path}")
print(f"DC protection radius: {dc_radius}")
img = Image.open(input_path).convert("RGB")
arr = np.array(img, dtype=np.float64)
print(
f"{'Threshold':<10} | "
f"{'FFT Avg':<9} | "
f"{'R%':<7} | "
f"{'G%':<7} | "
f"{'B%':<7} | "
f"{'Float RMSE':<11} | "
f"{'8-bit RMSE':<11} | "
f"{'RGB Px':<9} | "
f"{'Subpx':<9} | "
f"{'Max Imag'}"
)
print("-" * 120)
for t in THRESHOLDS:
result = evaluate_threshold(arr, t, dc_radius=dc_radius)
print(
f"{result['threshold']:<10} | "
f"{result['affected_overall_pct']:>7.2f}% | "
f"{result['affected_red_pct']:>6.2f}% | "
f"{result['affected_green_pct']:>6.2f}% | "
f"{result['affected_blue_pct']:>6.2f}% | "
f"{result['rmse_float']:>10.4f} | "
f"{result['rmse_8bit']:>10.4f} | "
f"{result['pixels_changed_pct']:>7.2f}% | "
f"{result['subpixels_changed_pct']:>7.2f}% | "
f"{result['max_imag_residue']:.2e}"
)
if __name__ == "__main__":
run_sweep(
"ai-generated-images/pure-white-blank-canvas1.png",
dc_radius=15,
)
This result ruled out my naive but tempting shortcut.
At 1500, I am changing about 5% of FFT bins, but only ~0.5% of RGB pixels survive quantization. But at 2000, I am changing about 2.6% of the FFT bins, but the 8-bit image is unchanged. This probably means threshold sweeps aren't a good way to target recurring peaks.
The strong FFT peaks are interesting, but blindly capping them was not a good way to isolate whatever the detector was seeing.
Comparing frequencies
Finally, I took the FFT area that were very strong in the AI image and treated them as candidates. I can now compare them with similar non candidate frequencies.
First is Frequency radius. I start from the center (FFT), and compare the radius of AI (candidate) and control frequencies. The result was:
- Median radius difference: 0.73 bins
- 95th percentile: 2.67 bins
So they were pretty close in frequency location.
Then I matched FFT magnitude. For every candidate, I looked for a control frequency within 0.95x and 1.05x (FTT) magnitude. The result was that the median control magnitude was about 95.6% of the candidate magnitude.
After matching radius and magnitude, I got RMSE of candidate: 0.105326 and RMSE of control: 0.100709, i.e. just 4.6% difference.
In other words, once I matched both sides properly, most of that difference disappeared.
Final words
I started this experiment thinking the watermark was simple stuff. But the data decided to consume my evening. The takeaway is that the AI-generated images contained small pixel variations, and they do not behave like random noise. When I compared them with random controls, the AI-generated ones showed lower spectral entropy and near-zero spectrum correlations with our controls. And my attempt to suppress the high-energy FFT coefficients didn't turn out the way I hoped.
So I didn't find the SynthID frequency (yet). But there was clearly some spectral structure hiding inside a plain white image.