When Metrics Said Fine and the Photo Was Ruined: Three Mask Bugs a Human Review Caught

Our watermark remover's synthetic bench was green. All thirteen acceptance cases passed their PSNR/SSIM gates, ring-damage was zero, and the precision/recall numbers hadn't moved in weeks. Then we ran a batch of real user photos through a new human-review loop — three columns per case (source, mask, result) where you circle what looks wrong and tag it — and within one batch the answer came back: the masks were eating dinner.

This post is about what the review found, why none of our metrics could see it, and the three fixes that came out of one afternoon of looking at actual outputs.

The review loop that earned its keep

Quick context: our batch pipeline ([batch-eval]) runs the real product code path — same detection, same mask builder, same inpainting engine — over a folder of real user cases and writes per-case artifacts. The new part is a local review server that renders three columns (source, mask overlay, result) and lets a human circle regions on the result with a tag (residue, halo, blur, over-fill…) and a severity. The verdict lands as structured JSON per batch, so "where exactly is it wrong" stops being prose and starts being data.

First batch of real photos, first findings — and they were not subtle:

On the left, a dinner photo where our "watermark mask" covered the frying pan, the bread, and the table — while the actual watermark at the bottom was only partially removed. On the right, the same photo after the fixes. Here's the mask itself, because this one is where the mechanism shows:

That scattered red mess isn't the mask "detecting too much" — it's a bug flinging pixels to the wrong rows entirely.

Bug 1: the 2048-cap stride mismatch

Our glyph mask builder caps its working grid at 2048px on the long side for speed. A 2560×1920 photo produces a 2048×1536 grid. The bug: when copying the glyph bitmap back into a full-image-sized mask, the code walked the grid linearly as if its row width were the image's row width — 2560 instead of 2048. Every row past the first landed at the wrong x-offset, so watermark glyph pixels rained down across the whole image in horizontal bands. The table-band artifacts in the food photo are literally the watermark's letter shapes, re-drawn onto the frying pan.

Every photo over 2048px was affected — which is to say, most photos from a modern phone. The bench never saw it because every synthetic case is smaller than the cap. One line of defense now covers it: when the glyph grid doesn't match the image dimensions, we remap proportionally with nearest-neighbor instead of trusting the stride. The food photo's watermark line went from 35.4% actually-changed to 75.1%, and the 63 guard-reverts it used to trigger went to zero.

Bug 2: flood fill with no gate

The second photo was worse: a portrait where the bottom-left corner of the result was filled with skin-colored hallucination over a large solid block. The chain: the text detector produced a big false-positive box touching the image edge; the edge-touching position polluted the ring-band median we use for color-difference candidates; the candidates collapsed; the shape-refinement flood fill — with nothing to stop it — solidly covered 85% of the region. An entire quadrant of someone's photo, repainted.

The fix is a two-condition gate: if flood coverage exceeds 60% of the region and the flood-to-seed ratio exceeds 8, we discard the flood and fall back to the seed core. The ratio matters — our first attempt gated on coverage alone and immediately false-flagged a legitimate giant text band on clouds (coverage 71%, but a flood/seed ratio of 3.0 = dense real letters). Real leaks run at 17×, real text stays under 7.4×. Two conditions, no casualties.

Bug 3: the bridged column that defeated the feather gate

The third bug hid in the opposite place — the mask was right, the pixels were covered, and letters still survived. A vertical watermark's letters, dilated and grown, bridged into a single solid 49×451px column. Our compactness gate (area/boundary ratio ≥ 12 → apply 8px inward feather before pasting) measured that column at 16.7 and passed it — so the column got feathered, and the stroke edges bled back into the result at ~20% opacity. A crisp vertical "CELEB.TO" watermark left a ghost outline precisely because its letters fused into one fat bar.

The fix is a solidity gate: feather only when the region is truly rectangle-like (area/bounding-box ≥ 0.92 — real rectangular selections measure ≥97%, glyph columns ≤90%). No mask expansion, no over-painting risk — ring-damage and precision metrics are bit-identical before and after.

The fourth thing we learned: not every residue is a removal bug

One case in the batch had a perfectly removed headline and a faint line of subtitle text still visible. Our first instinct was "the mask edge didn't dilate enough" — the pixel evidence said otherwise: the subtitle line (7px tall) sat outside the only detection box, and the OCR probability map there read exactly 0.000. The detector never fired; no mask tweak could have fixed it. Bonus lesson inside the lesson: the "ghost score" that flagged it at 0.536 was measuring a 16px ring around the box — which included the unprocessed subtitle. The metric was contaminated by its own padding.

That's the product principle we now hold: a miss is recoverable, a false positive isn't. If detection misses a faint watermark, the user brushes over it manually and the result is fine. If detection eats their photo, there's no undo for trust.

The scorecard

Case	Metric	Before	After
Food photo (2560px)	watermark line changed%	35.4	75.1
Food photo	guard reverts	63	0
Portrait	destroyed-block changed%	77.2	9.8
Cloud text band	tile seam strength	33.65	0.93
Regression case	result vs. pre-fix	—	bit-identical
Synthetic bench	13-case gates	PASS	PASS (per-case diff = 0)

The one thing we did not fix on purpose: a product-logo embossed on a pan handle ("GUAM") is now detected and removed cleanly as text — which is geometrically correct and semantically wrong, because it isn't a watermark. Content-vs-watermark judgment is a separate problem, tracked separately.

Try it

Everything above ships in the current free watermark remover — the mask discipline, the gates, all of it running in your browser. If you're curious how the same engine behaves on video, our video watermark remover shares the inpainting core, and the quality dashboard publishes the synthetic bench numbers mentioned here.

---

Part 12 of the ClearPix engineering series — how we build free, private, in-browser media tools at clearpix.org.