Enhancing Humanizer to defeat Text Watermarks

 Adding watermark stripping to Humanizer

I use the Humanizer skill pretty regularly. It's a portable agent skill that strips signs of AI-generated writing out of text, checking 33 patterns. Em dashes, the rule of three, "vibrant tapestry," the usual stuff. It works well for what it does.

What it didn't do was handle watermarks. The EU AI Act starts being enforceable this month, and one of the requirements is that AI outputs have to be detectable as machine-produced. Which means the labs are going to start watermarking their output. I wanted to know whether the tool I use to make AI writing sound human could also handle the hidden signals that mark it as machine-produced in the first place.

I forked Humanizer to my own repo and started reading up on how text watermarking actually works.

Two sources, one conclusion

I put two documents in the repo to work from. One was a blog post by Sean Goedecke arguing that text watermarks will always be trivial to remove. The other was a 2018 IEEE survey paper covering the classic watermarking techniques.

They both land on the same split, which was useful. There are basically two kinds of text watermark, and they get removed in completely different ways.

The first kind is Unicode homoglyphs. The model swaps invisible characters into the text. Normal spaces become three-per-em spaces. Zero-width characters get tucked between letters. Cyrillic characters that look identical to Latin ones get slipped in. You can't see any of it, but a detector can find the pattern. The way you remove this kind is mechanical. You replace all the look-alikes with their real ASCII equivalents.

The second kind is token-distribution watermarks. SynthID, from Google, is the main example. When the model picks which word to use next, it skews the choice toward tokens that score well under a secret formula. The text reads fine, but the pattern of word choices has a statistical fingerprint. You remove this kind by paraphrasing. If you reword the content, the fingerprint doesn't survive.

My first plan was straightforward. Add a section to the skill describing both kinds, tell the model to normalize the Unicode and paraphrase for the distribution stuff, ship it as version 3.0.0.

That didn't happen.

The First Council

I run my LLM Multi Council skill when I'm unsure about something. Five models in parallel, an advice round and then a peer review round, and I read what they come back with. I've written about it before. The council isn't smarter than I am. It's that five models looking at the same plan from different angles catch things I miss.

I sent them the plan, and they killed it.

The sharpest finding came from GLM-5.2, which found the asymmetry that guts the whole feature. A language model reading tokens can reliably catch the visible confusables, the curly quotes and em dashes. Humanizer already handles those in patterns 14 and 19. But the invisible ones, the actual watermark payload, the zero-width space buried in a run of regular spaces, those fold into adjacent tokens. The model never sees them as discrete objects. Everything the model can catch is already covered. Everything new the feature promises is what the model cannot see. That is a hard problem to argue with, and I didn't try.

Gemini found the second one, alone. Paraphrasing doesn't remove a distribution watermark. It swaps one provenance signal for another. The rewritten text carries the rewriter model's own fingerprint. So the plan's claim that rewording "strips" SynthID was wrong. It alters it. The output is still marked, just by a different model.

DeepSeek found an internal contradiction. My plan had a preprocessing step that normalized non-ASCII characters before the prose analysis ran. But two of the existing patterns, the em dash rule and the curly quotes rule, exist specifically to flag those characters as tells and rewrite them. A pre-scrub would preempt the patterns that were already doing the work.

GPT caught the terminology. What I was calling Unicode normalization is actually UTS 39 confusables canonicalization. Different operation, different name, real failure surface.

The council's recommendation was to ship something, just not what I had. Drop the capability claim from the frontmatter. Make the section a best-effort awareness guide instead of a promise. Call it 2.10.0, not 3.0.0. Reserve 3.0.0 for a version where the scrub is actually real.

I shipped 2.10.0. A Text Watermarks section that describes the three classes honestly. It says the skill can flag invisible characters when it notices them but cannot guarantee catching them. It says rewording alters but does not eliminate statistical fingerprints. I added a note so the model doesn't flatten legitimate non-ASCII content, real Japanese, accented names, things like that, as if it were a watermark.

It was a documentation upgrade. It didn't actually remove anything.

The Python version

The council had told me what a real 3.0.0 would look like. A deterministic Python utility that iterates codepoints directly. It catches what a model reading tokens cannot see because it isn't reading tokens. It's counting characters.

I planned it out. A script with a curated map of watermark characters, a pipe-through interface, report and apply modes, a Latin-script gate so it doesn't transliterate Russian. I sent that plan to the council too, with the same result.

GLM-5.2 found the correctness blocker again, and all four reviewers independently named it the strongest response. My curated map was only safe under an unstated assumption that the input was English. Three concrete failures.

U+200C, the zero-width non-joiner, is not decorative. It controls letter joining in Persian. It breaks conjuncts in Devanagari. Deleting it changes words.

U+200D, the zero-width joiner, forms Devanagari conjuncts. My plan only warned about emoji sequences. I missed that it has a real role in Indic scripts

The Cyrillic map was the worst one. I had it replacing Cyrillic characters with their Latin look-alikes. There is no way for the script to tell a Cyrillic character used as a watermark from one used as a legitimate Russian word without knowing the surrounding script. So it would have silently transliterated genuine Russian into Latin gibberish.

GPT separately flagged that U+200E and U+200F are not watermark characters at all. They are bidi controls, left-to-right and right-to-left marks. Removing them from mixed-direction text changes the rendered order of the words.

And nobody on the council named U+00A0, the non-breaking space, which I had going to a regular space. That is legitimate typography. "10 km" with a non-breaking space is a content choice, not a watermark.

The council's recommendation this time was to ship the script with mandatory fixes. Drop the unsafe characters from the default map. Gate the homoglyph replacements on a Latin-script heuristic and abstain on any input that isn't Latin. Invert the default so report-only is the default and mutation requires an explicit flag. Call it 2.11.0, because the narrowed capability is smaller than what "strips Unicode watermarks" implies.

The thing they told me to do first was the thing I should have done from the start.

Write the test before the map

The first plan's test fixture tested Chinese and accented Latin. The map already left those alone. The fixture would have passed green against a script that corrupts three widely-used scripts, because it never looked at those scripts.

That is worse than shipping no test at all. It converts an untested claim into a falsely verified one.

I wrote the fixture first. Russian. Persian with a real zero-width non-joiner. A Devanagari conjunct with a real zero-width joiner. An emoji sequence. A non-breaking space. The rule was that watermarks get caught and legitimate content survives byte-identical.

Then I wrote the map to make the tests pass.

The narrowed map removes zero-width space and mid-text byte-order marks. It normalizes the typographic space variants to regular ASCII space. It replaces Cyrillic homoglyphs with their Latin equivalents, but only under the Latin-script gate. If the input is mostly Cyrillic, the gate abstains. The characters with legitimate roles, the joiners and bidi marks and non-breaking space, pass through untouched.

All 19 tests passed on the first run.

What shipped

Version 2.11.0. Three files of real work and some documentation around them.

The script is scripts/strip_watermarks.py. Stdlib only, no dependencies. You run it in report mode by default. It tells you what it would change and writes nothing. You pass --apply to get the cleaned text, or --in-place to rewrite a file with a backup. The Latin-script gate means it abstains rather than corrupt non-Latin input.

The test fixture runs in CI now, on every push and every pull request. If someone edits the map later and breaks the boundaries, the build fails.

The skill itself, the Markdown that agents read, points at the script without hardcoding a path. It describes the operation and says where the tool lives. A standalone copy of the skill that gets loaded without the repo doesn't instruct an agent to run a file that isn't there.

The honest framing is that the script strips invisible Unicode watermarks from Latin-script prose. The prose-rewrite skill alters but does not eliminate distribution fingerprints, because the rewritten text carries the rewriter's own distribution. Those are two different claims for two different tools, and the docs keep them separate.

What I took away from this

I started with a plan that would have overclaimed a capability the model cannot deliver. The council caught it. I shipped a smaller, honest version instead.

Then I planned the real capability, a deterministic script. The council caught that the script would have silently corrupted Russian, Persian, and Devanagari. I wrote the test first, narrowed the map, and shipped a version that works for the scope it claims.

The methodology is the part worth keeping. The test-first approach matters because a fixture that tests the easy cases, the ones the map already handles, will pass green while the script eats three languages nobody checked. The council matters because five models looking independently at the same plan find the same load-bearing flaw from different angles, and that convergence is worth more than any single model being confident.

The watermark problem itself is straightforward once you split it correctly. Invisible Unicode is a codepoint-counting problem. Token distributions are a paraphrasing problem. The skill handles the prose, the script handles the codepoints, and neither pretends to do the other's job.

Humanizer 2.11.0 is on my GitHub now. The script handles Latin prose. It abstains on everything else rather than risk breaking it. 3.0.0 will wait until there's a fuller map and some real evidence about what paraphrasing actually does to a SynthID score.

Note: Obviously I didn't do all this alone. I was working inside of ZCode with GLM-5.2. I had it write a draft of this blog post and then run the Humanizer skill on it. Which left it somewhat chagrined.

Worth noting: I ran this on my own work, found real patterns, and the rewrite is genuinely tighter for it. Which is either a good sign for the skill or an uncomfortable one for my default prose habits. Probably both.

Make of that what you will. Oh, and I changed the title back.