← All posts

How we test Docwake

The four test sets, what counts as a pass, and where the agent still fails.

Close-up of the fan of type bars inside a typewriter.
Close-up of the fan of type bars inside a typewriter.
Close-up of the fan of type bars inside a typewriter.
Close-up of the fan of type bars inside a typewriter.

FIG. 07

Type bars, a strike at a time.

Engineering

By A. Sample

Engineering

8

min

Sample post

Every name, figure and line here is sample copy.

Every week Docwake drafts edits to documentation it did not write. Before a draft reaches a person, it has passed four sets of tests. This post explains what each set checks, what counts as a pass, and where the agent still gets it wrong.

The four test sets

  1. Stale page search: given a merge, did Docwake find every page that no longer agrees with the code?

  2. Draft quality: does the edit change only the lines that were wrong, in the voice of the page around them?

  3. Checks: do the links resolve, do the code samples run, and does the page follow its style rules?

  4. Restraint: when a page is still correct, does Docwake leave it alone?

What counts as a pass

A draft passes when a reviewer would merge it without touching it. We do not score how close a draft comes. A draft that needs one word changed is a fail, because it still costs a person their attention.

A worked example

When a merge renamed the setting timeout_ms to request_timeout_ms, Docwake found four pages that used the old name. It drafted edits for three of them and flagged the fourth, a migration guide, for a person, because that page should mention both names.

The best draft is the one nobody has to think about.

Where the agent still fails

  • Pages that explain why, not just how. Docwake can update a value, but not a reason.

  • Screenshots. It can tell an image is out of date, but it cannot take a new one.

  • Changes spread over several merges, where no single merge tells the whole story.

We publish the pass rate for each set with every release. The figures on this site are samples; yours will come from your own repository.

Next post

One year of release notes →

New posts by email.

One email per post. Unsubscribe at any time. See the privacy notice.

Docs that keep up with your code.

$ docwake run --on homepage
Run complete: 9 steps, about 3 min, exit 0
Steps: Docs that keep up with your code, Five steps, every merge, Tested on 1,240 doc changes, What Docwake does not do, Fits the tools you already use, The changelog is the roadmap, Free for one repo, Asked before every trial, Run it on your next merge.

© 2026 Docwake. Sample company.

Docs that keep up with your code.

$ docwake run --on homepage
Run complete: 9 steps, about 3 min, exit 0
Steps: Docs that keep up with your code, Five steps, every merge, Tested on 1,240 doc changes, What Docwake does not do, Fits the tools you already use, The changelog is the roadmap, Free for one repo, Asked before every trial, Run it on your next merge.

© 2026 Docwake. Sample company.

Docs that keep up with your code.

$ docwake run --on homepage
Run complete: 9 steps, about 3 min, exit 0
Steps: Docs that keep up with your code, Five steps, every merge, Tested on 1,240 doc changes, What Docwake does not do, Fits the tools you already use, The changelog is the roadmap, Free for one repo, Asked before every trial, Run it on your next merge.

© 2026 Docwake. Sample company.

Docs that keep up with your code.

$ docwake run --on homepage
Run complete: 9 steps, about 3 min, exit 0
Steps: Docs that keep up with your code, Five steps, every merge, Tested on 1,240 doc changes, What Docwake does not do, Fits the tools you already use, The changelog is the roadmap, Free for one repo, Asked before every trial, Run it on your next merge.

© 2026 Docwake. Sample company.

Create a free website with Framer, the website builder loved by startups, designers and agencies.