Every 508 testing tool is a rules engine over the rendered page, and the rules only cover the criteria a machine can decide. That is genuinely useful and it is roughly a third of the standard: missing alt text, missing labels, contrast below the ratio, duplicate ids, missing document language. It cannot tell you whether alt text describes the image, whether the reading order matches the visual one, or whether a custom component announces its state. This page is what to run, what each tool is honest about, and how to size the half no tool will find. On the worked example the criteria failing across nine templates are 126 of the 401 issues.
What the automated pass covers
Run any scanner over your top templates and you will find the machine-decidable failures quickly: images with no alt attribute, form controls with no accessible name, contrast under 4.5:1 for normal text, missing page language, heading levels skipped. These are worth fixing immediately because they are cheap and they are the ones a plaintiff's automated scan will also find.
Where 508 compliance software stops
No tool decides whether a keyboard user can complete your checkout, whether a modal returns focus to the control that opened it, or whether a data table's headers make the numbers comprehensible read aloud. Those are manual passes: keyboard-only, screen reader, and one run at 200% zoom. The audit sheet prices exactly that half: 19 sampled pages at 35 minutes each is 11 hours of manual testing on the worked example.
How to read a scanner's report without being misled
A scanner reports issues per page, so a site with one broken template reports the same issue two hundred times and looks catastrophic; a site whose failures are all manual-only reports nothing and looks clean. Count issues per TEMPLATE, not per page: nine templates carrying fourteen failing criteria is 126 template issues, and each one fixed reaches 26 pages.
Questions people ask about 508 testing tools
Which tool should we use?
Whichever your developers will actually run in the build. The overlap between the main engines is large; the discipline of running one on every pull request matters more than the choice.
Does a clean scan mean we are compliant?
No. It means the machine-decidable third is clean. A clean scan on an unusable site is the most common false comfort in this field.
Should testing be in CI?
Yes, on the templates rather than every page. It stops regressions cheaply, which is the only way a remediated site stays remediated.