One font rule put 15 wrong sites on a CMS detection list

Post from r/TechSEO: How accurate are CMS detection tools,
Post: r/TechSEO, u/Sima228

Summary

A custom CMS detector mislabeled roughly one in six of the sites it matched because one rule keyed on a font served from another vendor's CDN.

Before a stack-detection list leaves your hands, confirm each match on a second signal of a different kind.

A custom CMS detector returned 86 matches, and 15 of them were wrong. The team that built it reported in an r/TechSEO thread that one of the detection rules was keying on a font loaded from another vendor’s CDN, a file that says nothing about which CMS renders a page.

The scan covered 8,273 domains, and the match list went straight to a customer. The customer named two sites that did not belong on it within about a minute, and a re-check of the full list turned up 15 bad entries, roughly 17% of it.

A file served from a vendor’s CDN can be referenced by any site that loads it, so its presence shows what a page fetches, not what builds the page. That kind of rule is easy to write and hard to notice afterwards. When only 86 of 8,273 domains match, a rule that wrongly fires on a tiny fraction of the crawl still accounts for roughly a sixth of the list.

The 17% figure counts wrong entries among the matches, not the detector’s overall accuracy. That is the error that bites when a list drives outreach or scopes an audit. The post gives no figure for CMS users the detector passed over, and for a list handed to a client a wrong name usually costs more than a miss. The same gap between an asset being present and the asset meaning what a tool claims showed up when AI-readiness checkers scored sites for having schema rather than for serving it to crawlers.

One reply in the thread argued for at least two independent signals before calling a match, a generator meta tag plus a CMS-specific asset or header, on the grounds that single-rule matches produce most false positives. The stronger version of that advice is to require the second signal to be a different class of evidence. Two asset references from the same bundle fail together. A generator tag and a path that only one CMS serves do not.

The thread’s open question was whether BuiltWith or Wappalyzer, the two best-known technology profilers, deserve more trust than a homegrown script. Nothing in the post measures the commercial tools, and the safer assumption is that any fingerprint-based detector carries some version of this failure, since all of them infer a stack from files a site can load for any reason. Running a second detector and keeping only the overlap costs little, and where two tools disagree the disagreement is itself useful, as it was when AI Overview coverage trackers split 62% to 82% measuring the same thing.

What to do

  • Treat a single-rule match as a lead, not a finding. A match counts when two signals of different kinds agree, such as a generator tag and a path only that CMS serves.
  • Audit every rule that points at a third-party domain. If the file sits on a CDN the CMS does not control, the rule is matching that file’s users rather than the CMS’s.
  • Open a random sample of matches before the list leaves. At a one-in-six error rate the first few rows give it away, which is how the customer caught it in a minute.
  • Say what the number measures when you hand a list over: how many rows were checked by hand, and that sites the tool skipped were never counted.