A 95% accurate detector can be wrong almost every time it fires.

Every exhibit before this one asked whether a hidden channel can be detected. This one asks the question that decides whether detection is useful: when the detector alarms, how often is it right?

ARITHMETIC / NO QR ENCODING / APPLIES TO EVERY DETECTOR ABOVE

THE SEARCH POPULATION

Ten thousand symbols

Set how often hiding actually occurs, and how good the instrument is. The grid below is one dot per symbol.

Predict first

A detector is 95% sensitive and 95% specific, and hiding occurs in 1 symbol per 1,000. Of every 100 alarms it raises, about how many are real?

About 95 About 50 About 2

—

10,000 SYMBOLS / ONE DOT EACH

hidden, and caught hidden, and missed clean, but alarmed clean, and quiet

WHAT THE INSTRUMENT DOES

—

WHAT THE POPULATION DOES

—

WHAT AN ANALYST SHOULD REPORT

—

THE FOUR QUESTIONS

Ask four questions of a number, not a symbol.

WHAT THE SCANNER SEES

Nothing relevant. A base rate is a property of the population being searched, not of the symbol in front of you, and no amount of inspection of one code reveals it.

WHAT CHANGED UNDERNEATH

Nothing. This exhibit takes a detector that already works exactly as advertised and asks what one of its outputs is actually worth.

WHAT A SECOND READER CAN RECOVER

No hidden channel — there is none here. What a second reader needs is the prior: how often hiding truly occurs in the traffic being searched.

HOW AN ANALYST CAN NOTICE

By computing the positive predictive value before acting on an alarm. Sensitivity and specificity describe the instrument; only prevalence says what a positive means.

FINDING

Accuracy is a property of the instrument. Meaning is a property of the population.

Steganography is rare in ordinary traffic. When the thing you are looking for is rare, almost everything that alarms is a member of the large clean majority rather than the small hidden minority — even for an instrument that is right 95% of the time in both directions. Raising sensitivity barely moves this. Raising specificity moves it a great deal, because specificity is multiplied by the far larger number.

A detector that never fires on clean symbols is worth more than one that never misses a hidden one.

This is why every detection exhibit on this site reports what it cannot separate, and why regeneration-and-compare spends a full scenario on an innocent encoder mismatch. A false-positive condition is not a footnote to a detection method. At realistic prevalence it is the dominant term.

Basis: the arithmetic here is Bayes' rule and can be checked by hand from the grid. For why it matters specifically to steganalysis, see Andrew D. Ker et al., “Moving Steganography and Steganalysis from the Laboratory into the Real World,” IH&MMSec 2013, doi:10.1145/2482513.2482965 2ND — cited here as motivation, not independently re-verified.

Read the research note