How the test works, and where it doesn't.
This page documents how the Litmus Test works, the thresholds it uses, what our internal validation shows, and where the method is weak. A forensic instrument earns its standing by publishing its limits.
How the test works
The Litmus Test combines peer-reviewed extraction and membership-inference research into a single reproducible protocol. Given a target work and a target model, the protocol scores the rate at which the model reproduces the work under controlled prompting.
- Reporting thresholds: a standard tier and a strict evidentiary tier, calibrated against a published risk floor.
- Binary finding:
EXTRACTION CONFIRMEDorNO EXTRACTION DETECTEDat the reporting threshold used for the engagement. - Reproducibility: prompts, seeds, sampling parameters, and responses are preserved in the chain-of-custody record.
- Detailed protocol: the full test specification, thresholds, and citation matrix are provided to counsel and opposing experts under MNDA.
Internal validation (what it shows, what it does not)
The protocol has been run against synthetic memorization corpora and public-domain control sets across major frontier and open-weight models. On those internal corpora, the strict tier produces no false positives.
Where the test is weak
- Heavy paraphrase. Models tuned for aggressive paraphrase can evade verbatim-style detection. Every report discloses the exposure.
- Negative findings.
NO EXTRACTION DETECTEDmeans the model did not reproduce the work under the protocol. It is not proof the work was never in training. - Refusal-trained models. Aggressive refusal training reduces observable surface. Reports include a robustness score alongside the finding.
- Multimodal coverage. Image and audio coverage is narrower than text. Reports are scoped accordingly.
What this is not
- Not legal advice. Not a guarantee of case outcome. Not a substitute for counsel's independent judgment.
- Not a claim of prior court acceptance. The method is prepared under the Daubert / FRE 702 framework. Admission in any specific case is for the court.
- Not a black box to opposing counsel: every report is reproducible from its chain-of-custody record and the technical brief is disclosed under MNDA.
Available to counsel and opposing experts under MNDA. Includes the Daubert-factor matrix, prompt protocol, threshold calibration, and citation appendix.
Last reviewed 2026-06 · Protocol PROTO-2026.06