short answers, receipts attached
eight questions that keep arriving, answered in a paragraph each. every answer points at the page or committed receipt that carries the evidence.
the questions
is gaige a detector?
no. the confusion is understandable because both live in the same
room, but a detector reads an essay and leans human or machine, and gaige
stands behind the detector with a clipboard. run it over a labeled corpus you
chose and it writes down what the detector did there, which is usually messier
than the marketing: the AUROC came out here, the interval is this wide, the
threshold you would need for a 1% false-positive rate sits there, short texts
misfired more than long ones did. Fast-DetectGPT and Binoculars ship as
scorers it knows how to hold the clipboard for. what never comes out of it is
a ruling on any document, and the trademark terms are written so nobody can
borrow the name to fake one.
can it tell me whether a document was written by AI?
it can tell you what a calibrated scorer says about one document, with the error
rates you measured riding along: that is gaige score, and it emits
measurements only. what it will not do is convert that measurement into a verdict
about a person or a paper. the report’s base-rate arithmetic is there to show
how fast flags go wrong in mostly-human pools, before anyone acts on one.
the vendor published a threshold. why calibrate at all?
because thresholds do not transfer. the RAID benchmark study (ACL 2024) measured
open-source detectors at their default thresholds producing 47 to 100% false-positive
rates, whilst commercial tools ship factory-calibrated at 1.7% or less. its own
recommendation is to calibrate on in-domain data before use, and that sentence is
this project’s thesis. the evidence section carries
the receipts.
what is a receipt, concretely?
open one of the JSON files under
data/ and the shape answers the question. the
AUROC is in there, but so is the model that produced it, the quantization as
it was verified at load time, the device, the library versions, the sha256 of
the corpus, and one full command that rebuilds the run from scratch. that
bundle is the receipt: a statistic that cannot be quoted apart from its
instrument. the receipts page just renders those
committed files in place, which is why nothing on it is typed in by hand.
what does conformal buy me, and why do reports refuse?
start with what the ordinary
number cannot do. measure a threshold on 100 texts and you learn what happened
on those 100 texts, full stop. the conformal construction takes the same human
calibration scores, sorts them, and picks an order statistic, and that
mechanical little move is what earns a guarantee with actual reach: a
finite-sample, marginal bound that holds beyond the sample, assuming
exchangeability. the bill arrives as bluntness. the
committed
reference receipt shows α=.005 refused outright, because 100 human
texts cannot support it and 199 could, and gaige would rather name the
shortfall than invent the guarantee.
does it work offline?
yes, by design. scoring runs on local models once the weights are staged;
gaige analyze re-derives every number from stored scores with no model, no
GPU, and no network, bit-identically; receipts are plain files that cross an air gap.
pip install gaige phones nowhere, carries no telemetry, and disconnected
operation is a documented mode, not an accident.
what does it cost?
nothing, and the honest parts never will. the license is AGPL-3.0, and anything
that surfaces an error, an interval, or a caveat stays free forever, on principle.
commercial licensing exists for organizations that need different terms, and a
support page exists for fuel. neither changes what any
receipt says.
which detectors does it support today?
Fast-DetectGPT and Binoculars ship as scorers, each with its own guide:
fast-detectgpt in practice and
binoculars in practice. a watermark-class verifier is
queued on the roadmap. the scorer interface is pluggable
on purpose: anything that maps text to a score can be calibrated, compared, and
watched.
a question this page does not answer
bring it to github discussions. questions that keep arriving get promoted onto this page, with the receipt that answers them; the about page lists every other way to reach the bench.
adjacent: fast-detectgpt guide · binoculars guide · the package · about