an instrument is a promise: calibrate first, trust second.
every footer on this site calls gaige the instrument you check other instruments against. this page says what that word commits us to, and what the practice of ai metrology looks like when it has to produce receipts.
why the word instrument
a scale in a shipping bay carries a calibration sticker: who checked it, against what reference, when it is due again. the culture that puts stickers on scales is called metrology, and it exists because a measurement without a calibration story is just a confident number.
an AI-text detector is an instrument in exactly that sense. it takes material in and puts a number out, the number moves when the model, the quantization, or the corpus moves, and decisions get made on it. what detectors mostly do not have is the sticker. ai metrology, in plain words: calibration and receipts you can verify, applied to the tools that measure AI. gaige is the instrumentation layer that produces the sticker: it measures the measurers, on your material, and writes down everything a skeptic would ask for.
the word is a commitment, not a metaphor. numbers here ship with error bars, a fingerprint, and stated refusal conditions, or they do not ship.
the instrument fingerprint
an AUROC from march and an AUROC from today only compare if the same instrument produced both. that is the fingerprint’s whole job. each receipt writes down the model, the quantization as checked at load (not what the config claimed), which device actually ran, the library versions, the sha256 of the corpus, and the command that reruns everything. series are stricter still: once registered, a series will not take runs from a second instrument. swap a library version and gaige calls that a new instrument, not drift.
the committed receipts on the receipts page show the practice: fast-detect-gpt on hc3-mini at AUROC 0.9720 means that number, under that fingerprint, reproducible by that command, with the 95% interval (0.9458–0.9938) stated, because a point estimate without an interval is another confident number.
calibration, then trust
calibrating the instrument means running the detector you already use on material like yours, and deriving the operating point from what actually happened. gaige states thresholds two ways: the empirical threshold measured on your labeled set, and a split-conformal threshold with a finite-sample false-positive guarantee. the thresholds page works the arithmetic end to end, including the base-rate step most deployments skip, the one that decides how many of your flags are wrong.
aggregates get the same treatment: a falcon-7b run on a RAID slice showed one threshold catching 87.6% of greedy-decoded machine text while catching 39.7% of sampled text. the aggregate AUROC hid that split; the stratified receipt did not. calibration that stops at the headline number is not calibration.
instruments age
calibration has a shelf life. the model gets re-quantized, a library updates, the material you score changes shape under you. the fix is boring, and boring is the point: put the detector on a maintenance schedule the way a lab does with its equipment. register runs as a series, re-measure on a cadence, and let the alarm carry an interval so one noisy run cannot cry wolf. the mechanics live on the drift page.
what the instrument refuses
- corpora too small for the ask. if your pilot cannot support the false-positive guarantee you requested, the answer is a refusal with the floor stated, never a number with silent caveats.
- mixed instruments in one series. a history that changed hardware or quantization mid-stream is two histories; gaige will not average them.
- mismatched environments. a receipt produced on one device does not certify a different one; the tool says so instead of pretending.
instrumentation, never verdicts
gaige measures scoring processes. it never pronounces a document human or AI, never ships a universal threshold, and never claims a detector is accurate or bias-free; those claims belong to whoever makes them, with a receipt or without one. the terms page holds this as a condition of use, and the certificate states the full scope. that boundary is what makes the metrology honest: the instrument that checks instruments does not moonlight as a judge.
adjacent: certificate · package · drift · thresholds