one small package, operable by one person

How to get gaige, set it up, and read your first receipt. Every feature ships with a command-line path and a plain-English --help; nothing below assumes a cluster, a team, or prior calibration experience.

THE SHAPE

gaige is a Python package for python 3.10 and up. Most of it needs nothing but numpy and requests: corpus handling, the ROC and threshold math, conformal calibration, subgroups, receipts, the run registry. All of that runs anywhere. The GPU extra (torch, transformers, bitsandbytes, accelerate) exists only for scoring against a real model; analysis never touches it.

Once models are staged, nothing here needs a network: scoring is local, analysis is offline, and receipts replay bit-identically across an air gap.

License: AGPL-3.0, commercial licensing available. Anything that surfaces an error, an interval, or a caveat is free on principle, forever.

GET

FROM PYPI

pip install gaige · 0.0.1, on pypi.org/project/gaige, released with its own verification receipts. The core install is analysis-only (numpy + requests); add [gpu] when you need to score.

FROM SOURCE

github.com/ghxrk13/gaige: clone, then pip install -e .; the whole tool is one package directory and a CLI entry point. History, issues, and CI run in the open.

REQUIREMENTS

Python ≥ 3.10. Core: numpy, requests. GPU extra: torch, transformers (< 5, pinned so 4-bit stays verified), bitsandbytes, accelerate. A GPU is optional; a receipt is not.

SUPPORT

Issues and receipt-backed reproductions are welcome; improvement ideas and use cases go to Discussions. Maintainership is deliberately closed. Licensing: [email protected]

SET UP

$ python -m venv .venv && source .venv/bin/activate
$ pip install gaige             # core: analysis anywhere, no GPU needed
$ pip install "gaige[gpu]"      # scoring: torch + transformers<5 + bitsandbytes

$ gaige plan                    # what can THIS machine honestly run, measured
$ gaige corpora                 # built-in corpora + the labeled-jsonl format
$ gaige providers               # served-model providers + attestation grades

gaige plan answers with measured memory floors and honest refusals: “NO — needs 8.0 GB free VRAM, have 7.9” is a feature, not an error. 4-bit quantization is CUDA-only; on CPU, use fp32 and expect the fingerprint to record a different instrument.

USE · YOUR FIRST RECEIPT

$ gaige run --corpus hc3-mini --n 100 --seed 17 --detector fast-detect-gpt
→ reports/<timestamp>-fast-detect-gpt/
    report.md      the receipt, human-readable
    results.json   every statistic, wholesale
    scores.csv     per-document scores
    roc.json       the curve
    env.json       instrument fingerprint + the exact reproduce command
    run.json       run identity

$ gaige run --corpus your-labeled.jsonl      # rows: {"text": ..., "label": "human"|"ai"}
$ gaige analyze --report reports/<timestamp> # re-derive anywhere · no GPU · bit-identical

What the receipt says, and how to read it:

The headlineAUROC + interval

Separation with a bootstrap confidence interval, e.g. AUROC 0.9285 [0.9061, 0.9482] on a RAID benchmark slice. Never a bare number.

Thresholds, two waysnever confused

Empirical in-sample and conformal with a finite-sample guarantee, side by side. The two families answer different questions, and the report keeps them apart by construction.

Refusalsthe integrity beat

When your data cannot support the guarantee you asked for, the receipt says so and stops: α=.005: refused — needs ≥199 human calibration samples, got 120. A tighter guarantee than your data supports is not a guarantee.

Subgroupsaggregates hide

Error rates stratified over every axis your corpus metadata carries, each with an interval: on one RAID slice the single AUROC hid greedy 87.6% vs sampled 39.7% TPR at the same threshold.

Fingerprintno fingerprint, no number

Model, quantization verified at load, device recorded as resolved (never “auto”), library versions, corpus sha256, and the exact command that reproduces the number.

COMMANDS

gaige runscore a corpus

Score a labeled corpus with a pluggable detector and write the full receipts report.

gaige analyzere-derive anywhere

Re-derive every number from existing scores: laptop, CPU-only box, or air-gapped machine.

gaige scoreone document

Score a single document against a calibrated report’s stated operating points. Measurements, never a verdict.

gaige corpus prepare-raidbenchmark slices

Provenance-pinned, seeded, stratified calibration slices from the RAID benchmark; slice metadata rides into every downstream receipt.

gaige probe rundated probe sets

Run dated, provenance-carrying probes against a served model and grade deterministically: the longitudinal-measurement lane.

gaige series list · show · watchdrift monitors

Registered run series with frozen instrument identity; watch applies per-interval conformal alarm bounds.

gaige planfeasibility, measured

What this machine can run, at what measured cost, with what attestation, and no quality column, deliberately.

gaige providers · test-connectionserved models

List providers (local-hf, llama.cpp, ollama) with honestly graded attestation; verify reachability before a run wastes an evening.

Also in the box: gaige probe new and gaige probe lint: probe-set authoring with schema enforcement, manifest-gated so a probe run refuses material that violates the declared instrument decisions.

KNOWN EDGES

transformers stays below 5. Newer majors silently ignore the 4-bit config; gaige counts quantized modules at load and refuses a silent fp16 load rather than emit a receipt that lies about its instrument.
4-bit is CUDA-only. On CPU the honest option is fp32: the falcon-7b pair needs roughly 56 GB of RAM and real patience, and the fingerprint records it as the different instrument it is.
CPU-only boxes: install torch from the CPU index first. pip install torch --index-url https://download.pytorch.org/whl/cpu, then the [gpu] extra; otherwise pip resolves the multi-gigabyte CUDA build you cannot use.
Memory floors refuse. Loads check free VRAM/RAM against measured floors and name the numbers when they decline; co-resident work is protected on purpose.
Nothing transfers. Thresholds, scores, and separation are properties of one instrument on one corpus. Every receipt states what it was measured on, and no number is offered as universal.
Benchmark text stays outside. Corpus adapters download benchmark rows when you prepare a slice and record exactly where each row came from. None of that text ships with gaige or gets redistributed.