The Draft Grades Itself

That clip up top: the actual t3 export, row by row — the Roland Garros final as the machine drafted it. Every point gets a machine string, a confidence score, and a flag; the HIGH rows light green. Then three of those flagged points play with their draft row overlaid — an ace charted s4* at 0.969, a ?-riddled two-shot at 0.933 — each with the jump-to timestamp a volunteer charter would use. The tally mid-clip is the product claim: 59 draft points, 16 flagged "start from the draft."

cv-14 ended with arithmetic instead of opinion. This session spent the arithmetic: one last component grind ran into its ceiling, the fourteen posts of experiment sprawl folded into a package behind a brutal gate, and on top of the package shipped the thing the MVP actually needed — a confidence layer honest enough to tell a charter which third of the draft to trust. It became a tool.

Plain English: what actually happened here

The finale ships a confidence layer, which is three plain ideas. Coverage: what share of drafted points the system is willing to stamp HIGH. Precision: when it stamps HIGH, how often the draft actually holds up. The trade-off is line-calling: a judge who only calls balls a foot out is nearly always right and nearly useless. Calibration is how the flags get honest — take points humans already graded, look at signals the pipeline computes about itself (did the serve call commit? does the ball track have holes? do the striker votes conflict?), and fit those signals to the real outcomes, with each match flagged by a model that never saw it during fitting. The system never grades its own homework — and the fancier "sign off at a glance" tier that failed that test is reported dead, not hidden. What survived: HIGH means "start from the draft," right 93% of the time — a first-serve percentage, not a guarantee. Also here: $12 spent letting SAM re-track players, which improved letters and still lost the job to the free detector.

More on precision, coverage, and calibration: Court Vision in Plain English.

The letter sink, and the $12 question

The decomposition named letters the biggest post-direction sink (1.71 edits/point), and the suspected root cause has been in the LOG since the pilot: the letter is ball-x vs box-center-x at contact, so it inherits every sin of the $0 background-subtraction player boxes. Quantified before built, per the house rule — every aligned letter binned by the condition of the box that fed it:

outcome         sane   implaus  absent
right             70      31       0
wrong             17      18       0
refused           11      23       8
accuracy          71%     39%      --

45% of letters were read off a bad box. Ceiling if every box hit the sane-box rate: ~+26 strict letters. Cheap hygiene shipped first — court-half plausibility, x-only teleport rejection, gap interpolation, a letter gate widened to the clip's typical body height — and bought 117/209 from 114/209, with three dead ends measured and removed on the record. That's a plateau at an eighth of the ceiling, and the residual diagnosis is unambiguous: on the failing far shots, the far player is simply not in the CSV anywhere near contact. No temporal hygiene conjures a player the tracker never saw.

That was the gate for authorized spend: SAM-3 re-tracked the players on exactly one tree (t3 holds the box-driven failure mass), prompts derived automatically from the bgsub boxes, ~$12 all-in. Same eval, same hygiene: strict letters 59% → 67%, and 15 of the 21 gains are refusals turned right — t3_point_25 alone returns 7 letters because the far player finally exists at contact. But acceptance stayed 3/59 (letters alone don't cross the ≤1-edit bar, as the counterfactuals predicted), the losses concentrate in clips whose far player bgsub never boxed at all — the auto-prompt inherits the bootstrap blindness it was meant to cure — and the shipped default stays bgsub at $0. The delta is now a measured number instead of a suspicion, and it says the same thing every component lever has said since the decomposition: single-component grinding buys points, not multiples. The next multiple was product.

Fourteen posts of sprawl, one gate

Twenty-odd experiment scripts — four chart twins kept in sync by hand, four evals, shared modules imported by directory adjacency — are now the courtvision/ package: one chart assembler, one eval, one CLI, and every per-broadcaster difference the twins carried in code moved into data/matches/<id>.yaml. Adding a match is a YAML, not a fork.

The gate, because "refactor" is where numbers go to drift: the package had to reproduce the benchmark exactly before anything new was allowed on top. It did — all 138 chart CSVs byte-for-byte, all four scorecards byte-identical down to the per-clip lines, acceptance 7/135 untouched. The experiments/ directory stays frozen as history. And the whole loop is now one command:

courtvision draft t3
  -> chart CSVs -> confidence per point -> outputs/t3/export/t3_mcp_draft.csv

The tier that died, and the two that shipped

A 57%-within-5-edits draft is useless if the charter has to discover per point whether it's the 5%-acceptance kind or the 25-edit kind. The confidence layer feeds per-point signals the pipeline already computes — serve commit and stance margin, striker-chain conflicts, ball coverage and holes, refusal fractions, the direction signal-tier ladder, crossings-vs-shots consistency — into a small logistic, calibrated leave-one-match-out against token edit distance. Two finds on the way in: aces were being charged 100% letter refusal for having no rally letters to refuse (the cleanest points in the set, punished for being short), and the clay editor's mid-rally cuts leave a fingerprint — a "serve" fired 0 s into the clip whose launch point sits inside the court, where a real serve's toss projects 20+ m beyond the baseline. That one mechanistic gate took held-out t3 from 65% to 84% high-tier precision.

The tier I wanted — "sign off at a glance," within 2 edits at ≥85% precision — was built first and failed the audit: 50% held-out precision at 1.5% coverage. 135 points at an 11% acceptance base rate cannot support it. That tier is dead and on the record, not hidden behind an in-sample number. What survives is the usable-draft bar the effort curve already named:

LOMO (held out)   high prec (≤5 edits)   coverage   low ≤5 rate
t1 night              11/11 (100%)         50.0%        27%
t2 ctrl                 3/3 (100%)         60.0%        50%
t3 clay               16/19  (84%)         32.2%        43%
t4 grass              11/11 (100%)         22.4%        50%
pooled                41/44  (93%)         32.6%        44%

Read the flags honestly. HIGH means "start from the draft" — 93% of high-flagged points need ≤5 token edits, held out, with 3 disasters in 44 flags — not "the draft is right" (only 27% of HIGH are within 2 edits). LOW means "expect heavy correction" — 56% of low-flagged points are 6+ edits out; treat the draft as a hint. t3 is the weak fold for the honest reason: its disease is footage the editor never broadcast, and no per-point signal can see film that doesn't exist.

What a volunteer MCP charter actually opens: one CSV in the MCP points schema — match, point number, score state, server — with the machine string sitting in the 1st-serve column, plus confidence, conf_p, the clip name, and a jump-to timestamp. Sort by flag, play the clip from serve_s, correct the string. Across the four benchmark matches: 138 draft points, 46 flagged high.

What "reliable" still requires

Said plainly, the promise today: given a broadcast match with a fitted homography and a transcribed score-bug alignment, a draft chart for every tracked point at $0 marginal cost, and a HIGH flag on about a third of them that means "start from the draft" with 93% reliability. Not promised: that a high-flagged point is right, that a low-flagged point is chartable without the video, or anything about rallies the editor cut before the broadcast.

The north star hasn't moved since cv-14 named it: acceptance at ≤1 token edit sits at 5.2%, the ≤5-edit curve at 57%, and every confidence tier above "usable draft" is starved by the same number — 135 calibration points, 11% base rate. The binding constraint on the tool is no longer any component of the pipeline. It's ground truth. Three more matches are being staged as this posts, and calibrate refits the moment they land.

Session cost: ~$12 (the SAM A/B, under a one-off $30 authorization), then $0.00 for everything after. Project total: ~$16.