T1: A Test Set With Answers in the Back
That clip up top: the test match's night plate with two court fits A/B'd — the dev reel's homography transferred as-is in red, the manual four-corner fit in green, then both at once. The red one had 2.9-pixel residuals at one point in this story. Residuals lie.
Everything tuned so far — diff thresholds, swing gates, serve anatomy, vote weights — was tuned on the Zverev–Gasquet reel with my own frame checks as the only ground truth. That's a training set with a human in the loop, and it stops being evidence the moment the same eyes that tuned it also grade it. Best practice from here: freeze the pipeline, and test on matches somebody else charted.
Answers in the back of the book
Jeff Sackmann's Match Charting Project has 7,500+ men's matches hand-charted point by point — the exact notation this pipeline has been converging on. Our match isn't in it. But Canada Masters 2017 is: same tournament, same court, same broadcast package.
Test 1 is Nadal–Shapovalov, R16 — 225 charted points, a night session under floodlights, and both players left-handed. Those are precisely the two assumptions the dev reel can't test: every letter call assumes a right-hander, and every diff threshold was tuned in daylight. Same court and overlays, so failures attribute cleanly to what changed. In reserve: Federer–Haase, the SF — day, both right-handed, the nearest-transfer control. If the pipeline fails there, the problem isn't lefties or lights.
The eval protocol went on the record before any of it ran: constants
frozen; segmentation may be re-tuned for the new reel's cuts (finding
points isn't the hypothesis — charting them is); metrics per aligned
point are server end, serve zone, rally length, and letter accuracy
over committed letters only, with coverage reported beside accuracy.
No silent cherry-picking. Ground truth lives in the repo under
data/mcp/ (CC BY-NC-SA, attributed). Cost of a 225-point answer key:
$0.00 — it's a curl away.
The score bug is the join key
Footage is the 8:21 TennisTV highlights, and segmentation transferred clean — the stage that fought hardest on the dev reel needed only paths and a homography, and 25 court-view clips fell out with no threshold touched. Aligning them to MCP rows: read each clip's score bug, look up the row with that score. One trap cost six matches before I caught it — MCP's Pts column is server-first, the bug is Nadal-first. 17/25 matched before that transform; 23/25 after. The two leftovers are honest ambiguity, not failure: one 40–AD that happened twice in the same game, and one deuce from a ten-deuce game with ten candidate rows. The join even corrected me: my eyeball read of point_06's serve marker was wrong — Shapovalov served that game.
The homography does not transfer
The dev homography misses by ~20 pixels on the night footage — different production day, remounted camera. Refitting produced three instructive corpses before the fix:
- M1's recipe, verbatim. Under floodlights the blue-to-green paint boundary reads (V≈155, S≈55) — the same signature as a real line at grazing angle. The fit grabbed the paint edge and drew a court 15–20 pixels too big.
- Tighter white threshold. Kept only the camera-facing lines and lost the far baseline and sidelines entirely. Degenerate fit.
- Guided bands around the dev prior. The far-baseline band locked onto ground-paint whites 70 pixels off; a sideline band silently mixed doubles pixels into a "singles" line. Residuals: a beautiful 2.9 px — while the center line sat 32 pixels off its actual pixels. Plausible-but-wrong, caught only by rendering the reprojection and looking at it.
What worked: four manual corners off gridded 6× zooms of the median plate — one human input per match, the M0 ball-click precedent. Even that failed twice, instructively: the "corners" I kept reading at the near baseline were the singles corners; the doubles corners sit at the frame edges, buried in paint blur. Two arithmetic cross-checks settled it. The far-corner midpoint lands at x=603 against a measured center line of 606 — my near-corner reads implied 643. And pixels-per- meter is linear in image y: 48 at the far baseline through 80 at the near service line extrapolates to ~100 at the near baseline, predicting a doubles span of ~1095 pixels — exactly the white band's raw extent.
Final fit — far doubles corners plus near singles corners, with the tight-mask lines held back as untouched validators: service line 0.8 px, near baseline 1.4 px, center line 6.1 px.
The verdict
Third time this project has learned the same meta-lesson, so it's now policy: masks and residuals can agree on a wrong answer; a rendered overlay cannot. Every geometric fit gets an eyeball artifact from here on — that's what the clip up top is.
25 clips, 23 with human-charted answer strings attached, a ground plane that survives validation, and a frozen pipeline waiting to be graded. Both sessions combined: $0.00. The scorecard is next.