The $0 Specialist and the Load-Bearing Bug
That clip up top: the same twelve-shot Federer rally twice. Left, SAM-3's sparse track — yellow comet dying mid-rally. Right, WASB's dense track riding the ball to the end. Identical clip, identical frozen chart loop; the only difference is the tracker. SAM charted this point as 6 shots. WASB got 12/12 with all 7 committed letters.
Before spending another dollar on ball tracking, I spent an evening finding out whether anyone had already built this project. The answer reshaped the roadmap, fired a foundation model, and then taught the nastiest lesson of the series so far.
Nobody charts
A deep-research pass — 23 sources fetched, top 25 claims adversarially verified, 25 survived — settled two things. First: the perception layer of tennis CV is a solved, open, free problem. WASB-SBDT (BMVC 2023, out of NTT, MIT license) ships pretrained tennis weights — 6 MB, downloadable today, F1 94–96 on the broadcast-tennis benchmark. Open pipelines like TennisProject already replicate our entire classical stack: ball, court keypoints, homography, players, bounces. Our SAM-3 ball tracking was reinventing all of it, worse, at $0.07 a clip.
Second, and this is the finding: every one of those systems stops before notation. No commercial product, paper, or repo found that both ingests plain broadcast video and emits a symbolic shot-by-shot chart — SwingVision wants your own phone footage, TennisViz consumes Hawk-Eye's tracking feed, Hawk-Eye itself runs ~EUR 60–70k per court per week. The novelty of this project was never the pixels. It's the grammar — striker chains, letters, zones, endings, and grading against human charts. Which somebody wants: Jeff Sackmann's Match Charting Project has ~17,800 matches charted by just 32 active volunteers, covers about a quarter of tour matches, and his January post ends with an explicit plea for more charters. Machine-drafts-human-corrects has an obvious customer.
One more absence worth money: no published SAM-vs-specialist comparison on small fast balls exists. So we ran one.
The A/B nobody published
WASB's tennis weights, a ~200-line driver (their code hard-asserts CUDA; ours runs the model on MPS at 15–20 fps), and the frozen chart loop untouched. On our own MCP-graded clips — the first SAM-vs-specialist ball comparison we know of — the specialist wins every case that matters: mean coverage 87% vs 82% on the t1 night clips, 88% vs 74% on the t2 day clips where SAM was starving in the glare. SAM's disasters evaporate (55%→93%, 55%→87%); its only wins are 3–7 points on the easy floodlit clips. And the 13 t1 clips SAM never touched because no bootstrap earned an eyeball approval? WASB tracked all of them, 73–97% coverage, zero prompts, zero interaction, zero dollars.
The scorecard verdict is the t2 control — the match that had just caught SAM losing the ball. Rally length 1/5 → 4/5 on identical clips. The twelve-shot Federer rally SAM saw as six comes out 12/12 with all seven committed letters correct. Marginal tracking cost per match: $0.00, forever, minutes on a laptop GPU.
SAM retires from ball duty with honors, not shame: the box-prompt bootstrap was the right way to get a pipeline standing before knowing whether any of this would work, and it remains the leading candidate for player segmentation, where the object is large and a prompt is cheap. But a promptable everything-model was never going to beat a 6 MB network that has only ever looked at tennis balls.
The bug was load-bearing
The same A/B carried a regression, reported in full: t1 rally length went 7/11 → 5/11 — all over-counts. Frame-checking the phantom "hits" found them in pixels: a dead ball bouncing in the corner while both players stand around; match point, ball at the frame's bottom edge, crowd mid-standing-ovation; a ball drifting through the crowd above the back fence. The point ends. The WASB track doesn't.
SAM's sparse track always died with the rally — and it turns out the chart loop had been leaning on that. Track death was an accidental end-of-point detector nobody designed, nobody named, and nothing replaced. WASB never loses the ball, so it faithfully follows it into the aftermath, and the frozen cusp detector charts the coda as rally.
The fix that failed first, for the record: velocity magnitude. Coda bounces swing 109–210 px/frame against real hits' 8–33 — a beautiful gap, until t2's serves landed inside it. What held is position: phantoms don't live on the court. Every frame-checked coda event sits outside the playable envelope in court meters; real hits across both test matches never leave it. One positional gate with a serve-window exemption, three new constants in the A/B twins, every frozen-era constant untouched — and t1 rally length lands at 10/11, the best on the project, while t2 is byte-identical before and after, which is exactly the claim the gate makes.
The meta-lesson goes in the permanent file: a component's bug can be load-bearing. Upgrade one part and its hidden job surfaces somewhere else — as a regression in a metric the new part has nothing to do with. The only reason this took an evening instead of a week is that the eval harness caught the number moving the wrong way immediately.
The verdict
The landscape says the gap is the grammar; the A/B frees the effort to live there. Ball tracking is now a solved input: dense, free, prompt- free, and gated to the court it plays on. All 60 dev-reel clips are retracked at $0, including 37 SAM never got to touch.
Session cost: $0.00. Project total: ~$4.15 of $9 — and nothing ahead of us charges by the frame anymore.