M3: Who Hit That?

That clip up top: both players boxed and tracked, FOREHAND or BACKHAND called at every swing in the striker's color, their feet moving live on the court diagram — and along the bottom, the match notation typing itself.

M2 ended with a proto-chart that was honest about its ignorance: ??2?1?1?2?2?2???. Every question mark is an M3 requirement, and the biggest one — shot type — needs to know where the players are. That's M0's unfinished business finally coming due: the ball got tracked back in week one, and the players got waved at.

M3 won't fit in one session. This was the first: player tracking, and the first two letters of the charting alphabet.

Two prompts, one blob

The plan was the exact trick that caught the game ball in M0: draw a box around each player in one frame, let SAM 3 propagate it both directions. Two box prompts, one call.

The two prompts, drawn by eye and render-checked before spending API money. Note the red-shirted line judge behind Gasquet — deliberately outside the box.

The response came back merged. One box per frame — the union of both players. This is M0's oldest lesson wearing a new outfit: text prompts match every instance of a concept, and it turns out multiple box prompts in one call fuse into a concept too. Per-object structure doesn't survive the endpoint.

And the union box was too wide. It stretched past Gasquet to about pixel 807 — which is exactly where that line judge stands. The segmentation had quietly picked up a third man nobody prompted.

The mask still knows

The boxes were ruined, but the response also carries the raw RLE masks, and a mask can't hide its structure. Decoded (space-separated start/run pairs over the flattened frame — undocumented, like the box format before it) and split into connected components, every frame contains three clean, spatially disjoint blobs: Zverev, Gasquet, line judge. Baseline rally — nobody ever crosses anybody in image space.

Two of those blobs are players. The trick for the third: anything segmented that never moves isn't a player. One pass over the rally marks every pixel that's on in more than 85% of frames — that's the judge, standing at parade rest for all sixteen seconds — and subtracts him from every frame.

The one figure the mask captured that never moved. The static filter is the whole de-judging algorithm.

He erased himself. No person detector, no jersey-color classifier — just np.mean over time, the same flavor of free lunch as M1's clean plate.

480 for 480

What's left splits near/far by which half of the frame each component's feet are in:

Both players, every frame. The two short blue flat spots around frames 390 and 425 are the track wandering to another far-side figure — both after the rally ends, but on the record.

480 out of 480 frames, both players, zero gaps. The ball managed 470 with ten dropouts. Player-sized objects are easy mode for SAM 3.

And here the pipeline collected a small inversion of M1's big caveat. The homography only maps the ground plane, which made the airborne ball's positions approximate — but feet are on the ground plane. Box bottom-center through the homography is exact. The most trustworthy positions in the pipeline now belong to the players, not the ball.

Who hit that?

With players placed, the striker question has three independent answers: which player's box is nearest the ball at the hit frame; which end of the court the ball was struck from (M2's heuristic); and the emergent far/near alternation that fell out of the event detector for free.

All three agree, seven hits out of seven. When separate signals converge like that on data none of them were tuned for, the pipeline is telling you something true.

The letters

Shot type is a geometry question: which side of the body was the ball on at contact? Ball x minus striker center x — mirrored for the far player, who faces the camera, so his right side is image-left. Right side of a right-hander means forehand. (Both these players are right-handed. The pipeline assumes that rather than detects it — owned out loud, and solving it is already on the M3 requirements list as player identity.)

The verdict: f f f b f f b, and pulled contact strips for all seven swings to check by eye — the same eyes-beat-derivatives discipline that saved M2.

Shot 7, the rally's last swing: Gasquet's one-handed backhand, takeback coiled high. If you have to eyeball-verify a backhand, make it Gasquet's.

Seven for seven. One of them, though, deserves an asterisk: on shot 3 the ball crossed Gasquet's center line at contact and the offset came out at −1.5 pixels. The video confirms forehand — a low pickup off the right shoe — so the letter stands, but a 1.5-pixel margin is a coin flip that happened to land right. Before this meets a rally it wasn't tuned on, that offset needs a confidence threshold and a plan for bodyline shots.

The chart, v2

before:  ??2?1?1?2?2?2???
after:   ?f2f1f1b2f2f2b??

Every letter earned. The chart also gains a column the proto-chart couldn't imagine: where the striker was standing.

The rally as seven arrows: who hit, with which wing, from where, to where. Dashed arrows land on the far side, where positions are still ±meters.

Zverev struck his three shots from 0 to 1.3 meters behind his baseline. Gasquet struck his four from more than three meters behind his — which anyone who watched Gasquet play will tell you is just accurate scouting. (Honest footnote: far-side foot pixels are meters-per-pixel territory, so far-court depths stay soft.)

The verdict

M3 session one: both players tracked through the whole rally, striker identified three independent ways, forehand/backhand called on all seven shots and frame-verified. Session cost: $0.15 — one API call, everything after it numpy. Project total: about seventy-five cents.

The remaining question marks didn't move: the serve, the final landing, the ending code, and rally segmentation are all waiting on video this clip doesn't contain. A mid-rally highlight can't show you a point starting or ending. So the next session goes back to the least glamorous layer of the stack — clip sourcing, round two: full-match VOD, complete points, camera cuts and all.

The string says ?f2f1f1b2f2f2b??. Time to go earn the punctuation.