May 2026
One expert eye serves about a dozen batsmen at best. Most of the coaching attention in a typical academy session falls to whichever player is in the highest-stakes net at the moment. The other eleven net for fifteen minutes and walk away with the same general note they walked in with. The academy partner asked us to close that gap. We built the model that does.
01. The brief
Cricket coaching is genuinely expert work. Reading a delivery, classifying what shot it should have produced, and grading what the batsman actually did takes years to develop. The academy had a handful of full-time coaches for several hundred enrolled batsmen across multiple sessions a day. The math does not work. They asked us to give every batsman an expert eye after every ball — delivery video to coaching paragraph in under ten seconds.
02. What the model needs to know about the delivery
Before we can recommend the optimal shot, we need to characterise the ball. Per delivery the pipeline computes:
- Type — pace, swing, seam, off-spin, leg-spin, googly
- Speed — derived from frame timing across the ball’s trajectory
- Line — relative to the stumps and to the batsman’s stance
- Length — yorker, full, good, short, bouncer
- Angle of arrival — incoming or going away, with magnitude
Tracking the ball is the hardest of these: a small object moving fast in a frame where the model also tracks bowler and batsman. The pipeline uses a ball-specific detector plus trajectory smoothing; the trajectory feeds a small physics model that estimates speed and predicted bounce, which classifies length more reliably than visual estimation alone.
03. What the model needs to know about the batsman
Pose estimation and bat tracking handle the human side across four phases of each ball. Each phase produces a pose vector; the shot classifier sees the full sequence and labels what was actually played — forward defensive, cover drive, pull, cut, hook, leave, or anything else in the taxonomy.
04. The shot taxonomy and training data
Twenty-four shot classes cover the realistic strokes, plus a “no shot” leave class. Broadcast footage with public commentary provided weak labels for the base classifier; a smaller set of academy footage, hand-labelled by the partner’s coaching staff, fine-tuned the model to the partner’s own coaching framework.
05. The decision and the scoring
For every delivery the model produces two answers: the optimal shot, conditional on the delivery, and the actual shot played. If they match, feedback focuses on execution quality; if not, it explains the gap. Execution is scored 0–10 on four axes:
- Timing — contact relative to the optimal contact frame for that delivery
- Footwork — did the batsman get to the right position
- Weight transfer — did body weight move with the shot
- Balance — did the batsman recover or fall away
06. The VLM layer
The earliest version output a JSON object. Coaches did not engage with it — their feedback happens in language, not numerics. So we added a vision-language model on top: it ingests the structured output plus the original video and writes a short, templated coaching paragraph in the academy’s voice, never longer than three sentences.
“Good length, off stump, going away at pace. The shot you wanted was a leave or a defensive prod with soft hands. You played a drive and the bat came down forty milliseconds late, the edge would have carried.”
Sample output from production
Coaches accepted this output where they had rejected the JSON.
07. Latency
The whole pipeline runs within ten seconds of the ball. The constraint is operational: a batsman should see feedback before the next ball is bowled, or the loop breaks. It meets the budget on standard academy hardware.
08. What did not work
An early rules engine could not handle the conditional structure of coaching, where the optimal shot depends on the batsman’s strengths as much as the delivery; a learned model trained on the coaching staff’s labels replaced it. The early pose pipeline also lost the bat through the follow-through — tracking the bat as a separate object, rather than via the wrist joint, fixed it and is now standard.
09. Handoff, and where else this works
The system is fully handed off to the academy; we do quarterly maintenance — retraining on new footage, evaluation against a held-out set, pipeline updates for new hardware. The pattern is expert eye at scale for a coached physical skill, and it transfers wherever three preconditions hold:
- The skill has a clear taxonomy of correct moves
- An expert can grade execution by watching a single repetition
- Repetitions are dense enough that human attention is the bottleneck
Golf swing analysis, tennis strokes, gymnastics routines, martial-arts form, physiotherapy gait analysis. The hard work is the taxonomy and training data per domain; the architecture transfers. We are open to building the next one.
End · Entry №04 · Filed May 2026
