How AI spots legendary cricket batting backlift using video only

The greatest cricket batsman – Sir Donald Bradman – used a batting technique that fans loved, but some critics slammed. A critical component was his lateral batting backlift technique. Highly successful batters use it to this day.

Artificial Intelligence (AI) can now tell straight backlift batters from lateral ones, using video only. Researchers from the University of Johannesburg (UJ) built the first deep learning computer vision AI model to do this. Their research is published in Nature Scientific Reports.

In future, coaches may give more detailed feedback to players using such technology. Or more easily identify players with lateral backlift components.

Legendary technique

In that split second just before a cricket bowler releases a ball at the crease, the batter initiates a very subtle movement. This is when the batter starts moving their bat into position. They select it depending on the format of the game and the type of bowler approaching them. Most fast bowlers tend to bowl at between 140 and 150 km/h (87 to 93 mph), says Prof Habib Noorbhai.

Noorbhai is currently Director of the Biomedical Engineering and Healthcare Technology (BEAHT) Research Centre at the University of Johannesburg (UJ). He is also a sports scientist and has worked with local and international cricket teams since 2010. His doctoral thesis was the first research to theorise the lateral batting backlift technique in cricket.

Another factor is the batter’s instincts and training. Will they bring up the bat straight with its face towards the ground or the stumps?

Or will they angle the face of the bat outward towards second slip or the wicket keeper? This is the technique legendary batter Sir Donald Bradman used. Successful Test Cricket batters also use it to this day.

AI helping out coaches

The researchers – Mr Tevin Moodley, Prof Dustin van der Haar and Prof Habib Noorbhai – all played cricket at school.

Moodley is a doctoral student and Van der Haar a pattern recognition researcher. Both are at the Academy of Computer Science and Software Engineering (ACSSE), University of Johannesburg.

Moodley played provincial cricket in primary school and continued playing cricket throughout high school. He has always shared a love for the sport and has combined AI with cricket with the goal of making meaningful contributions within the cricketing world.

He used AI to recognise different batting strokes in previous research.

For this study, he focused on getting AI to do the subtle task of identifying batting backlift style.

“I was wondering if I can automate that process,” says Moodley.

“Why does a coach have to think…. Is that straight or lateral? If they know, they can say ‘You have a lateral backlift, I can now help you in this manner’. I’m trying to take very manual processes and automate them, so we can help batsmen with their technique.”

Record-setting and self-taught

Sir Bradman was born in 1908 in the tiny rural town of Cootamundra in Australia about 350km from Sydney. He was self-taught. He exploded onto the Australian Test team as a teen with eye-opening scores.

Bradman’s unconventional lateral batting backlift batting technique was somewhat controversial in his time. He was harshly criticized by some. But spectators loved what he brought to the game, and he set records that still stand.

To this day, a lateral backlift can be a significant advantage at international Test level cricket. But the batting style is mostly not recognised as part of standard training. Players at cricket academies are mainly trained the straight backlift only, with others not placing much attention on the backlift.

Strategic advantage

“What we find is that a lateral backlift provides a batter with a more optimal position when facing the ball. Especially with fast bowlers. It gives the batter more time to get behind the ball,” says Noorbhai.

And the lateral backlift has yet another strategic advantage.

“A cricket field is round, so we use a ‘wagon wheel’ to indicate in which areas batters score runs. When batters have a straight backlift, they usually only score in certain areas. Tthis has been proven. It will be more towards the straight side of the wagon wheel called the ‘V’,” adds Noorbhai.

“When they have a lateral backlift, they score along more areas within the wagon wheel.”

Finding batters for the AI

First, Moodley had to find video footage for their AI to learn from, since no open datasets were available. He combed through online video footage of First-Class International Cricket Tests. He used the years 1995 to 2021.

From there he selected 10 batters with a straight backlift: Babar Azam, Themba Bavuma, Rahul Dravid, JP Duminy, Dean Elgar, Mahela Jayawardene, Ajinkya Rahane, Joe Root, Rory Burns, Ben Stokes, and David Warner.

And another 10 with a lateral backlift: AB de Villiers, Hashim Amla, Quintin de Kock, Faf du Plessis, Kevin Pieterson, Kumar Sangakara, Brian Lara, Ricky Ponting, Steve Smith, and Virat Koli.

In total, Moodley manually selected 160 frames (images) from the video clips as the AI’s ‘training class’. He selected another 40 as the ‘testing class’.

AI learns by itself

“The beauty of deep learning in AI is that you don’t have to tell the AI what to look for,” says Moodley. “A coach would need to focus on where the bat’s face is, the angle of the bat, where the batter’s feet and head are, and so on.”

The neural network in a deep learning AI can learn by itself, figuring out what factors are involved with each backlift style. It needs to see good batting images for both the straight and lateral backlift from a good dataset of course. If there is more data, it can teach itself better. It doesn’t even have to know what a factor is called, like downswing, or backlift angle for example.

Lateral vs straight

Next Moodley ran his deep learning model on four different AI architectures: AlexNet, Inception V3, Inception Resnet V2, and Xception. Would the older AlexNet best the newer Xception or not? All this was accomplished on a 7-year-old i7 processor computer.

But 160 images is very little data for an AI to learn from. It is a truly complex task to distinguish between the straight and lateral backlift.

“From a coach’s perspective, they would first look at the angle of the bat. Is the bat’s face angled towards the second slip?

“Or is the bat face towards the stumps or the ground – for a straight backlift? Within lateral and straight there are further variations as well. There are also other things that affect it, like hand positioning and how high you raise your bat,” says Moodley.

A human coach may take years to accurately tell the difference, even when aided by video analysis software.

Pre-training AI

This is where Moodley and Van der Haar employed ‘transfer learning’ to compensate for the small dataset.

“Deep learning methods require significant amounts of data to perform well,” says Van der Haar. “If we trained the model from scratch without using an already-trained neural network, it would not be very accurate. There would be too little data.

“With transfer learning, you pre-train a neural network with more general data as a starting point. Then you train it further using more specific data, such as lateral and straight backlifts,” he adds.

They pre-trained the model with transfer learning, so it would learn well from only 160 images, and then perform well in a test with 40 images.

Inception vs Xception

The model using the Xception AI architecture emerged as the most accurate. It could correctly distinguish players with a lateral backlift technique, from those who tend to play with a straight backlift, 98.2% of the time. Close on its heels was Inception Resnet V2 with accuracy of 96.1%.

“The Xception architecture takes each video frame and aggressively applies sequences of block transformations in a neural network to help separate the lateral and straight backlift categories,” says Van der Haar.

Once the model is trained, it delivers ‘end-to-end’ recognition between straight and lateral, says Moodley. With a press of one button on the computer the entire process is completed.

 “There are not multiple little steps. It’s like we say to the AI: ’Here are the images, go learn, done’”, he adds.

Telling grass from arms and legs

A coach analysing video footage to better assist a player, has to deal with a lot of ‘noise’, says Van der Haar. There are other players on the field also, and other objects. So an AI has to be good at recognising stuff relevant to batting technique in the video frame.

If the AI model only looks at texture in the video frame to distinguish between grass and a batter, it will come up short a lot of the time. Same with the AI only looking for transitions between light and dark – the analysis would be very inaccurate.

“What makes deep learning AI so phenomenal, is that it doesn’t worry about all that. It figures out if an area of the video frame is relevant or not,” says Van der Haar.

The neural network used for this AI model has several layers, he says. The first layer looks at low level stuff such as texture, or transitions from light to dark, called gradients.

Each higher-level layer of the neural net looks at something a bit closer to what we would look at. At the highest level, the neural net is looking at things like arms and legs and bat angles.

Once the neural net figures out the different layers in a video frame, it can then say ‘this is a lateral or straight backlift, says Van der Haar.

Coaching insight from AI

Conventionally, cricket coaches rely on analysis tools to see what their players are doing. It is a three or four-step process. They have to take a camera and record the players. They then export that footage to analysis software. Then they analyse the footage and provide recommendations to the players, says Noorbhai.

“What deep learning AI does, is a form of automation where the four steps are combined into one. All they would then have to do in future, is take that video footage, and use the deep learning AI as a form of automating what they are seeing,” he adds.

“I don’t think this research is about analysing more players. I think it is about providing coaches and players with more insight than they are currently getting,” says Noorbhai.

Untrained players use lateral

Untrained players often instinctively use a lateral backlift, says Noorbhai.

“What we have found, is if young players are not coached using traditional methods, they do not pick up the bat straight. They pick the bat up in a lateral direction. This indirectly suggests that a straight backlift is not a natural movement.”

Switching from straight to lateral

At the other end of the spectrum, some highly successful batters use both a straight and lateral backlift.

“At the very start of a batter’s innings, we find that they have more of straight backlift. Why? Because they have to be a little bit more defensive in their approach, so they can get used to the pitch, the bowlers, the conditions,” says Noorbhai.

“Once they develop confidence in their innings, they start to open up, then their backlift goes more in a lateral direction with an open face of the bat.”

Moodley, Van der Haar and Noorbhai’s research show how it is already possible to automate some cricket coaching. Their model can help coaches with promising and established cricket players, using video alone.

The legendary Bradman’s backlift was called a ‘loop backlift’ or ‘rotary batting technique’ in his time. Today we recognise it as a lateral backlift, as scientifically theorised by Noorbhai and Noakes in 2016. The style is used instinctively by untrained players – and strategically or inherently by successful professionals.