Enhanced YOLOv8-pose network reads tennis actions and body poses from a single image

A single photograph of a tennis player contains far more information than most people realize. The angle of the shoulders, the bend of the elbow, the position of the hips relative to the knees — these static body configurations are enough for a well-trained neural network to decide whether the player is serving, hitting a forehand, preparing a backhand, or simply standing ready. That premise underpins a new study published in Discover Artificial Intelligence, in which a research team from four Chinese universities presents an enhanced YOLOv8-Pose network that simultaneously locates players in an image, classifies their tennis action, and estimates seventeen body keypoints — all from one static RGB frame, with no video input at all.

The challenge the team set out to solve is deceptively hard. Tennis is a sport of explosive, rapidly changing postures: the hitting arm moves so fast that it blurs across a single camera exposure, and visually similar actions can differ only in subtle joint positions. Full-court images frequently contain athletes occupying just a few dozen pixels, complex court backgrounds, motion blur, and self-occlusion caused by the racket, arms, and torso. Conventional pose estimation models, which perform well on clean studio benchmarks, often degrade substantially when body regions are partially hidden or when limbs move at high speed. Rather than chasing maximum accuracy or maximum speed in isolation, the researchers aimed for the best possible trade-off between the two within a compact single-stage model.

Technically, the proposed network builds on YOLOv8-Pose, a one-stage architecture that normally predicts bounding boxes and keypoints in a single pass. The team made three coupled modifications. First, they replaced the original convolutional backbone with an enhanced StarNet-based feature extractor. StarNet relies on the so-called star operation, an element-wise multiplication of two learnable pointwise transformations that introduces multiplicative nonlinear feature interactions without widening the network. The researchers augmented this vanilla star block with a multi-branch depthwise-convolution module containing identity, standard 3×3, asymmetric 1×5 and 5×1, and dilated 3×3 branches with dilation rates of two and three. The asymmetric branches capture directional limb patterns — elongated arms and legs — while the dilated branches gather broader contextual information, all before the concatenated features are projected back to the original channel dimension through a 1×1 convolution with a residual connection.

Second, the team inserted a motion-relevant attention module, or MRAM, at the three output levels of the backbone corresponding to strides 8, 16, and 32, before path-aggregation feature fusion. Despite its name, MRAM performs no temporal modeling and receives no optical flow or adjacent frames; it learns single-frame body configurations that are statistically associated with tennis actions. The module follows a sequential channel-then-spatial refinement strategy. In the channel branch, global average pooling and global maximum pooling generate complementary descriptors that pass through two pointwise transformations with a reduction ratio of 16, producing a sigmoid-gated channel attention map. In the spatial branch, channel-wise average and maximum pooling are compressed and processed by three depthwise-convolution branches — a standard 3×3, a directional 1×5 followed by 5×1, and a dilated 3×3 — whose concatenated outputs generate the spatial attention map. A residual fusion preserves the original pose information while amplifying action-relevant responses and suppressing background clutter.

Third, the researchers redesigned the keypoint loss function. The baseline YOLOv8-Pose objective is based on object keypoint similarity, a scale- and joint-aware COCO-style criterion whose exponential form, however, provides relatively weak gradients when coordinate residuals are large — precisely the situation caused by motion blur and self-occlusion. The proposed composite loss retains the OKS term and complements it with a normalized Euclidean regression term, which divides coordinate errors by the square root of the player-box area so that labeled joints are treated uniformly, and a Smooth-L1 term, which transitions from quadratic to linear growth and limits the influence of extreme outliers. The three terms, weighted 1.0, 1.0, and 0.5 respectively, provide complementary gradient profiles rather than three independent supervision sources. Keypoint-presence supervision is retained separately through a binary cross-entropy objectness term, and the complete training objective simultaneously optimizes player localization, action classification, and keypoint estimation.

The experiments were conducted in PyTorch on an NVIDIA GeForce RTX 5070 GPU, with 640×640 input resolution, 200 training epochs, a batch size of 8, and the AdamW optimizer with a cosine learning-rate schedule. The primary dataset was version 3 of the publicly available Tennis Action keypoint dataset from Roboflow Universe, distributed under CC BY 4.0, containing 2,000 images split into 1,600 for training, 202 for validation, and 198 for testing. The four action classes — Backhand, Forehand, Ready Position, and Serve — contain 500 images each in the complete collection, and every player instance carries an action label, a bounding box, and a 17-keypoint COCO-compatible pose annotation. All architecture choices, attention positions, and loss weights were selected exclusively on the validation split, and the configuration was frozen before any test-set evaluation.

The results are striking. On the primary test set, the proposed model achieves a Pose AP50 of 0.975 and a Pose AP50:95 of 0.746, together with a matched-instance action accuracy of 0.990 and perfect matching coverage of 1.000 — meaning it detected every one of the 200 annotated player instances in the test images. It correctly classified 15 Backhand, 24 Forehand, 52 Ready Position, and 107 Serve instances, with only one Forehand mistaken for a Backhand and one Ready Position mistaken for a Forehand. Equally important is what the model does not cost: it contains just 3.08 million parameters and requires 8.71 GFLOPs of computation, both lower than every comparator tested, and it runs at 51.7 frames per second in FP32 with batch size 1 on the RTX 5070. Compared with the original YOLOv8-Pose baseline, the enhanced network improves box precision by 3.0 percentage points, box recall by 6.0 points, and Pose AP50 by 4.4 points, while reducing parameters by roughly 6.4 percent and complexity by 5.3 percent and increasing inference speed from 47.8 to 51.7 FPS.

The ablation study dissects the contribution of each component with unusual rigor. Replacing the backbone with StarNet alone raises Pose AP50 from 0.931 to 0.950 while cutting parameters from 3.29 million to 2.96 million and boosting speed to 54.6 FPS. Adding MRAM lifts Pose AP50 further to 0.962 at a modest cost of 0.12 million parameters and 3 FPS. The improved keypoint loss, which adds nothing at inference time since it operates only during training, pushes the final Pose AP50 to 0.975. Crucially, the team repeated the four principal configurations with three random seeds — 0, 42, and 3407 — and the standard deviation never exceeded 0.003, while the complete model outperformed the baseline in every paired seed run with gains of 4.5, 4.4, and 4.3 percentage points. The improvement is therefore not attributable to a single favorable initialization. In head-to-head comparisons retrained under an identical protocol, the proposed method exceeded ProbPose, the second-best comparator, by 1.2 percentage points in Pose AP50 and by 2.0 points in box recall, and outperformed RTMPose, DWPose, RTMO, and PoseSynViT as well.

To test whether the pose-estimation gains persist beyond a single dataset, the researchers independently trained, validated, and tested the same architecture on a second public dataset, tennis-player-pose version 1, which contains 999 keypoint-annotated images with only two action categories, Backhand and Forehand, and an official split of 699 training, 150 validation, and 150 test images. The classification head was reconfigured for two classes rather than converted from the four-class primary model, and no images or annotations were shared between the two experiments. On this independent test set, the proposed method achieved a Pose AP50 of 0.820, a Pose AP75 of 0.661, a Pose AP50:95 of 0.549, and a [email protected] of 0.874, improving over YOLOv8-Pose by 8.0, 10.3, 9.3, and 7.6 percentage points respectively and exceeding the strongest competitor, PoseSynViT, by 2.0 to 2.7 points. The larger gains at the stricter AP75 threshold indicate more precise joint localization, confirming that the pose-oriented architectural changes remain effective when retrained in a second dataset setting.

The authors are candid about the limitations that define the scope of their conclusions. A single static image provides no temporal ordering, so the model cannot distinguish action phases that share nearly identical poses, and its action output is image-based classification from one frame rather than full video action recognition. The primary dataset contains only 2,000 images and four classes, and because the images were extracted from a limited collection of recorded tennis sequences, residual visual correlation across subsets cannot be completely excluded — the team explicitly calls for future benchmarks with source-grouped, sequence-disjoint splitting. Observed errors concentrate in visually similar Forehand and Backhand frames, small players, partially hidden wrists and elbows, motion-blurred limbs, and court structures that resemble elongated body parts. Still, the compact single-frame design points toward real-time applications well beyond tennis: the same architecture could be adapted to other racket sports, fitness assessment, rehabilitation feedback, and coaching systems in which a small set of posture-defined actions must be recognized instantly. For a sport where a serve takes barely a second, a network that reads the entire story from a frozen frame — and does so fifty times per second on consumer hardware — may prove to be exactly the tool courtside analysts have been waiting for.

Subject of Research: Computer Vision

Article Title: An enhanced YOLOv8-pose network for joint image-based tennis action classification and human pose estimation

Article References: Zhen, B., Bai, Y., Liu, Q., & Cui, C. (2026). An enhanced YOLOv8-pose network for joint image-based tennis action classification and human pose estimation. Discover Artificial Intelligence, 6(1), Article 1542. https://doi.org/10.1007/s44163-026-02513-4

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02513-4

Keywords: YOLOv8-Pose, human pose estimation, tennis action classification, computer vision, deep learning, keypoint detection, sports analytics, StarNet, attention module, real-time inference, enhanced, network