Lightweight AI Tracks Every Cow in the Herd in Real Time

Researchers in China have built a deep learning system that can watch a barn full of Holstein dairy cows, recognize what each animal is doing, and keep track of every individual identity in real time on modest hardware. The team, led by Zhihua Diao and colleagues, combined a slimmed-down object detector with an improved tracking algorithm and tested it on video collected at two dairy farms in Henan Province. Their results, published in Artificial Intelligence in Agriculture, show that the system detects three everyday behaviors—eating, standing, and lying—with high accuracy while running fast enough for continuous monitoring on edge devices.

The motivation is practical. Traditional cow health monitoring relies on manual observation, which is inefficient, resource-intensive, and limited by human subjectivity, making round-the-clock monitoring difficult. Wearable sensors can capture physiological parameters such as heart rate and body temperature, but high maintenance costs and potential discomfort for the animals limit their widespread use. Camera-based monitoring avoids those problems, yet real farms present hard technical challenges: varying illumination, cows blocking one another, and frequent posture changes that cause false detections, missed detections, and interrupted tracks. Most existing research tackles accuracy through heavier network structures, which the authors argue overlooks the computational load that undermines real-time monitoring efficiency.

To build the dataset, the researchers installed DS-2CD3T46WD-I3 network cameras at a height of 2.8 meters with a 45-degree downward angle, spaced 5 meters apart to cover the breeding area without blind spots. The cameras recorded 1920 × 1080 video at 30 frames per second, capturing roughly 10 Holstein cows per area. Videos were recorded between June and December 2024, sampled evenly across morning, afternoon, and night. The team segmented footage into short clips, extracted one frame every 20 frames with ffmpeg, and used stratified sampling to split images into training, validation, and test sets at an 8:1:1 ratio, with all samples drawn from different original clips to prevent cross-set leakage. After manual annotation with LabelImg, the result was a YOLO-format dataset of 8,000 images labeled for eating, standing, and lying behaviors. Test videos were additionally annotated with Darklabel, assigning each cow a unique continuous identity in MOT17 format for tracking evaluation.

The detection model starts from YOLOv10s and applies two lightweight strategies. First, inspired by the GhostNet architecture, the researchers designed GhostConv, GhostC2f, and GhostNet Block modules, which reduce redundant computation by generating a small set of intrinsic feature maps through standard convolution and then producing the remaining feature maps via cheap linear transformations. Second, they replaced the components of YOLOv10’s polarized self-attention module with an enhanced PSA design: a lightweight efficient local attention (ELA) mechanism takes over from the computationally intensive multi-head self-attention, using efficient pooling and one-dimensional convolution in horizontal and vertical directions to focus on key behavioral areas such as the head and legs, while a feature transformation sequence (FTS) module replaces the feedforward network with a bottleneck-style design that compresses and expands dimensions to sharpen feature expression.

The ablation experiments quantify what each component contributes. The baseline YOLOv10s achieved 90.1% mAP@50 and 54.3% mAP@50–95 on cow behavior detection, with 8.06 million parameters and 24.5 GFLOPs. Adding the Ghost module cut parameters by 24.2%, GFLOPs by 38.4%, and model size by 22.3%, at a cost of a 2.5-point drop in mAP@50 to 87.6%. Integrating the ELA attention mechanism maintained accuracy while compressing the model further, and adding the FTS module raised mAP@50–95 from 52.7% to 53.2% while shrinking the model from 12.1 MB to 11.7 MB. The final Lightweight-YOLOv10s configuration reached 88.6% mAP@50 and 53.2% mAP@50–95 with 5.48 million parameters and 14.4 GFLOPs—a 32% parameter reduction, 41.2% lower GFLOPs, and 29.5% smaller model size versus baseline, trading just 1.5 points of mAP@50. Per-behavior AP values reached 86.3% for eating, 88.9% for standing, and 90.7% for lying, with lying detected slightly better because standing and eating look similar in the spatial dimension.

Comparisons against mainstream detectors reinforce the trade-off. RT-DETRv2-s achieved the highest accuracy at 91.3% mAP@50 but with 20 million parameters and 60 GFLOPs, which the authors note fails edge deployment requirements. DEIM-s and YOLOv8s also scored higher on accuracy but carried nearly twice the parameters and GFLOPs of the proposed model. The lightweight model outperformed Faster-RCNN, DAMO YOLO-s, YOLOv5s, YOLOv7, and YOLOv9s on mAP while remaining the smallest and cheapest to run. On the public CBVD-5 dataset, which covers five behaviors including foraging, drinking, and rumination, the lightweight model scored 54.5% overall mAP@50 versus 54.9% for the baseline—only a 0.4-point drop—while improving detection of lying down (67.8% versus 60.4%) and drinking water (40.7% versus 34.0%), though rumination performance fell to 20.5% from 32.5%. Visual comparisons on four CBVD-5 scene videos showed the model correcting one false detection and alleviating one missed detection relative to the baseline, with stronger adaptability under uneven illumination.

Tracking posed a different problem. The team built GSI-Byte, an improved version of ByteTrack, with two targeted changes. First, a noise scale adaptive Kalman filter dynamically adjusts the weight of measurement data based on each cow’s detection confidence: clearly visible, unoccluded cows with high confidence get higher weight in the state update, while severely occluded targets with blurred features get lower weight so noisy detections do not distort the predicted trajectory. Second, drawing on Gaussian-based motion modeling from StrongSORT, the algorithm applies Gaussian regression at the trajectory interpolation stage only, leaving ByteTrack’s core association mechanism untouched. This preserves ByteTrack’s advantage of using low-confidence detection boxes to expand the search space while smoothing nonlinear motion such as sudden lying down or rapid running. The radial basis function kernel’s smoothness parameter λ depends on a trajectory smoothing coefficient τ and trajectory length l, set to a maximum gap of 30 frames and τ of 10.

On the cow test set, evaluated across 15 experimental samples of 10 videos each—samples 1–7 from Qixian Cattle Farm in Kaifeng and samples 8–15 from the Cattle Farm of the Henan Academy of Agricultural Sciences—GSI-Byte achieved an overall MOTA of 75.3%, IDF1 of 80.2%, 94 identity switches, and 52.1 frames per second. Lighting mattered: the best sample, under favorable daytime light with resting cows, reached 82.3% MOTA with only 45 identity switches, while a nighttime sample with insufficient light dropped to 68.5% MOTA. Against competing trackers on the same data, StrongSort managed 69.8% MOTA with 128 identity switches at 32.7 fps, Bot-Sort 72.6% with 125 switches, and OC-Sort 74.0% with 107 switches, while GSI-Byte led on every metric. Ablations showed the full pipeline improved MOTA by 1.2 points and IDF1 by 1.5 points over basic ByteTrack with YOLOv10s, cut identity switches by 27, and raised frame rate 34.3% to 52.1 fps. A comparison with reproduced livestock-tracking studies—YOLOv5s plus DeepSORT, an improved YOLOv7-tiny plus StrongSORT, and improved YOLOv7 plus ByteTrack—found GSI-Byte fastest at 19.19 milliseconds average fitting time with the highest MOTA of 75.3%.

The Gaussian interpolation analysis, conducted on trajectories from three cows, showed the normalized velocity of original trajectories fluctuating with dense spike regions, which GSI suppressed while keeping phase-consistent rises and falls. When τ was set to 4.0, the curve barely changed, because the Gaussian kernel becomes too narrow to filter noise; at τ = 12, smoothness peaked but real motion changes were excessively attenuated with a slight time lag that could blur behavioral boundaries. The authors identified τ = 10 as the best trade-off on this dataset. The system also showed its limits: in one sequence, a cow hidden for a prolonged period behind a steel column was re-instantiated with a new identity on reappearance, and another cow partially occluded by a neighbor was likewise re-instantiated. The researchers state that identity persistence can still fail after long or repeated occlusions and complete disappearance followed by reentry, and that occlusion-aware association remains a primary direction for future improvement.

Edge deployment tests on a Jetson Nano confirmed the practical payoff. After converting the trained PyTorch weights to ONNX and applying TensorRT operator fusion with FP16 quantization, the lightweight model at an input size of 480 processed each frame in 24.5 milliseconds—exceeding 40 frames per second—while the baseline YOLOv10s required 59.0 milliseconds at the same resolution, with only slightly higher accuracy (87.9% versus 87.6% mAP@50). Reducing input size below 480 noticeably degraded both mAP and MOTA, likely because fine-grained texture and posture cues become ambiguous. The authors acknowledge limitations: the behavior label space covers only basic behaviors and omits health-related ones such as walking and lameness; the dataset lacks extreme weather and low-light nighttime infrared conditions; and the GSI component is sensitive to hyperparameters, with fixed interpolation gap and τ values. Future work, they write, will explore multimodal fusion with audio, temperature and humidity sensors, and cow body temperature data, aiming toward a deployable commercial monitoring solution for dairy farms.

Subject of Research: A lightweight deep learning algorithm for real-time recognition and tracking of daily behaviors in multiple dairy cows

Article Title: A deep learning-based lightweight algorithm for multi-dairy cow daily behavior recognition and tracking

Article References: Diao, Z., Guo, Y., Zhao, S., Zhang, R., Li, J., Zhang, J., Li, X., He, Y., & Cheng, J. (2026). A deep learning-based lightweight algorithm for multi-dairy cow daily behavior recognition and tracking. Artificial Intelligence in Agriculture. https://doi.org/10.1016/j.aiia.2026.11.003

Image Credits: AI Generated

DOI: 10.1016/j.aiia.2026.11.003

Keywords: dairy cows, deep learning, behavior recognition, multi-object tracking, YOLOv10, ByteTrack, Gaussian interpolation, edge deployment, precision livestock farming, computer vision, Kalman filter, lightweight neural networks