Agricultural engineers and animal scientists have developed an advanced artificial intelligence surveillance architecture capable of performing continuous, long-term, and non-contact monitoring of individual pig behaviors within crowded commercial pen enclosures. Published in the journal Artificial Intelligence in Agriculture, the study presents an innovative dual-network framework that couples an Improved YOLO11 object detection architecture with an Improved StrongSORT multi-object tracking algorithm. Operating on top-view video streams recorded by fixed overhead cameras, the system systematically resolves the long-standing computational bottlenecks that have historically hampered automated livestock management. By simultaneously localizing individual animals, classifying five primary physical behaviors, and maintaining uninterrupted identity persistence despite intense physical contact and mutual visual occlusion, the engineering pipeline achieved a mean average precision of 99.1 percent for multi-behavior recognition alongside a multiple object tracking accuracy of 98.70 percent. This integrated vision pipeline provides precision farming operations with an automated diagnostic tool to track health, welfare, and circadian rhythm variations across individual animals without requiring invasive wearable hardware.
Understanding individual animal behavior serves as a vital diagnostic indicator of physiological homeostasis, early disease onset, and general herd welfare, capturing essential functional activities such as feeding, standing, climbing, and resting. Traditionally, monitoring behavioral dynamics in swine production facilities has relied on direct visual observation by trained personnel, an approach that is labor intensive, inherently subjective, and practically impossible to scale across industrial herds. While recent precision livestock engineering has incorporated radio-frequency identification tags combined with automated performance-testing feeding stations to log nutritional intake, these physical sensor apparatuses present severe operational drawbacks. Physical tags and wearable monitors can elicit acute stress responses in pigs, remain largely restricted to logging feeding interactions, and frequently detach during social contact or pen abrasion, leading to irrecoverable data loss. Video surveillance offers an appealing non-invasive alternative, yet earlier computer vision techniques relying on foreground segmentation and handcrafted geometric parameters suffered from severe sensitivity to changing ambient lighting and dynamic background contamination. Although modern deep neural networks have established reliable classification benchmarks at the aggregate pen level, extracting continuous individual-level behavior profiles has remained an unaddressed engineering hurdle.
To overcome the limitations of prior vision systems, the research team established an extensive empirical data collection protocol within an operational commercial swine breeding facility located in Wuhan. The experimental setup monitored group-housed pigs maintained in pens containing populations of nine to ten individuals, where animals retained unrestricted access to automated feeding stations delivering feed into troughs on demand. Video footage was captured during September and October 2021 using high-definition Hikvision DS-2CD3346FWDA3-1 optical cameras mounted three meters vertically above the geometrical center of each enclosure. This overhead installation angle provided an unobstructed, perpendicular top-down viewing angle that encompassed all distinct functional zones within the pens, including high-activity feeding troughs and communal resting areas. The recording devices captured continuous full-color video streams encoded in MP4 format at an ultra-high resolution of 2560 by 1440 pixels and a steady capture speed of 25 frames per second. These calibrated video streams provided the sharp spatial and temporal fidelity required to distinguish subtle behavioral postures, minor limb adjustments, and close physical interactions among visually uniform animals under variable illumination.
From this extensive surveillance repository, the researchers established two separate, rigorously structured benchmark datasets dedicated respectively to behavior detection and continuous multi-target identity tracking. To construct the dedicated behavior detection dataset, investigators isolated 150 distinct video segments lasting between two and three minutes each, ensuring balanced representation across five standardized behavioral categories: feeding, standing, climbing, prone lying, and side lying. Frames were extracted at standardized three-second intervals to eliminate redundant temporal correlations, and subsequent manual cleaning yielded 7,220 validated images. Standardized annotations were produced using LabelMe software through a rigorous consensus protocol overseen by an animal behavior specialist, with bounding boxes demarcating physical target boundaries and assigned posture labels. To prevent data contamination stemming from adjacent frame resemblance, the detection dataset was split strictly at the raw video-clip boundary into training, validation, and testing partitions containing 5,776 images, 722 images, and 722 images, respectively, maintaining an eight-to-one-to-one proportion.
In parallel, the researchers engineered a specialized tracking dataset designed to evaluate multi-object identity persistence across challenging temporal horizons under varying frame rates. They manually annotated two hours of continuous surveillance video recorded across two experimental enclosures, designated as Pen 2 and Pen 3, utilizing DarkLabel software to assign and maintain permanent target identifiers for every individual across consecutive frames. Pen 2 housed an active population of 10 pigs and supplied two 30-minute sequences subsampled down to a sparse temporal frequency of one frame per second to evaluate algorithmic persistence across significant frame-to-frame displacements. Pen 3 contained nine pigs and provided four 15-minute sequences evaluated at the native frame rate of 25 frames per second. Because pigs possess highly similar skin textures, lack distinguishing individual pigmentation patterns, and constantly engage in mutual contact, maintaining invariant ID tags across these extended validation sequences established an exceptionally demanding benchmark for tracking consistency, identity switch suppression, and occlusion recovery.
The first core pillar of the newly introduced dual-network framework is the Improved YOLO11 multi-behavior detection architecture, engineered to optimize the trade-off between localization precision and computing overhead. Built upon the standard Backbone-Neck-Head topology, the baseline network was strategically redesigned at both the modular and topological network levels. To eliminate redundant channel-wise arithmetic without compromising feature representation, the developers restructured the Bottleneck units of the C3k2 modules by introducing grouped convolutions, generating an optimized processing unit designated as C3k2-I. Furthermore, the cross-scale concatenation operations were augmented to create Concat-I, a specialized feature aggregation module that combines zero-mean Gaussian noise perturbation with depthwise separable convolutions and learnable sigmoid-gated dynamic weighting parameters. The noise perturbation, implemented with a standard deviation of 0.01 during backpropagation training, fortifies the network against feature corruption caused by crowded pig overlap. Finally, these modules were linked within an enhanced Neck-I topology featuring dedicated cross-scale interaction pathways that fuse shallow spatial resolution with rich, deep semantic abstractions.
Quantitative evaluations across the standardized detection test suite proved that Improved YOLO11 substantially outperformed both historical baselines and contemporary state-of-the-art vision models. In comparative benchmarks conducted under identical hardware environments using dual NVIDIA RTX 4090 GPUs, Improved YOLO11 achieved a mean average precision at an intersection-over-union of 0.50 of 99.1 percent and an overall mAP across the 0.50 to 0.95 range of 0.877. This performance surpassed RT-DETRv2, which scored 97.6 percent and 0.864, as well as YOLOv10s, YOLOv8s, YOLOv5s, DETR-R50, and Faster R-CNN. Ablation experiments demonstrated that C3k2-I decreased network parameters from the baseline 9.41 million down to 7.92 million while elevating detection accuracy, and the complete unified network containing Neck-I and Concat-I operated at an economical 8.95 million parameters and 20.7 GFLOPs. Visual inspections confirmed that while conventional models frequently suffered from target omission or misclassified occluded feeding and climbing postures as generic standing, Improved YOLO11 maintained complete bounding box localization and precise posture categorization during severe physical aggregation.
The second pillar of the framework is Improved StrongSORT, a multi-object tracking algorithm specifically augmented to resolve identity switching during severe body overlap and prolonged visual occlusion. Baseline StrongSORT couples motion forecasting via an NSA Kalman filter with visual appearance embeddings extracted using a BoT re-identification model operating on a ResNeSt50 backbone, matching detections through Hungarian optimization and bounding-box intersection-over-union metrics. However, when pigs crowd together inside feeding stations or mount one another during climbing events, bounding-box displacement and abrupt posture changes frequently shatter appearance similarity and spatial overlap. To resolve this vulnerability, the investigators capitalized on the biological and operational reality of enclosed commercial pig farming: pen populations remain structurally constant without spontaneous exits or entries. They integrated an additional association module that computes the normalized Euclidean distance between the spatial center points of remaining unmatched detections and Kalman-predicted trajectory centroids, executing structured track retention and conservative ID assignment across different candidate population scenarios.
When subjected to empirical tracking evaluation on the Pen 3 test sequences, Improved StrongSORT delivered extraordinary performance gains over existing industry-standard trackers. The enhanced algorithm achieved a multiple object tracking accuracy of 98.70 percent, representing an improvement of more than ten percentage points over the next closest competitor, OC-SORT, which posted an 88.69 percent accuracy. Furthermore, Improved StrongSORT recorded an identification F1-score of 95.70 percent, a higher order tracking accuracy of 96.27 percent, and suppressed total identity switches down to just 10 occurrences across the entire multi-video test sequence, compared to 37 switches for baseline StrongSORT, 46 for OC-SORT, 47 for ByteTrack, and 54 for DeepSORT. In cross-pen generalization experiments conducted on Pen 2 under a sparse one-frame-per-second sampling rate, Improved StrongSORT sustained its association reliability, securing a 97.20 percent MOTA, a 94.59 percent IDF1 score, and limiting identity switches to 16, whereas DeepSORT experienced 610 identity switches. Stepwise ablation confirmed that combining center-point distance associations with pen-occupancy retention rules was directly responsible for eliminating trajectory fragmentation.
To validate the practical utility of the complete dual-network framework in real-world livestock operations, the researchers deployed the integrated system to reconstruct the behavioral duration profiles of nine individual pigs across distinct daily operational intervals: morning, noon, and afternoon. The automated pipeline successfully generated uninterrupted behavior sequences, demonstrating that resting postures dominated the total time budget across all observation periods. Activity peaked during the morning window, characterized by extended feeding bouts as pigs sequentially visited the feeding station, whereas the midday period exhibited a collective shift toward side lying accompanied by substantial drops in feeding, standing, and climbing interactions. More importantly, the system exposed critical behavioral variations between individual animals that group-level statistics invariably obscure; for instance, Pig ID 4 displayed high activity with prolonged feeding and frequent climbing, whereas Pig ID 9 exhibited marked lethargy, spending the vast majority of time resting in side and prone postures with minimal trough interaction. These granular, uninterrupted individual profiles demonstrate that the dual-network vision system can serve as an autonomous diagnostic framework for precision livestock management, with planned future developments focusing on edge-device deployment and multi-herd validation.
Subject of Research: Automated individual pig behavior detection and multi-object tracking in group-housed livestock pens using computer vision
Article Title: An innovative dual-network framework for monitoring multiple behaviors of individual pigs in group-housed environments
Article References: Shi, W., Cao, J., Wang, S., Li, J., Fu, Y., Li, G., Xu, X., Li, X., & Wang, H. (2026). An innovative dual-network framework for monitoring multiple behaviors of individual pigs in group-housed environments. Artificial Intelligence in Agriculture. https://doi.org/10.1016/j.aiia.2026.11.007
Image Credits: AI Generated
DOI: 10.1016/j.aiia.2026.11.007
Keywords: Precision Livestock Farming, Computer Vision, YOLO11, StrongSORT, Multi-Object Tracking, Object Detection, Pig Behavior Monitoring, Animal Welfare, Group Housing, Deep Learning, Convolutional Neural Networks, Trajectory Tracking, Identity Switches, Ablation Study, Agricultural Automation, Overhead Video Surveillance
News Source: Blake Davidson. (November 19, 2026). Dual-Network Computer Vision Framework Accurately Tracks Individual Group-Housed Pig Behaviors. Scienmag.



