In the sprawling industrial shrimp farms of eastern China, where more than 20,000 cubic meters of recirculating water sustain billions of Pacific white shrimp, a quiet revolution is underway. Researchers have unveiled an artificial intelligence system that replaces one of aquaculture’s most labor-intensive and error-prone rituals—the manual feeding tray observation—with an autonomous pipeline that watches the shrimp, weighs them from photographs alone, and decides exactly how much food each pond should receive. The work, published in the journal Artificial Intelligence in Agriculture, represents one of the first times large language models and large vision models have been woven into the day-to-day decision loop of a commercial shrimp operation, and the results suggest the experience-driven era of aquaculture management may be giving way to something far more precise.
The challenge the researchers set out to solve is deceptively simple to describe and notoriously difficult to solve in practice. Feeding is the single most consequential decision in intensive shrimp farming. Underfeed the animals and growth slows while size variation among individuals widens, dragging down the value of the harvest. Overfeed them and the consequences compound rapidly: uneaten feed decomposes, water quality degrades, disease outbreaks become more likely, and production costs climb. In traditional operations, farm workers periodically hoist a circular mesh tray out of the pond by rope and visually judge whether leftover feed pellets remain, while separate teams capture hundreds of shrimp, drain them, and weigh them by hand to calculate growth rates and rations. Both practices depend heavily on individual judgment, produce results that are difficult to reproduce, and—the sampling in particular—inflict minutes of out-of-water stress on the very animals the farmers are trying to nurture.
The new system, developed and validated at the Dongying Tongwei Fishery facility in Shandong Province, which the authors describe as the world’s largest single-entity recirculating aquaculture shrimp base, automates nearly all of it. At its heart is a motorized feeding tray fitted with a five-megapixel industrial wide-dynamic-range camera. The tray rises automatically to a fixed height above the water surface, captures ten seconds of continuous imagery, and descends again, all without human intervention. From those frames, a cascade of machine learning models extracts exactly the information a skilled farmer would try to eyeball: how many shrimp are on the tray, how large they are, how much they weigh collectively, whether they appear lively, and whether any residual feed remains.
The technical machinery behind this pipeline is a carefully engineered stack of models. The team trained a YOLOv8 segmentation network on 2,687 annotated shrimp images containing 18,579 labeled instances, collected across two workshops and an entire growth cycle, from newly stocked juveniles to market-sized animals. The segmentation masks do more than simply outline each animal. By computing the pixel count within each mask, the system derives each shrimp’s projected area; by skeletonizing the mask and tracing the longest path along the body’s main axis, it obtains the visual total length, a measurement designed to tolerate the constant bending that shrimp bodies undergo underwater. Those pixel-level measurements are then converted into true physical units through a calibration factor specific to the tray environment, and fed into a LightGBM regression model trained on 843 manually measured shrimp spanning weights from 0.1 to 25 grams. The result is a non-contact weight estimate for every animal in the frame—no nets, no draining, no stress.
The researchers also tackled a problem that has long undermined computer vision in aquaculture: messy water. Underwater imaging of shrimp is plagued by surface reflections, bubbles, suspended particles, and uneven lighting, and in high-density industrial tanks these problems are amplified by frequent clustering and occlusion among individuals. By training on data gathered from real production ponds rather than clean laboratory tanks, and by standardizing the tray’s proportional size in every image to dampen background scale variation, the team built a system robust enough for the environments where it actually needs to work. The pipeline even estimates population vitality by computing the proportion of shrimp lying dorsal-up rather than lateral, a posture-based proxy for how energetically the animals are moving.
Residual feed detection is handled by a YOLOv8 classification model trained on 2,476 tray images, 1,331 of which contained visible feed residue. But the authors went further, enlisting large vision models to describe what the classifier detects. Four frontier vision-language models—GLM-4.5V, Qwen3-VL-Plus, and two generations of ByteDance’s Doubao vision models—were evaluated on a set of 337 independently annotated images spanning low, moderate, and high residue levels. Using a standardized few-shot prompting scheme with nine reference images, each model was asked to categorize the residue and generate structured descriptions of spatial coverage, particle density, and overall status. The approach turns a binary yes-or-no classification into a rich semantic account of feeding state, the kind of nuanced description an experienced pond manager might offer, but generated automatically and consistently.
The most distinctive component of the system, however, is its feeding decision engine, which blends classical machine learning with a large language model acting as an adjudicator. The team assembled a production dataset of more than 2,000 daily records spanning farming batches from February 2024 to August 2025, each containing 17 parameters including water temperature, dissolved oxygen, pH, ammonia nitrogen, nitrite, water exchange rates, culture age, body weight, and daily feed totals. Against this data they benchmarked 36 regression models across five architectural families, from support vector regression and neural networks to gradient-boosted tree ensembles and cutting-edge architectures such as Kolmogorov-Arnold Networks and Transformers. Notably, the tree-based ensembles crushed the time-series alternatives: even though shrimp farming data exhibit temporal periodicity, models such as LSTM, GRU, TCN, and Transformer all trailed far behind the regression approaches on the same prediction task.
The standout performer was the Bagging ensemble, which achieved a coefficient of determination of 95.5 percent and a mean absolute percentage error of just 8.6 percent on daily total feed prediction, with a correlation of 0.978 against observed values in the Taylor diagram analysis. Among the top four models—Bagging, XGBoost, CatBoost, and LightGBM—LightGBM offered the best training time, inference speed, and storage footprint, making it the natural choice for the deployed system. After hyperparameter optimization using Bayesian optimization, particle swarm optimization, and several other metaheuristic search strategies, the final production model emerged as a Bayesian-optimized LightGBM, validated on an independent workshop cycle that had been withheld from training entirely, confirming that the model generalizes across time and scenarios rather than merely memorizing a particular batch’s quirks.
What elevates the architecture from a well-tuned regression model to something genuinely new is the decision layer built on top of it. The system operates a dual-path mechanism. When no human input is provided, it defaults to the data-driven Bayesian-LightGBM prediction, cross-checked against a parallel empirical calculation that applies body-weight-specific feeding rate coefficients to the shrimp loading capacity and tank volume. But when a farm manager supplies remarks—perhaps about a recent health observation or an expected weather change—the Qwen-Plus large language model is activated as a kind of “chief expert.” Through prompt engineering, the LLM aggregates the quantitative model output, historical records drawn from a cloud-hosted knowledge base, and curated aquaculture expertise to render a final adjudicated feeding amount. The entire workflow is exposed through a RESTful API built on Flask, and the physical tray interface is conversational: operators issue voice commands parsed by streaming speech recognition, the vision model analyzes the resulting footage, and speech synthesis delivers the findings back aloud.
The implications extend well beyond a single facility. Pacific white shrimp, Litopenaeus vannamei, account for more than half of global shrimp production according to the FAO, and intensive recirculating systems are expanding rapidly as a sustainable alternative to pond farming. Feeding decisions sit at the intersection of economics, animal welfare, and environmental impact, and the study’s authors argue that their practice-tested pipeline offers a realistic template for intelligent transformation across the sector. The system’s voice-driven, multimodal interface is particularly notable: it does not demand that farmers become data scientists, but instead allows them to interact with the AI in natural language, asking it to raise the tray, inspect the shrimp, or explain its reasoning.
There are, of course, caveats. The LLM’s adjudication role was validated within a single commercial operation, and the authors acknowledge that rapid model turnover in the vision-language landscape—one of their evaluated models lost API access before the manuscript was even revised—poses an ongoing maintenance challenge. But the broader trajectory is unmistakable. Just as computer vision transformed livestock farming by enabling non-contact weight assessment of pigs and feeding-intensity estimation in fish, the fusion of segmentation networks, gradient-boosted predictors, and language-model reasoning now appears ready to do the same for shrimp. In the ponds of Dongying, the rope-and-mesh tray that generations of farmers have hauled up by hand is still there—but now it rises on its own, looks, thinks, and answers back.
Subject of Research: Development and commercial validation of a multimodal AI system combining computer vision, large vision models, large language model agents, and optimized machine learning regression for automated shrimp growth assessment and intelligent feeding decisions in industrial recirculating aquaculture.
Subject of Research: Agriculture
Article Title: From computer vision to LLM agents: A multimodal AI system for shrimp growth assessment and intelligent feeding decisions
Article References: Zhao, H., Liu, M., Wang, Q., Jiang, K., Wang, B., & Wang, L. (2026). From computer vision to LLM agents: A multimodal AI system for shrimp growth assessment and intelligent feeding decisions. Artificial Intelligence in Agriculture. https://doi.org/10.1016/j.aiia.2026.08.011
Image Credits: AI Generated
DOI: 10.1016/j.aiia.2026.08.011
Keywords: Artificial intelligence in aquaculture, Pacific white shrimp, computer vision, YOLOv8 segmentation, LightGBM, large language models, multimodal AI, intelligent feeding, recirculating aquaculture system, precision livestock farming
Cite Scienmag News
APA MLA Chicago
Alan Morgan. (September 3, 2026). Multimodal AI system tracks shrimp growth and guides smart feeding decisions. Scienmag. https://scienmag.com/multimodal-ai-system-tracks-shrimp-growth-and-guides-smart-feeding-decisions/
Alan Morgan. “Multimodal AI system tracks shrimp growth and guides smart feeding decisions.” Scienmag, 3 September 2026, https://scienmag.com/multimodal-ai-system-tracks-shrimp-growth-and-guides-smart-feeding-decisions/. Accessed 3 September 2026.
Alan Morgan. “Multimodal AI system tracks shrimp growth and guides smart feeding decisions.” Scienmag. September 3, 2026. https://scienmag.com/multimodal-ai-system-tracks-shrimp-growth-and-guides-smart-feeding-decisions/
Copy citation Download RIS
Tags: AI-based shrimp weight estimationAI-driven growth tracking in aquacultureAI-powered fish farm managementAI-powered shrimp growth monitoringAquaculture AI systemsaquaculture automationautonomous shrimp monitoringautonomous water quality managementindustrial shrimp farm automationlarge language and vision models in aquaculturelarge language and vision models in farmingmultimodal AI in aquaculturemultimodal artificial intelligence in aquacultureprecision aquaculture technologyreal-time decision making in aquacultureshrimp farming automationsmart feeding decision systemssustainable aquaculture practicessustainable aquaculture practices with AIwater quality and disease prevention through AIwater quality management in shrimp farms
