Fish feeding intensity assessment (FFIA) aims to evaluate fish appetite changes during feeding, which is crucial in industrial aquaculture applications. Existing FFIA methods are limited by their robustness to noise, computational complexity, and the lack of public datasets for developing the models. To address these issues, we first introduce AV-FFIA, a new dataset containing 27,000 labeled audio and video clips that capture different levels of fish feeding intensity. Then, we introduce multi-modal approaches for FFIA by leveraging the models pre-trained on individual modalities and fused with data fusion methods. We perform benchmark studies of these methods on AV-FFIA, and demonstrate the advantages of the multi-modal approach over the single-modality based approach, especially in noisy environments. However, compared to the methods developed for individual modalities, the multimodal approaches may involve higher computational costs due to the need for independent encoders for each modality. To overcome this issue, we further present a novel unified mixed-modality based method for FFIA, termed as U-FFIA. U-FFIA is a single model capable of processing audio, visual, or audio-visual modalities, by leveraging modality dropout during training and knowledge distillation using the models pre-trained with data from single modality. We demonstrate that U-FFIA can achieve performance better than or on par with the state-of-the-art modality-specific FFIA models, with significantly lower computational overhead, enabling robust and efficient FFIA for improved aquaculture management.

Note to Practitioners: Feeding is one of the most important costs in aquaculture. However, current feeding machines usually operate with fixed thresholds or human experiences, lacking the ability to automatically adjust to fish feeding intensity. FFIA can evaluate the intensity changes in fish appetite during the feeding process and optimize the control strategies of the feeding machine to avoid inadequate feeding or overfeeding, thereby reducing the feeding cost and improving the well-being of fish in industrial aquaculture.

Four Different FFIA: The video frames show a clear relationship between the feeding intensity and the degree of fish aggregation, with higher intensity leading to denser fish grouping. Moreover, the continuous video footage reveals that the fish swimming speed and feeding activity increase with higher feeding intensity

Fish Feeding in Real-World:Fish feeding in ponds can be accomplished through automated machines or manual human methods. Feeding machines dispense predetermined amounts of food at set intervals, offering consistency and reduced labour. Human feeding allows for direct observation and immediate adjustment but requires more time and effort from the pond manager.

This work was supported by the National Natural Science Foundation of China ''Dynamic regulation mechanism of nitrogen in industrial aquaponics under the condition of asynchronous life cycle''[Grant no. 32373186], Digital Fishery Cross-Innovative Talent Training Program of the China Scholarship Council (DF-Project) and a Research Scholarship from the China Scholarship Council. Ethics approval for this study was obtained from the Welfare and Ethical Committee of China Agricultural University (Ref: AW30901202-5-1). For the purpose of open access, the authors have applied a Creative Commons Attribution (CC BY) licence to any author-accepted manuscript version arising.