A Method for Obtaining 3D Point Cloud Data by Combining 2D Image Segmentation and Depth Information of Pigs

Overview

Journal Animals (Basel)

Date 2023 Aug 12

PMID 37570282

Authors

Shunli Wang

Honghua Jiang

Yongliang Qiao

Shuzhen Jiang

Affiliations

Soon will be listed here.

Abstract

This paper proposes a method for automatic pig detection and segmentation using RGB-D data for precision livestock farming. The proposed method combines the enhanced YOLOv5s model with the Res2Net bottleneck structure, resulting in improved fine-grained feature extraction and ultimately enhancing the precision of pig detection and segmentation in 2D images. Additionally, the method facilitates the acquisition of 3D point cloud data of pigs in a simpler and more efficient way by using the pig mask obtained in 2D detection and segmentation and combining it with depth information. To evaluate the effectiveness of the proposed method, two datasets were constructed. The first dataset consists of 5400 images captured in various pig pens under diverse lighting conditions, while the second dataset was obtained from the UK. The experimental results demonstrated that the improved YOLOv5s_Res2Net achieved a mAP@0.5:0.95 of 89.6% and 84.8% for both pig detection and segmentation tasks on our dataset, while achieving a mAP@0.5:0.95 of 93.4% and 89.4% on the Edinburgh pig behaviour dataset. This approach provides valuable insights for improving pig management, conducting welfare assessments, and estimating weight accurately.

References

Ren S, He K, Girshick R, Sun J . Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell. 2016; 39(6):1137-1149. DOI: 10.1109/TPAMI.2016.2577031. View

Zhang L, Gray H, Ye X, Collins L, Allinson N . Automatic Individual Pig Detection and Tracking in Pig Farms. Sensors (Basel). 2019; 19(5). PMC: 6427794. DOI: 10.3390/s19051188. View

Tu S, Yuan W, Liang Y, Wang F, Wan H . Automatic Detection and Segmentation for Group-Housed Pigs Based on PigMS R-CNN. Sensors (Basel). 2021; 21(9). PMC: 8125813. DOI: 10.3390/s21093251. View

Lin T, Goyal P, Girshick R, He K, Dollar P . Focal Loss for Dense Object Detection. IEEE Trans Pattern Anal Mach Intell. 2018; 42(2):318-327. DOI: 10.1109/TPAMI.2018.2858826. View

Bo Z, Atif O, Lee J, Park D, Chung Y . GAN-Based Video Denoising with Attention Mechanism for Field-Applicable Pig Detection System. Sensors (Basel). 2022; 22(10). PMC: 9143193. DOI: 10.3390/s22103917. View

Wutke M, Heinrich F, Das P, Lange A, Gentz M, Traulsen I . Detecting Animal Contacts-A Deep Learning-Based Pig Detection and Tracking Approach for the Quantification of Social Contacts. Sensors (Basel). 2021; 21(22). PMC: 8619108. DOI: 10.3390/s21227512. View

Wang S, Jiang H, Qiao Y, Jiang S, Lin H, Sun Q . The Research Progress of Vision-Based Artificial Intelligence in Smart Pig Farming. Sensors (Basel). 2022; 22(17). PMC: 9460267. DOI: 10.3390/s22176541. View

Woonwong Y, Do Tien D, Thanawongnuwech R . The Future of the Pig Industry After the Introduction of African Swine Fever into Asia. Anim Front. 2020; 10(4):30-37. PMC: 7596796. DOI: 10.1093/af/vfaa037. View

Camuffo E, Mari D, Milani S . Recent Advancements in Learning Algorithms for Point Clouds: An Updated Overview. Sensors (Basel). 2022; 22(4). PMC: 8963024. DOI: 10.3390/s22041357. View

10.

Yu H, Lee K, Morota G . Forecasting dynamic body weight of nonrestrained pigs from images using an RGB-D sensor camera. Transl Anim Sci. 2021; 5(1):txab006. PMC: 7906448. DOI: 10.1093/tas/txab006. View

11.

de Paulo Farias D, Dos Santos Gomes M . COVID-19 outbreak: What should be done to avoid food shortages?. Trends Food Sci Technol. 2020; 102:291-292. PMC: 7293531. DOI: 10.1016/j.tifs.2020.06.007. View

12.

Benjamin M, Yik S . Precision Livestock Farming in Swine Welfare: A Review for Swine Practitioners. Animals (Basel). 2019; 9(4). PMC: 6523486. DOI: 10.3390/ani9040133. View

13.

Zhang J, Zhuang Y, Ji H, Teng G . Pig Weight and Body Size Estimation Using a Multiple Output Regression Convolutional Neural Network: A Fast and Fully Automatic Method. Sensors (Basel). 2021; 21(9). PMC: 8124602. DOI: 10.3390/s21093218. View

14.

Gao S, Cheng M, Zhao K, Zhang X, Yang M, Torr P . Res2Net: A New Multi-Scale Backbone Architecture. IEEE Trans Pattern Anal Mach Intell. 2019; 43(2):652-662. DOI: 10.1109/TPAMI.2019.2938758. View

15.

Xie J, Zheng Z, Gao R, Wang W, Zhu S, Wu Y . Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis. IEEE Trans Pattern Anal Mach Intell. 2020; 44(5):2468-2484. DOI: 10.1109/TPAMI.2020.3045010. View

16.

Kim J, Suh Y, Lee J, Chae H, Ahn H, Chung Y . EmbeddedPigCount: Pig Counting with Video Object Detection and Tracking on an Embedded Board. Sensors (Basel). 2022; 22(7). PMC: 9002707. DOI: 10.3390/s22072689. View

17.

Lu J, Wang W, Zhao K, Wang H . Recognition and segmentation of individual pigs based on Swin Transformer. Anim Genet. 2022; 53(6):794-802. DOI: 10.1111/age.13259. View