TRex, a Fast Multi-animal Tracking System with Markerless Identification, and 2D Estimation of Posture and Visual Fields

Overview

Journal Elife

Specialty Biology

Date 2021 Feb 26

PMID 33634789

Citations 69

Authors

Tristan Walter

Iain D Couzin

Affiliations

Soon will be listed here.

Abstract

Automated visual tracking of animals is rapidly becoming an indispensable tool for the study of behavior. It offers a quantitative methodology by which organisms' sensing and decision-making can be studied in a wide range of ecological contexts. Despite this, existing solutions tend to be challenging to deploy in practice, especially when considering long and/or high-resolution video-streams. Here, we present TRex, a fast and easy-to-use solution for tracking a large number of individuals simultaneously using background-subtraction with real-time (60 Hz) tracking performance for up to approximately 256 individuals and estimates 2D visual-fields, outlines, and head/rear of bilateral animals, both in open and closed-loop contexts. Additionally, TRex offers highly accurate, deep-learning-based visual identification of up to approximately 100 unmarked individuals, where it is between 2.5 and 46.7 times faster, and requires 2-10 times less memory, than comparable software (with relative performance increasing for more organisms/longer videos) and provides interactive data-exploration within an intuitive, platform-independent graphical user-interface.

Citing Articles

Reynolds rules in swarm fly behavior based on KAN transformer tracking method.

Yang Q, Ji J, Jing R, Su H, Wang S, Guo A Sci Rep. 2025; 15(1):6982.

PMID: 40011603 PMC: 11865518. DOI: 10.1038/s41598-025-91674-w.

vmTracking enables highly accurate multi-animal pose tracking in crowded environments.

Azechi H, Takahashi S PLoS Biol. 2025; 23(2):e3003002.

PMID: 39928646 PMC: 11845028. DOI: 10.1371/journal.pbio.3003002.

Impact of developmental temperature on neural growth, connectivity, and function.

Zufle P, Batista L, Brandao S, DUva G, Daniel C, Martelli C Sci Adv. 2025; 11(3):eadp9587.

PMID: 39813340 PMC: 11734716. DOI: 10.1126/sciadv.adp9587.

Non-invasive eye tracking and retinal view reconstruction in free swimming schooling fish.

Wu R, Deussen O, Couzin I, Li L Commun Biol. 2024; 7(1):1636.

PMID: 39668195 PMC: 11638265. DOI: 10.1038/s42003-024-07322-y.

Segmentation tracking and clustering system enables accurate multi-animal tracking of social behaviors.

Tang C, Zhou Y, Zhao S, Xie M, Zhang R, Long X Patterns (N Y). 2024; 5(11):101057.

PMID: 39568468 PMC: 11573910. DOI: 10.1016/j.patter.2024.101057.

References

Branson K, Robie A, Bender J, Perona P, Dickinson M . High-throughput ethomics in large groups of Drosophila. Nat Methods. 2009; 6(6):451-7. PMC: 2734963. DOI: 10.1038/nmeth.1328. View

Lin T, Goyal P, Girshick R, He K, Dollar P . Focal Loss for Dense Object Detection. IEEE Trans Pattern Anal Mach Intell. 2018; 42(2):318-327. DOI: 10.1109/TPAMI.2018.2858826. View

Dell A, Bender J, Branson K, Couzin I, de Polavieja G, Noldus L . Automated image-based tracking and its application in ecology. Trends Ecol Evol. 2014; 29(7):417-28. DOI: 10.1016/j.tree.2014.05.004. View

Gernat T, Rao V, Middendorf M, Dankowicz H, Goldenfeld N, Robinson G . Automated monitoring of behavior reveals bursty interaction patterns and rapid spreading dynamics in honeybee social networks. Proc Natl Acad Sci U S A. 2018; 115(7):1433-1438. PMC: 5816157. DOI: 10.1073/pnas.1713568115. View

Hubel D, Wiesel T . Receptive fields of single neurones in the cat's striate cortex. J Physiol. 1959; 148:574-91. PMC: 1363130. DOI: 10.1113/jphysiol.1959.sp006308. View

Iwata H, Ebana K, Uga Y, Hayashi T . Genomic prediction of biological shape: elliptic Fourier analysis and kernel partial least squares (PLS) regression applied to grain shape prediction in rice (Oryza sativa L.). PLoS One. 2015; 10(3):e0120610. PMC: 4380318. DOI: 10.1371/journal.pone.0120610. View

Bianco I, Engert F . Visuomotor transformations underlying hunting behavior in zebrafish. Curr Biol. 2015; 25(7):831-46. PMC: 4386024. DOI: 10.1016/j.cub.2015.01.042. View

Walter T, Couzin I . TRex, a fast multi-animal tracking system with markerless identification, and 2D estimation of posture and visual fields. Elife. 2021; 10. PMC: 8096434. DOI: 10.7554/eLife.64000. View

Brembs B, Heisenberg M . The operant and the classical in conditioned orientation of Drosophila melanogaster at the flight simulator. Learn Mem. 2001; 7(2):104-15. PMC: 311324. DOI: 10.1101/lm.7.2.104. View

10.

Rosenthal S, Twomey C, Hartnett A, Wu H, Couzin I . Revealing the hidden networks of interaction in mobile animal groups allows prediction of complex behavioral contagion. Proc Natl Acad Sci U S A. 2015; 112(15):4690-5. PMC: 4403201. DOI: 10.1073/pnas.1420068112. View

11.

Strandburg-Peshkin A, Twomey C, Bode N, Kao A, Katz Y, Ioannou C . Visual sensory networks and effective information transfer in animal groups. Curr Biol. 2013; 23(17):R709-11. PMC: 4780851. DOI: 10.1016/j.cub.2013.07.059. View

12.

Sockman K, Schwabl H . Plasma corticosterone in nestling american kestrels: effects of age, handling stress, yolk androgens, and body condition. Gen Comp Endocrinol. 2001; 122(2):205-12. DOI: 10.1006/gcen.2001.7626. View

13.

Fukunaga T, Kubota S, Oda S, Iwasaki W . GroupTracker: Video tracking system for multiple animals under severe occlusion. Comput Biol Chem. 2015; 57:39-45. DOI: 10.1016/j.compbiolchem.2015.02.006. View

14.

Pereira T, Aldarondo D, Willmore L, Kislin M, Wang S, Murthy M . Fast animal pose estimation using deep neural networks. Nat Methods. 2018; 16(1):117-125. PMC: 6899221. DOI: 10.1038/s41592-018-0234-5. View

15.

Humphrey G, Khan S . Recognizing novel views of three-dimensional objects. Can J Psychol. 1992; 46(2):170-90. DOI: 10.1037/h0084320. View

16.

Bilotta J, Saszik S . The zebrafish as a model visual system. Int J Dev Neurosci. 2001; 19(7):621-9. DOI: 10.1016/s0736-5748(01)00050-8. View

17.

Romero-Ferrero F, Bergomi M, Hinz R, Heras F, de Polavieja G . idtracker.ai: tracking all individuals in small or large collectives of unmarked animals. Nat Methods. 2019; 16(2):179-182. DOI: 10.1038/s41592-018-0295-5. View

18.

Wild B, Dormagen D, Zachariae A, Smith M, Traynor K, Brockmann D . Social networks predict the life and death of honey bees. Nat Commun. 2021; 12(1):1110. PMC: 7889932. DOI: 10.1038/s41467-021-21212-5. View

19.

Perez-Escudero A, Vicente-Page J, Hinz R, Arganda S, de Polavieja G . idTracker: tracking individuals in a group by automatic identification of unmarked animals. Nat Methods. 2014; 11(7):743-8. DOI: 10.1038/nmeth.2994. View

20.

Hughey L, Hein A, Strandburg-Peshkin A, Jensen F . Challenges and solutions for studying collective animal behaviour in the wild. Philos Trans R Soc Lond B Biol Sci. 2018; 373(1746). PMC: 5882975. DOI: 10.1098/rstb.2017.0005. View