Psudo: Exploring Multi-Channel Biomedical Image Data with Spatially and Perceptually Optimized Pseudocoloring

Overview

Journal bioRxiv

Date 2024 Apr 25

PMID 38659870

Authors

Simon Warchol

Jakob Troidl

Jeremy Muhlich

Robert Krueger

John Hoffer

Tica Lin

Johanna Beyer

Elena Glassman

Peter K Sorger

Hanspeter Pfister

Affiliations

Soon will be listed here.

Abstract

Over the past century, multichannel fluorescence imaging has been pivotal in myriad scientific breakthroughs by enabling the spatial visualization of proteins within a biological sample. With the shift to digital methods and visualization software, experts can now flexibly pseudocolor and combine image channels, each corresponding to a different protein, to explore their spatial relationships. We thus propose psudo, an interactive system that allows users to create optimal color palettes for multichannel spatial data. In psudo, a novel optimization method generates palettes that maximize the perceptual differences between channels while mitigating confusing color blending in overlapping channels. We integrate this method into a system that allows users to explore multi-channel image data and compare and evaluate color palettes for their data. An interactive lensing approach provides on-demand feedback on channel overlap and a color confusion metric while giving context to the underlying channel values. Color palettes can be applied globally or, using the lens, to local regions of interest. We evaluate our palette optimization approach using three graphical perception tasks in a crowdsourced user study with 150 participants, showing that users are more accurate at discerning and comparing the underlying data using our approach. Additionally, we showcase psudo in a case study exploring the complex immune responses in cancer tissue data with a biologist.

References

Kuhne L, Giesen J, Zhang Z, Ha S, Mueller K . A Data-Driven Approach to Hue-Preserving Color-Blending. IEEE Trans Vis Comput Graph. 2015; 18(12):2122-9. DOI: 10.1109/TVCG.2012.186. View

Allan C, Burel J, Moore J, Blackburn C, Linkert M, Loynton S . OMERO: flexible, model-driven data management for experimental biology. Nat Methods. 2012; 9(3):245-53. PMC: 3437820. DOI: 10.1038/nmeth.1896. View

Szafir D . Modeling Color Difference for Visualization Design. IEEE Trans Vis Comput Graph. 2017; 24(1):392-401. DOI: 10.1109/TVCG.2017.2744359. View

Dunn K, Kamocka M, McDonald J . A practical guide to evaluating colocalization in biological microscopy. Am J Physiol Cell Physiol. 2011; 300(4):C723-42. PMC: 3074624. DOI: 10.1152/ajpcell.00462.2010. View

Quadri G, Rosen P . A Survey of Perception-Based Visualization Studies by Task. IEEE Trans Vis Comput Graph. 2021; 28(12):5026-5048. DOI: 10.1109/TVCG.2021.3098240. View

Weiss P . 2008 Nobel Prize in Chemistry: green fluorescent protein, its variants and implications. ACS Nano. 2009; 2(10):1977. DOI: 10.1021/nn800671h. View

Rashid R, Chen Y, Hoffer J, Muhlich J, Lin J, Krueger R . Narrative online guides for the interpretation of digital-pathology images and tissue-atlas data. Nat Biomed Eng. 2021; 6(5):515-526. PMC: 9079188. DOI: 10.1038/s41551-021-00789-8. View

Reda K . Rainbow Colormaps: What are they good and bad for?. IEEE Trans Vis Comput Graph. 2022; PP. DOI: 10.1109/TVCG.2022.3214771. View

Krueger R, Beyer J, Jang W, Kim N, Sokolov A, Sorger P . Facetto: Combining Unsupervised and Supervised Learning for Hierarchical Phenotype Analysis in Multi-Channel Image Data. IEEE Trans Vis Comput Graph. 2019; 26(1):227-237. PMC: 7045445. DOI: 10.1109/TVCG.2019.2934547. View

10.

Wang L, Giesen J, McDonnell K, Zolliker P, Mueller K . Color design for illustrative visualization. IEEE Trans Vis Comput Graph. 2008; 14(6):1739-46. DOI: 10.1109/TVCG.2008.118. View

11.

Tan W, Nerurkar S, Cai H, Ng H, Wu D, Wee Y . Overview of multiplex immunohistochemistry/immunofluorescence techniques in the era of cancer immunotherapy. Cancer Commun (Lond). 2020; 40(4):135-153. PMC: 7170662. DOI: 10.1002/cac2.12023. View

12.

Thierry G, Athanasopoulos P, Wiggett A, Dering B, Kuipers J . Unconscious effects of language-specific terminology on preattentive color perception. Proc Natl Acad Sci U S A. 2009; 106(11):4567-70. PMC: 2657373. DOI: 10.1073/pnas.0811155106. View

13.

Lin J, Izar B, Wang S, Yapp C, Mei S, Shah P . Highly multiplexed immunofluorescence imaging of human tissues and tumors using t-CyCIF and conventional optical microscopes. Elife. 2018; 7. PMC: 6075866. DOI: 10.7554/eLife.31657. View

14.

Kumar A, Zhang X, Xin H, Yan H, Huang X, Xu W . RadVolViz: An Information Display-Inspired Transfer Function Editor for Multivariate Volume Visualization. IEEE Trans Vis Comput Graph. 2023; 30(8):4464-4479. DOI: 10.1109/TVCG.2023.3263856. View

15.

He H, Li J, Xiao Q, Jiang S, Yang Y, Zhi S . Language and Color Perception: Evidence From Mongolian and Chinese Speakers. Front Psychol. 2019; 10:551. PMC: 6426779. DOI: 10.3389/fpsyg.2019.00551. View

16.

Huang L, Pashler H . A Boolean map theory of visual attention. Psychol Rev. 2007; 114(3):599-631. DOI: 10.1037/0033-295X.114.3.599. View

17.

Kirkpatrick S, Gelatt Jr C, Vecchi M . Optimization by simulated annealing. Science. 1983; 220(4598):671-80. DOI: 10.1126/science.220.4598.671. View

18.

Liu S, Maljovec D, Wang B, Bremer P, Pascucci V . Visualizing High-Dimensional Data: Advances in the Past Decade. IEEE Trans Vis Comput Graph. 2017; 23(3):1249-1268. DOI: 10.1109/TVCG.2016.2640960. View

19.

Reda K, Szafir D . Rainbows Revisited: Modeling Effective Colormap Design for Graphical Inference. IEEE Trans Vis Comput Graph. 2020; 27(2):1032-1042. DOI: 10.1109/TVCG.2020.3030439. View

20.

Heinzmann K, Carter L, Lewis J, Aboagye E . Multiplexed imaging for diagnosis and therapy. Nat Biomed Eng. 2019; 1(9):697-713. DOI: 10.1038/s41551-017-0131-8. View