The Potential of Short-wave Infrared Hyperspectral Imaging and Deep Learning for Dietary Assessment: a Prototype on Predicting Closed Sandwiches Fillings

Overview

Journal Front Nutr

Date 2025 Feb 3

PMID 39897532

Authors

Esther Kok

Aneesh Chauhan

Michele Tufano

Edith Feskens

Guido Camps

Affiliations

Soon will be listed here.

Abstract

Introduction: Accurate measurement of dietary intake without interfering in natural eating habits is a long-standing problem in nutritional epidemiology. We explored the applicability of hyperspectral imaging and machine learning for dietary assessment of home-prepared meals, by building a proof-of-concept, which automatically detects food ingredients inside closed sandwiches.

Methods: Individual spectra were selected from 24 hyperspectral images of assembled closed sandwiches, measured in a spectral range of 1116.14 nm to 1670.62 nm over 108 bands, pre-processed with Standard Normal Variate filtering, derivatives, and subsampling, and fed into multiple algorithms, among which PLS-DA, multiple classifiers, and a simple neural network.

Results: The resulting best performing models had an accuracy score of ~80% for predicting type of bread, ~60% for butter, and ~ 28% for filling type. We see that the main struggle in predicting the fillings lies with the spreadable fillings, meaning the model may be focusing on structural aspects and not nutritional composition.

Discussion: Further analysis on non-homogeneous mixed food items, using computer vision techniques, will contribute toward a generalizable system. While there are still significant technical challenges to overcome before such a system can be routinely implemented in studies of free-living subjects, we believe it holds promise as a future tool for nutrition research and population intake monitoring.

References

Shim J, Oh K, Kim H . Dietary assessment methods in epidemiologic studies. Epidemiol Health. 2014; 36:e2014009. PMC: 4154347. DOI: 10.4178/epih/e2014009. View

Barbin D, ElMasry G, Sun D, Allen P . Near-infrared hyperspectral imaging for grading and classification of pork. Meat Sci. 2011; 90(1):259-68. DOI: 10.1016/j.meatsci.2011.07.011. View

Furbank R, Silva-Perez V, Evans J, Condon A, Estavillo G, He W . Wheat physiology predictor: predicting physiological traits in wheat from hyperspectral reflectance measurements using deep learning. Plant Methods. 2021; 17(1):108. PMC: 8527791. DOI: 10.1186/s13007-021-00806-6. View

Zhao X, Xu X, Li X, He X, Yang Y, Zhu S . Emerging trends of technology-based dietary assessment: a perspective study. Eur J Clin Nutr. 2020; 75(4):582-587. DOI: 10.1038/s41430-020-00779-0. View

Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E . Deep Learning for Computer Vision: A Brief Review. Comput Intell Neurosci. 2018; 2018:7068349. PMC: 5816885. DOI: 10.1155/2018/7068349. View

Manley M . Near-infrared spectroscopy and hyperspectral imaging: non-destructive analysis of biological materials. Chem Soc Rev. 2014; 43(24):8200-14. DOI: 10.1039/c4cs00062e. View

Chen H, Jia W, Li Z, Sun Y, Sun M . 3D/2D Model-to-Image Registration for Quantitative Dietary Assessment. Proc IEEE Annu Northeast Bioeng Conf. 2014; 2012:95-96. PMC: 4058710. DOI: 10.1109/NEBC.2012.6206979. View

Burrows T, Ho Y, Rollo M, Collins C . Validity of Dietary Assessment Methods When Compared to the Method of Doubly Labeled Water: A Systematic Review in Adults. Front Endocrinol (Lausanne). 2020; 10:850. PMC: 6928130. DOI: 10.3389/fendo.2019.00850. View

Feng Y, Sun D . Application of hyperspectral imaging in food safety inspection and control: a review. Crit Rev Food Sci Nutr. 2012; 52(11):1039-58. DOI: 10.1080/10408398.2011.651542. View

10.

Tahir G, Loo C . A Comprehensive Survey of Image-Based Food Recognition and Volume Estimation Methods for Dietary Assessment. Healthcare (Basel). 2021; 9(12). PMC: 8700885. DOI: 10.3390/healthcare9121676. View

11.

Chu H, Zhang C, Wang M, Gouda M, Wei X, He Y . Hyperspectral imaging with shallow convolutional neural networks (SCNN) predicts the early herbicide stress in wheat cultivars. J Hazard Mater. 2021; 421:126706. DOI: 10.1016/j.jhazmat.2021.126706. View

12.

Trijsburg L, de Vries J, Boshuizen H, Hulshof P, Hollman P, van t Veer P . Comparison of duplicate portion and 24 h recall as reference methods for validating a FFQ using urinary markers as the estimate of true intake. Br J Nutr. 2015; 114(8):1304-12. DOI: 10.1017/S0007114515002871. View

13.

Cade J . Measuring diet in the 21st century: use of new technologies. Proc Nutr Soc. 2016; 76(3):276-282. DOI: 10.1017/S0029665116002883. View

14.

Zhou X, Sun J, Tian Y, Yao K, Xu M . Detection of heavy metal lead in lettuce leaves based on fluorescence hyperspectral technology combined with deep learning algorithm. Spectrochim Acta A Mol Biomol Spectrosc. 2021; 266:120460. DOI: 10.1016/j.saa.2021.120460. View

15.

Thiem D, Romer P, Gielisch M, Al-Nawas B, Schluter M, Plass B . Hyperspectral imaging and artificial intelligence to detect oral malignancy - part 1 - automated tissue classification of oral muscle, fat and mucosa using a light-weight 6-layer deep neural network. Head Face Med. 2021; 17(1):38. PMC: 8414848. DOI: 10.1186/s13005-021-00292-0. View