A Comprehensive Assessment of Current Methods for Measuring Metacognition

Overview

Journal Nat Commun

Date 2025 Jan 15

PMID 39814749

Authors

Dobromir Rahnev

Affiliations

Soon will be listed here.

Abstract

One of the most important aspects of research on metacognition is the measurement of metacognitive ability. However, the properties of existing measures of metacognition have been mostly assumed rather than empirically established. Here I perform a comprehensive empirical assessment of 17 measures of metacognition. First, I develop a method of determining the validity and precision of a measure of metacognition and find that all 17 measures are valid and most show similar levels of precision. Second, I examine how measures of metacognition depend on task performance, response bias, and metacognitive bias, finding only weak dependences on response and metacognitive bias but many strong dependencies on task performance. Third, I find that all measures have very high split-half reliabilities, but most have poor test-retest reliabilities. This comprehensive assessment paints a complex picture: no measure of metacognition is perfect and different measures may be preferable in different experimental contexts.

Citing Articles

On the complexity of metacognitive judgments of memory: evidence from retrospective confidence, feeling of knowing, and older adults.

Meunier-Duperray L, Mazancieux A, Souchay C, Bastin C, Angel L, Moulin C Neurosci Conscious. 2025; 2025(1):niaf003.

PMID: 40008270 PMC: 11850298. DOI: 10.1093/nc/niaf003.

References

Shekhar M, Rahnev D . Sources of Metacognitive Inefficiency. Trends Cogn Sci. 2020; 25(1):12-23. PMC: 8610081. DOI: 10.1016/j.tics.2020.10.007. View

Shekhar M, Rahnev D . How do humans give confidence? A comprehensive comparison of process models of perceptual metacognition. J Exp Psychol Gen. 2023; 153(3):656-688. PMC: 10922729. DOI: 10.1037/xge0001524. View

Maniscalco B, Lau H . Manipulation of working memory contents selectively impairs metacognitive sensitivity in a concurrent visual discrimination task. Neurosci Conscious. 2018; 2015(1):niv002. PMC: 5989484. DOI: 10.1093/nc/niv002. View

Fleming S, Lau H . How to measure metacognition. Front Hum Neurosci. 2014; 8:443. PMC: 4097944. DOI: 10.3389/fnhum.2014.00443. View

Clark L, Watson D . Constructing validity: New developments in creating objective measuring instruments. Psychol Assess. 2019; 31(12):1412-1427. PMC: 6754793. DOI: 10.1037/pas0000626. View

McCurdy L, Maniscalco B, Metcalfe J, Liu K, de Lange F, Lau H . Anatomical coupling between distinct metacognitive systems for memory and visual perception. J Neurosci. 2013; 33(5):1897-906. PMC: 4696871. DOI: 10.1523/JNEUROSCI.1890-12.2013. View

Rouault M, Seow T, Gillan C, Fleming S . Psychiatric Symptom Dimensions Are Associated With Dissociable Shifts in Metacognition but Not Task Performance. Biol Psychiatry. 2018; 84(6):443-451. PMC: 6117452. DOI: 10.1016/j.biopsych.2017.12.017. View

Evans S, Azzopardi P . Evaluation of a 'bias-free' measure of awareness. Spat Vis. 2007; 20(1-2):61-77. DOI: 10.1163/156856807779369742. View

Mazancieux A, Fleming S, Souchay C, Moulin C . Is there a G factor for metacognition? Correlations in retrospective metacognitive sensitivity across tasks. J Exp Psychol Gen. 2020; 149(9):1788-1799. PMC: 7397761. DOI: 10.1037/xge0000746. View

10.

Kornell N, Son L, Terrace H . Transfer of metacognitive skills and hint seeking in monkeys. Psychol Sci. 2007; 18(1):64-71. DOI: 10.1111/j.1467-9280.2007.01850.x. View

11.

Luck S, Stewart A, Simmons A, Rhemtulla M . Standardized measurement error: A universal metric of data quality for averaged event-related potentials. Psychophysiology. 2021; 58(6):e13793. PMC: 8169536. DOI: 10.1111/psyp.13793. View

12.

Morales J, Lau H, Fleming S . Domain-General and Domain-Specific Patterns of Activity Supporting Metacognition in Human Prefrontal Cortex. J Neurosci. 2018; 38(14):3534-3546. PMC: 5895040. DOI: 10.1523/JNEUROSCI.2360-17.2018. View

13.

Rausch M, Hellmann S, Zehetleitner M . Measures of metacognitive efficiency across cognitive models of decision confidence. Psychol Methods. 2023; . DOI: 10.1037/met0000634. View

14.

Kelemen W, Frost P, Weaver 3rd C . Individual differences in metacognition: evidence against a general metacognitive ability. Mem Cognit. 2000; 28(1):92-107. DOI: 10.3758/bf03211579. View

15.

Kunimoto C, Miller J, Pashler H . Confidence and accuracy of near-threshold discrimination responses. Conscious Cogn. 2001; 10(3):294-340. DOI: 10.1006/ccog.2000.0494. View

16.

Konishi M, Berberian B, de Gardelle V, Sackur J . Multitasking costs on metacognition in a triple-task paradigm. Psychon Bull Rev. 2021; 28(6):2075-2084. DOI: 10.3758/s13423-021-01967-0. View

17.

Galvin S, Podd J, Drga V, Whitmore J . Type 2 tasks in the theory of signal detectability: discrimination between correct and incorrect decisions. Psychon Bull Rev. 2004; 10(4):843-76. DOI: 10.3758/bf03196546. View

18.

Rahnev D . Visual metacognition: Measures, models, and neural correlates. Am Psychol. 2022; 76(9):1445-1453. PMC: 9295341. DOI: 10.1037/amp0000937. View

19.

Mamassian P, de Gardelle V . Modeling perceptual confidence and the confidence forced-choice paradigm. Psychol Rev. 2021; 129(5):976-998. DOI: 10.1037/rev0000312. View

20.

Rahnev D, Fleming S . How experimental procedures influence estimates of metacognitive ability. Neurosci Conscious. 2019; 2019(1):niz009. PMC: 6556214. DOI: 10.1093/nc/niz009. View