BarrettDienesSeth_PSYCHMETH2013.pdf (379.19 kB)
Measures of metacognition on signal-detection theoretic models
journal contribution
posted on 2023-06-08, 16:46 authored by Adam BarrettAdam Barrett, Zoltan DienesZoltan Dienes, Anil SethAnil SethAnalysing metacognition, specifically knowledge of accuracy of internal perceptual, memorial or other knowledge states, is vital for many strands of psychology, including determining the accuracy of feelings of knowing, and discriminating conscious from unconscious cognition. Quantifying metacognitive sensitivity is however more challenging than quantifying basic stimulus sensitivity. Under popular signal detection theory (SDT) models for stimulus classification tasks, approaches based on type II receiver-operator characteristic (ROC) curves or type II d-prime risk confounding metacognition with response biases in either the type I (classification) or type II (metacognitive) tasks. A new approach introduces meta-d': the type I d-prime that would have led to the observed type II data had the subject used all the type I information. Here we (i) further establish the inconsistency of the type II d-prime and ROC approaches with new explicit analyses of the standard SDT model, and (ii) analyse, for the first time, the behaviour of meta-d' under non-trivial scenarios, such as when metacognitive judgments utilize enhanced or degraded versions of the type I evidence. Analytically, meta-d' values typically reflect the underlying model well, and are stable under changes in decision criteria; however, in relatively extreme cases meta-d' can become unstable. We explore bias and variance of in-sample measurements of meta-d' and supply MATLAB code for estimation in general cases. Our results support meta-d' as a useful measure of metacognition, and provide rigorous methodology for its application. Our recommendations are useful for any researchers interested in assessing metacognitive accuracy.
History
Publication status
- Published
File Version
- Accepted version
Journal
Psychological Methods.ISSN
1082-989XPublisher
American Psychological AssociationExternal DOI
Issue
4Volume
18Page range
535-552Department affiliated with
- Informatics Publications
Notes
This article may not exactly replicate the final version published in the APA journal. It is not the copy of record.Full text available
- Yes
Peer reviewed?
- Yes
Legacy Posted Date
2014-02-26First Open Access (FOA) Date
2014-02-26First Compliant Deposit (FCD) Date
2014-02-26Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC