“Knowing about knowing” is called metacognition and it’s a big issue in education. Knowing how you learn and what your strong points and your working points are. In Hattie’s Visible Learning metacognitive strategies rank pretty high with an effect size of .69. But how do your measure metacognition? This the topic of a new review article in Frontiers by Fleming and Lau. The article is rather technical, but the distinction they make between bias and sensitivity is interesting.

Abstract of the research:

The ability to recognize one’s own successful cognitive processing, in e.g., perceptual or memory tasks, is often referred to as metacognition. How should we quantitatively measure such ability? Here we focus on a class of measures that assess the correspondence between trial-by-trial accuracy and one’s own confidence. In general, for healthy subjects endowed with metacognitive sensitivity, when one is confident, one is more likely to be correct. Thus, the degree of association between accuracy and confidence can be taken as a quantitative measure of metacognition. However, many studies use a statistical correlation coefficient (e.g., Pearson’s r) or its variant to assess this degree of association, and such measures are susceptible to undesirable influences from factors such as response biases. Here we review other measures based on signal detection theory and receiver operating characteristics (ROC) analysis that are “bias free,” and relate these quantities to the calibration and discrimination measures developed in the probability estimation literature. We go on to distinguish between the related concepts of metacognitive bias (a difference in subjective confidence despite basic task performance remaining constant), metacognitive sensitivity (how good one is at distinguishing between one’s own correct and incorrect judgments) and metacognitive efficiency (a subject’s level of metacognitive sensitivity given a certain level of task performance). Finally, we discuss how these three concepts pose interesting questions for the study of metacognition and conscious awareness.