New paper “A reflection on cognitive reflection” published in Judgment and Decision Making
Our article “A reflection on cognitive reflection” has just been published in Judgment and Decision Making. So, what is it about?
Cognitive reflection test (CRT) and belief-bias syllogisms (BBS) are supposed to measure cognitive reflection, i.e. the ability and/or disposition to resist reporting a first response that comes to a mind and, instead, to think more carefully before coming up with a correct response. These two types of tasks are popular for several reasons. First is their defining characteristic – they are designed to lure a person into giving an intuitive, but incorrect response. Second, they fit nicely into a dual-process narrative – lures produce intuitive but incorrect response (System 1 response) that needs to be recognized and overridden before calculating a correct response (System 2 tasks). Finally, they are predictive of a wide range of other tasks from the decision-making domain as well as of different real life outcomes.
In this article, we question the relevance of lures for the predictiveness of cognitive reflection measures. We find that cognitive reflection is factorially indistinguishable from a numeracy test consisting of tasks without lures. i.e these two measures assess same constructs. Therefore, lures don’t give some additional predictive power to CRT over other math tasks without lures and cannot, therefore, be responsible for its predictiveness.
Similarly, the correlations between belief bias tasks and different tasks from decision-making domain are substantially diminished after partialling out the effects of numeracy. This also suggest that the lures do not give additional predictive power to belief-bias syllogisms.
We conclude that, for whatever reasons these cognitive reflection measures are able to predict different outcomes, it has little to do with lures, and more with other constructs simultaneously assessed by cognitive reflection measures AND other measures without lures (such as dispositions of reflectivity and actively open-minded thinking). Therefore, cognitive reflection test does not appear to be a particularly unique measure of cognitive reflection and it does not seem to measure cognitive reflection defined as ability and/or disposition to resist reporting initial response.
Check out the full article here.
