Scribbles #2

Here is something you do not see everyday: a meta-analysis on GenAI use and critical thinking in education in Educational Psychology Review reveals that contrary to what the newspapers say, GenAI might be promoting critical thinking. According to Sun et al.:

The results indicate that GenAI-supported learning was, on average, associated with higher CT performance (g = 0.544, 95% CI [0.314, 0.775], 95% PI [-0.588, 1.677]), with significant heterogeneity observed across studies (Q = 280.994, p < 0.001).

The whole meta-analysis has been done with great care (PRISMA, small-sample correction, publication-bias check and the works, for those who care). This carefulness leaves the authors unable to explain what is driving the positive effect, but at the same time makes it difficult to just argue away the finding. Still, performance varies so much that the true effect in any classroom could be null or negative.

Then, in Current Psychology, Wang et al. took data from college students to look at critical thinking and GenAI use. They argue that GenAI is a double-edged sword, which can help students become disposed to critical thinking by improving their confidence, but also can cause them to become lean onto it too much. Most importantly, they state the former effect is bigger than the latter:

The findings indicate that GAI usage positively affects CT disposition through perceived self-efficacy, likely because it enhances students’ confidence in their abilities. However, it undermines CT disposition through perceived dependence, as students may become overly reliant on GAI for problem-solving. When comparing the two mediating pathways, it was found that the positive influence of GAI on CT disposition, as measured by increased confidence and problem-solving skills, outweighed the negative effects of perceived dependence.

Since disposition has been found especially challenging when teaching critical thinking, this finding matters if it generalises. Higher education's failure at teaching critical thinking is arguably driven by dispositions being underdeveloped. If GenAI can help there, it is not unreasonable to see it move the needle on critical thinking training. Something to watch, but some notes:

• Both self-efficacy and dependency are self-report measures. You would want to see similar results using a different type of measure.
• Under time pressure, self-efficacy did not boost critical thinking disposition so much, but the negative link between dependency and critical thinking disposition remained in place.
• The study is more correlational than the quote above suggests. It is quite possible that students with a low critical thinking disposition are more likely to grab GenAI and then label their behaviour as dependence, rather than the dependence causing their lower disposition. Similar caveats hold for the positive effects on critical thinking.
• There are some stats issues, too, which I do not want to discuss here, but which make the "this number is bigger than that number" reasoning less straightforward than it may sound here.

Umes Shrestha wrote an excellent opinion piece on constructivism in education. His point is that we have turned the constructivist description of learning into a particular prescription for teaching, but that the underlying reasoning is fallacious. That learners construct their world by fitting new information in with existing structures does not say anything about how the teaching should be done.

This nicely fits with what I guess I would call an enactivist view on teaching and learning, in which learners are navigating an environment to build a world model, but not all landscape features are equal, with teacher playing a special, salient role in the construction of world models.

While I have not written about conspiracy thinkers for a while, I do still monitor the literature and was pleased to see the one and only Hugo Mercier has a working paper (not yet published, unreviewed) on the topic: People have an accurate folk theory of conspiracy believers. Here, Mercier says his research reveals that people can generally guess to which conspiracy theories specific people will subscribe (those who subscribe to outlandish theories, will probably also subscribe to less outlandish ones) and can also provide a description of the underlying personality type. If you want to, you can read it more carefully than I did.

I can imagine such research sounds incredibly niche, but there is an interesting question out there about the way in which societies deal with people whose beliefs are outside of the norm. We calibrate our understanding of the world socially and if we would just average all the stuff we are hearing, the outliers would be running the show. This is, however, not the case. From a very young age onwards, humans track how reliable and cogent others are, so that testimony learning can happen rationally. Mercier's suggestions fit in with this – we somehow need to make sense of people coming in with conspiracy theories, especially if there are naturalistic roots to such ideation, as Jan-Willem van Prooijen and Mark van Vugt once proposed.