The benefit of taking a blogging break is that you can wait a bit with bringing some news. While many people shared this meta-analysis, published in a top journal, there are some serious issues with it.

The highlights found by Ruiqi Deng and colleagues:

• ChatGPT enhances academic performance.
• ChatGPT boosts affective-motivational states.
• ChatGPT improves higher-order thinking propensities.
• ChatGPT reduces mental effort.
• ChatGPT does not influence self-efficacy.

However, the trouble with meta-analysis can be garbage in and garbage out, and that’s precisely what has happened here, as Ilka Tuomi explains on Linkedin. To quote Ilka

Of the 22 “systematic ChatGPT review studies” used for the meta-analysis, five were published in journals that are classified at level 0 in the Finnish & Norwegian journal classification systems.

This means “not at the basic academic level of quality.” At this level, the journals are typically predatory or “gray level” journals from publishers that, for varying publication fees, push out large quantities of articles without decent quality control. Most of the MDPI and Frontiers journals are now at this level.

Three of the analyzed articles were at level 2, the “leading level.” Elsevier’s Computers & Education, where the article was published is at level 3, the highest.

And he adds:

Some of the journals that are now classified at the basic level probably should be at the zero level.

Personally shocking was to find that one of the three highest-level journals, at level 2, with a meta-review of the “strengths, weaknesses, opportunities, and threats,” was the European Journal of Education. Until the last year, I was editorial board member for some 15+ years. This is a Wiley journal that changed its publication policy, with the result that the whole editorial board left the journal, ending its 50+ years of history. I have reviewed many articles for the journal, and I would not have recommended the published article (which also relied on 0-quality reviews in its “meta-review”). So, it’s not only MDPI and some obscure publishers, anymore.