Category Archives: Technology

by | September 10, 2018 · 8:33 pm

New meta-analysis begs: Don’t throw away your printed books in education

I just found a new meta-analysis soon to be published in which Pablo Delgado, Cristina Vargas, Rakefet Ackerman & Ladislao Salmerón examine the effects of reading media on reading comprehension. Well, the title gives away the conclusion, I guess.

But this is the longer version:

The results of the two meta-analyses in the present study yield a clear picture of screen inferiority, with lower reading comprehension outcomes for digital texts compared to printed texts, which corroborates and extends previous research (Kong et al., 2018; Singer & Alexander, 2017b; Wang et al. 2007). These results were consistent across methodologies and theoretical frameworks.

And while the effects are relatively low, the researchers do warn:

Although the effect sizes found for media (-.21) are small according to Cohen’s guidelines (1988), it is important to interpret this effect size in the context of reading comprehension studies. During elementary school, it is estimated that yearly growth in reading comprehension is .32 (ranging from .55 in grade 1, to .08 in grade 6) (Luyten, Merrel & Tymms, 2017). Intervention studies on reading comprehension yield a mean effect of .45 (Scammacca et al., 2015). Thus, the effects of media are relevant in the educational context because they represent approximately 2/3 of the yearly growth in comprehension in elementary school, and 1/2 of the effect of remedial interventions.

The analysis also has some clear practical consequences:

A relevant moderator found for the screen inferiority effect was time frame. This finding sheds new light on the mixed results in the existing literature. Consistent with the findings by Ackerman and Lauterman (2012) with lengthy texts, mentioned above, Sidi et al. (2017) found that even when performing tasks involving reading only brief texts and no scrolling (solving challenging logic problems presented in an average of 77 words), digital-based environments harm performance under time pressure conditions, but not under a loose time frame. In addition, they found a similar screen inferiority when solving problems under time pressure and under free time allocation, but framing the task as preliminary rather than central. Thus, the harmful effect of limited time on digital-based work is not limited to reading lengthy texts. Moreover, consistently across studies, Ackerman and colleagues found that people suffer from greater overconfidence in digital-based reading than in paper-based reading under these conditions that warrant shallow processing.

…

Our findings call to extend existing theories about self-regulated learning (see Boekaerts, 2017, for a review). Effects of time frames on self-regulated learning have been discussed from various theoretical approaches. First, a metacognitive explanation suggests that time pressure encourages compromise in reaching learning objectives (Thiede & Dunlosky, 1999). Second, time pressure has been associated with cognitive load. Some studies found that time pressure increased cognitive load and harmed performance (Barrouillet, Bernardin, Portrat, Vergauwe, & Camos, 2007). However, others suggested that it can generate a germane (“good”) cognitive load by increasing task engagement (Gerjets & Scheiter, 2003). In these theoretical discussions, the potential effect of the medium in which the study is conducted has been overlooked. We see the robust finding in the present meta-analyses about the interaction between the time frame and the medium as a call to theorists to integrate the processing style adapted by learners in specific study environments into their theories

What I really appreciate is that the researchers also checked for publication bias, and good news, the different indicators that were used, suggested no risk of publication bias.

There is only a small bit of irony… I read the study online and you read this online too.

Abstract of the meta-analysis:

With the increasing dominance of digital reading over paper reading, gaining understanding of the effects of the medium on reading comprehension has become critical. However, results from research comparing learning outcomes across printed and digital media are mixed, making conclusions difficult to reach. In the current metaanalysis, we examined research in recent years (2000-2017), comparing the reading of comparable texts on paper and on digital devices. We included studies with betweenparticipant (n = 38) and within-participant designs (n = 16) involving 171,055 participants. Both designs yielded the same advantage of paper over digital reading (Hedge’s g = -.21; dc = -.21). Analyses revealed three significant moderators: (1) time frame: the paper-based reading advantage increased in time-constrained reading compared to self-paced reading; (2) text genre: the paper-based reading advantage was consistent across studies using informational texts, or a mix of informational and narrative texts, but not on those using only narrative texts; (3) publication year: the advantage of paper-based reading increased over the years. Theoretical and educational implications are discussed.

Leave a comment

Filed under Education, Research, Review, Technology

by | August 14, 2018 · 4:55 pm

Oh, I missed the new Hype Cycle for education. Well…

It’s a yearly tradition for Gartner to publish a string of hype cycles, including one for education in July. And I admit: I didn’t pay attention to it.

So, there is a new one, but besides the many issues one can have with the hype cycle by this company, I do think this time it’s pretty bland as if everybody with a bit of knowledge about EdTech could have written it.

  • On the Rise
    • AV Over IP in Education
    • Social CRM: Education
    • Li-Fi
    • Emotion AI
    • Virtual Reality/Augmented Reality Applications in Education
  • At the Peak
    • Blockchain in Education
    • Artificial Intelligence Education Applications
    • Design Thinking
    • Exostructure Strategy
    • Classroom 3D Printing
    • Digital Assessment
    • SaaS SIS
  • Sliding Into the Trough
    • Education Analytics
    • Competency-Based Education Platforms
    • Bluetooth Beacons
    • Semantic Knowledge Graphing
    • Citizen Developers
    • Digital Credentials
    • Alumni CRM
    • Master Data Management
    • Adaptive Learning Platforms
  • Climbing the Slope
    • Student Retention CRM
    • IDaaS
    • Enterprise Video Content Management
  • Entering the Plateau
    • Integration Brokerage

2 Comments

Filed under Education, Technology, Trends

by | August 13, 2018 · 7:31 pm

Do read this great little tweet tirade on #edtech predicting the future and cognitive science by Benjamin Riley (Deans for Impact)

When I read the first tweet of this thread by Benjamin Riley I had the feeling we were up to something good. And Benjamin didn’t disappoint. I won’t make it into a habit of posting something like this on this blog, but I do wanted to share this here as I know that many of my readers would otherwise miss this:

And thus the conclusion?

2 Comments

Filed under Education, Technology

by | August 7, 2018 · 9:16 am

What works and doesn’t work with instructional video, a new short overview

There is a special issue of Computers in Human Behavior on learning from video and in their Editorial Fiorella and Mayer give an overview of effective and ineffective methods that are being trialed in the special issue:

What are the effective methods?

…two techniques that appear to improve learning outcomes with instructional video are segmenting—breaking the video into parts and allowing students to control the pace of the presentation—and mixed perspective—filming from both a first-person perspective and third-person.

And what isn’t worth the effort?

…some features that do not appear to be associated with improved learning outcomes with instructional video are matching the gender of the instructor to the gender of the learner, having the instructor’s face on the screen, inserting pauses throughout the video, and adding practice without feedback.

Abstract of the editorial:

In this commentary, we examine the papers in a special issue on “Developments and Trends in Learning with Instructional Video”. In particular, we focus on basic findings concerning which instructional features improve learning with instructional video (i.e., breaking the lesson into segments paced by the learner; recording from both first- and third-person perspectives) and which features or learner attributes do not (i.e., matching the instructor’s gender to the learner’s gender; having the instructor’s face on the screen; adding practice without feedback; inserting pauses throughout the video; and spatial ability). In addition, we offer recommendations for future work on designing effective video lessons.

Leave a comment

Filed under Education, Research, Review, Technology

by | August 1, 2018 · 9:15 pm

Again: the bad effect of checking your phone in class

There has been quite some debate because of the French decision to ban mobile phones from schools from this school year on. While I’m also a bit critical – I think we should rather teach children how to deal with phones and focus – I do understand where this thinking stems from, from studies like this: the negative correlation between mobile phone use and grades.

From the press release:

Students perform less well in end-of-term exams if they are allowed access to an electronic device, such as a phone or tablet, for non-academic purposes in lectures, a new study in Educational Psychology finds.

Students who don’t use such devices themselves but attend lectures where their use is permitted also do worse, suggesting that phone/tablet use damages the group learning environment.

Researchers from Rutgers University in the US performed an in-class experiment to test whether dividing attention between electronic devices and the lecturer during the class affected students’ performance in within-lecture tests and an end-of-term exam.

118 cognitive psychology students at Rutgers University participated in the experiment during one term of their course. Laptops, phones and tablets were banned in half of the lectures and permitted in the other half. When devices were allowed, students were asked to record whether they had used them for non-academic purposes during the lecture.

The study found that having a device didn’t lower students’ scores in comprehension tests within lectures, but it did lower scores in the end-of-term exam by at least 5%, or half a grade. This finding shows for the first time that the main effect of divided attention in the classroom is on long-term retention, with fewer targets of a study task later remembered.

In addition, when the use of electronic devices was allowed in class, performance was also poorer for students who did not use devices as well as for those who did.

The study’s lead author, Professor Arnold Glass, added: “These findings should alert the many dedicated students and instructors that dividing attention is having an insidious effect that is impairing their exam performance and final grade.

“To help manage the use of devices in the classroom, teachers should explain to students the damaging effect of distractions on retention – not only for themselves, but for the whole class.”

Abstract of the study:

The intrusion of internet-enabled electronic devices (laptop, tablet, and cell phone) has transformed the modern college lecture into a divided attention task. This study measured the effect of using an electronic device for a non-academic purpose during class on subsequent exam performance. In a two-section college course, electronic devices were permitted in half the lectures, so the effect of the devices was assessed in a within-student, within-item counterbalanced experimental design. Dividing attention between an electronic device and the classroom lecture did not reduce comprehension of the lecture, as measured by within-class quiz questions. Instead, divided attention reduced long-term retention of the classroom lecture, which impaired subsequent unit exam and final exam performance. Students self-reported whether they had used an electronic device in each class. Exam performance was significantly worse than the no-device control condition both for students who did and did not use electronic devices during that class.

Leave a comment

Filed under Education, Research, Social Media, Technology

by | June 28, 2018 · 9:19 am

Don’t multitask during class (again)

Yesterday I discovered this small randomized controlled trial via a tweet by Daniel Willingham. The study confirms what we’ve seen in different other studies: multitasking in class is bad for learning.

The results showed that when students were given the opportunity of non-lecture-related multi- tasking using mobile phones writing/sending SMSs and looking at Facebook profiles/reading news feed/looking at shared multimedia/reading wall messages during the lecture, their grade performance was hindered compared to traditional pen and paper note-taking.

Although I have to correct myself, one should better say: not multitasking is better for learning

Although there was a significant difference between participants on the traditional pen and paper note-taking lectures (no technology multitasking) and social media and SMS multitasking groups in terms of academic achievement, students in multitasking with social media and SMS groups also improved their pretest results.

As said, the study isn’t that big with 122 participants spread over 3 groups, but adds to the existing body of knowledge.

Abstract of the study

The purpose of this study is to investigate whether off-task multitasking activities with mobile technologies, specifically social networking sites and short messaging services, used during real-time lectures have an effect on grade performance in higher education students. Two experimental groups and one control group were used in this research. While participants in experimental groups 1 and 2 were allowed to navigate Facebook and to exchange short messaging service messages via mobile phones during real time in class lecturing, the control group participants were allowed to take notes using only pen and paper in the same lecturing conditions during three consecutive experimental sessions. The results showed that when students were given the opportunity of non-lecture-related multitasking using mobile phones writing/sending short messaging services and looking at Facebook profiles/reading news feed/looking at shared multimedia/reading wall messages during the lecture, their grade performance was hindered compared to traditional pen and paper note-taking. Engaging in social media use while trying to follow instruction may reduce learners’ capacity for cognitive processing causing poor academic performance.

1 Comment

Filed under Education, Research, Technology

by | March 20, 2018 · 4:04 pm

New study in Nature: playing a lof of violent games doesn’t make players more violent

It’s a very popular idea, dating back to the theories and studies by Bandura: seeing violence teaches people to act violently. And more recently there was an American president linking computer games to school shootings. This new study shows that this maybe unwarranted.

In this study by Kühn et al published in Nature the researchers did a randomized controlled trial with 3 groups

  • 1 group who played Grand Theft Auto intensively during 2 months
  • 1 group who played The Sims 3 intensively during 2 months
  • 1 group who didn’t play games at all.

And what did the researchers find?

Within the scope of the present study we tested the potential effects of playing the violent video game GTA V for 2 months against an active control group that played the non-violent, rather pro-social life simulation game The Sims 3 and a passive control group. Participants were tested before and after the long-term intervention and at a follow-up appointment 2 months later. Although we used a comprehensive test battery consisting of questionnaires and computerised behavioural tests assessing aggression, impulsivity-related constructs, mood, anxiety, empathy, interpersonal competencies and executive control functions, we did not find relevant negative effects in response to violent video game playing. In fact, only three tests of the 208 statistical tests performed showed a significant interaction pattern that would be in line with this hypothesis. Since at least ten significant effects would be expected purely by chance, we conclude that there were no detrimental effects of violent video gameplay.

Will this study end all discussions? No, I’m sure this won’t be the case even if this study is very relevant. It’s worth noticing that the average age of the participants was 28, on which I would suggest that a replication with younger participants would be a very good idea.

Abstract of the study:

It is a widespread concern that violent video games promote aggression, reduce pro-social behaviour, increase impulsivity and interfere with cognition as well as mood in its players. Previous experimental studies have focussed on short-term effects of violent video gameplay on aggression, yet there are reasons to believe that these effects are mostly the result of priming. In contrast, the present study is the first to investigate the effects of long-term violent video gameplay using a large battery of tests spanning questionnaires, behavioural measures of aggression, sexist attitudes, empathy and interpersonal competencies, impulsivity-related constructs (such as sensation seeking, boredom proneness, risk taking, delay discounting), mental health (depressivity, anxiety) as well as executive control functions, before and after 2 months of gameplay. Our participants played the violent video game Grand Theft Auto V, the non-violent video game The Sims 3 or no game at all for 2 months on a daily basis. No significant changes were observed, neither when comparing the group playing a violent video game to a group playing a non-violent game, nor to a passive control group. Also, no effects were observed between baseline and posttest directly after the intervention, nor between baseline and a follow-up assessment 2 months after the intervention period had ended. The present results thus provide strong evidence against the frequently debated negative effects of playing violent video games in adults and will therefore help to communicate a more realistic scientific perspective on the effects of violent video gaming.

2 Comments

Filed under Media literacy, Psychology, Research, Technology

by | January 29, 2018 · 2:05 pm

What are the effects of giving each child a laptop?

This study I found via Gabriel Bouchaud examines the possible effects of the One Laptop per Child in Peru. Other than my normal procedure I want to start with the abstract as it summarizes a lot already:

This paper presents results from a large-scale randomized evalua- tion of the One Laptop per Child program, using data collected after 15 months of implementation in 318 primary schools in rural Peru. The program increased the ratio of computers per student from 0.12 to 1.18 in treatment schools. This expansion in access translated into substantial increases in use of computers both at school and at home. No evidence is found of effects on test scores in math and language. There is some evidence, though inconclusive, about positive effects on general cognitive skills.

This doesn’t sound that bad. The pupils use computers more – what a suprise if they didn’t own a computer before – but does it have an effect on education?

Well? The computers were packed with over 200 books, but… the pupils didn’t start to read more. They didn’t spent more time on education. And… they didn’t really seem to do better in class.

Or as the researchers summarize:

In general, we do not find conclusive evidence indicating clear changes in behavior in these dimensions. Regarding study time at home, we document some positive effects on whether the student studied at home the prior day. Nonetheless, results indicate small effects on whether the student studied one or more hours daily the prior week. In terms of reading, results suggest some negative effects on whether the student read a book the prior day but small effects on whether the student read a book the prior week.20 Overall, we find no statistically significant effects on the specific outcomes analyzed or on the learn- ing behavior summary measure.

…

…did increased computer access affect academic and cognitive skills? Table 9 shows that there are no statistically signi – cant effects on the academic achievement summary measure when focusing on all students and also when restricting to those in the interviewed sample (columns 1 and 2, respectively). Small standard errors allow ruling out modest effects. Also, there are no statistically signi cant effects on either math or language achievement for both samples.
…

Our results suggest that computers by them- selves, at least as initially delivered by the OLPC program, do not increase achieve- ment in curricular areas.

I’m a bit in a bind here. I do not want to state that it is a bad idea to give computers to children in need. But this study does show that just giving kids a computer – or putting them in a wall – doesn’t do much.

 

1 Comment

Filed under Education, Research, Technology

by | January 8, 2018 · 4:14 pm

New study on smartphone use and academic performance (spoiler: it’s bad)

This morning big news in our Belgian media about a new study by Stijn Baert and his colleagues in which they checked the impact of smartphone usage on the academic performance of the students:

In this study, we contributed to recent literature concerning the association between smartphone use and educational performance by providing the first causal estimates of the effect of the former on the latter. To this end, we analysed unique data on 696 first-year university students in Belgium. We found that a one-standard-deviation increase in their overall smartphone use yields a decrease in their average exam score of about one point (out of 20). This negative relationship is robust to the use of alternative indicators of smartphone use and academic performance. As our results add to the literature evidence for heavy smartphone use not only being associated with lower exam marks but also causing lower marks, we believe that policy-makers should at least invest in information and awareness campaigns to highlight this trade-off.

I have to admit that I do think that while the researchers have taken a lot into account, there always still can be something else maybe causing this differences. The researchers have attempted to bypass this:

This study is the first to attempt to measure the causal impact of (overall) smartphone use on educational performance. To this end, we exploit data from 696 first-year students at two Belgian universities, who were surveyed in December 2016 using multiple scales on smartphone use as well as predictors of this smartphone use and a battery of questions concerning (potential) other drivers of success at university. This information is merged with the students’ scores on their first exams, taken in January 2017. We analyse the merged data by means of instrumental variable estimation techniques. More concretely, to be able to correctly identify the influence of smartphone use on academic achievement, in a first stage, the respondents’ smartphone use is predicted by diverging sets of variables that are highly significantly associated with smartphone use, but not directly associated with educational performance. In a second stage, the exam scores are regressed on this exogenous prediction of smartphone use and the largest set of control variables used in the literature to date.

In the interview this morning on the radio, the researchers didn’t plea for a total ban of smartphones, but still think it can be a very important element for students to take into consideration.

Abstract of the study:

After a decade of correlational research, this study is the first to measure the causal impact of (general) smartphone use on educational performance. To this end, we merge survey data on general smartphone use, exogenous predictors of this use, and other drivers of academic success with the exam scores of first-year students at two Belgian universities. The resulting data are analysed with instrumental variable estimation techniques. A one-standard-deviation increase in daily smartphone use yields a decrease in average exam scores of about one point (out of 20). When relying on ordinary least squares estimations, the magnitude of this effect is substantially underestimated.

5 Comments

Filed under Education, Research, Technology

by | January 2, 2018 · 7:52 pm

Does VR help to learn? Well, this app doesn’t seem to.

I have to admit: despite the fact that I get sick every time I’m using virtual reality goggles, I really think VR and augmented reality (AR) is just impressive. But… does it help learning. It is too early to tell, but this study by Makransky et al that I found via a tweet by Paul Kirschner is pretty clear: in this study VR doesn’t improve learning . The study is extra interesting as it looks at some important principles for learning such as the redundancy principle (Mayer was involved in the study), and while the students did get more motivated, the learning was not better (even worse). Do note that the the amount of participants was pretty low: 52 (22 males and 30 females) students from a large European university.

Also interesting is to know what application the researchers were using:

The virtual simulation used in this experiment was on the topic of mammalian transient protein expression and was developed by the simulation development company, Labster. It was designed to facilitate learning within the field of biology at a university level by allowing the user to virtually work through the procedures in a lab by using and interacting with the relevant lab equipment and by teaching the essential content through an inquiry-based learning approach.

In short:

 

  • The consequences of adding immersive virtual reality to a simulation was examined.
  • The impact of the level of immersion on the redundancy principle was investigated.
  • EEG was used to obtain a direct measure of cognitive processing during learning.
  • Students reported higher presence but learned less in the immersive VR condition.
  • Students also had higher cognitive load based on EEG in the immersive VR condition.

Abstract of the study:

Virtual reality (VR) is predicted to create a paradigm shift in education and training, but there is little empirical evidence of its educational value. The main objectives of this study were to determine the consequences of adding immersive VR to virtual learning simulations, and to investigate whether the principles of multimedia learning generalize to immersive VR. Furthermore, electroencephalogram (EEG) was used to obtain a direct measure of cognitive processing during learning. A sample of 52 university students participated in a 2 × 2 experimental cross-panel design wherein students learned from a science simulation via a desktop display (PC) or a head-mounted display (VR); and the simulations contained on-screen text or on-screen text with narration. Across both text versions, students reported being more present in the VR condition (d = 1.30); but they learned less (d = 0.80), and had significantly higher cognitive load based on the EEG measure (d = 0.59). In spite of its motivating properties (as reflected in presence ratings), learning science in VR may overload and distract the learner (as reflected in EEG measures of cognitive load), resulting in less opportunity to build learning outcomes (as reflected in poorer learning outcome test performance).

 

2 Comments

Filed under Education, Research, Technology