Category Archives: Review

How do kids and parents worry about privacy with internet-connected toys

Your daughters’ Barbie can be hacked to spy on your children. So, got you worried? Researchers have now conducted a new study that explores the attitudes and concerns of both parents and children who play with internet-connected toys. Through a series of in-depth interviews and observations, the researchers found that kids didn’t know their toys were recording their conversations, and parents generally worried about their children’s privacy when they played with the toys.

From the press release:

University of Washington researchers have conducted a new study that explores the attitudes and concerns of both parents and children who play with internet-connected toys. Through a series of in-depth interviews and observations, the researchers found that kids didn’t know their toys were recording their conversations, and parents generally worried about their children’s privacy when they played with the toys.

“These toys that can record and transmit are coming into a place that’s historically legally very well-protected ? the home,” said co-lead author Emily McReynolds, associate director of the UW’s Tech Policy Lab. “People have different perspectives about their own privacy, but it’s crystalized when you give a toy to a child.”

The researchers presented their paper May 10 at the CHI 2017 Conference on Human Factors in Computing Systems.

Though internet-connected toys have taken off commercially, their growth in the market has not been without security breaches and public scrutiny. VTech, a company that produces tablets for children, was storing personal data of more than 200,000 children when its database was hacked in 2015. Earlier this year, Germany banned the Cayla toy over fears that personal data could be stolen.

It’s within this landscape that the UW team sought to understand the privacy concerns and expectations kids and parents have for these types of toys.

The researchers conducted interviews with nine parent-child pairs, asking each of them questions ? ranging from whether a child liked the toy and would tell it a secret to whether a parent would buy the toy or share what their child said to it on social media.

They also observed the children, all aged 6 to 10, playing with Hello Barbie and CogniToys Dino. These toys were chosen for the study because they are among the industry leaders for their stated privacy measures. Hello Barbie, for example, has an extensive permissions process for parents when setting up the toy, and it has been complimented for its strong encryption practices.

The resulting paper highlights a wide selection of comments from kids and parents, then makes recommendations for toy designers and policymakers.

Most of the children participating in the study did not know the toys were recording their conversations. Additionally, the toys’ lifelike exteriors probably fueled the perception that they are trustworthy, the researchers said, whereas kids might not have the tendency to share secrets and personal information when communicating with similar tools not intended as toys, such as Siri and Alexa.

“The toys are a social agent where you might feel compelled to disclose things that you wouldn’t otherwise to a computer or cell phone. A toy has that social exterior which might fool you into being less secure on what you tell it,” said co-lead author Maya Cakmak, an assistant professor at the Allen School. “We have this concern for adults, and with children, they’re even more vulnerable.”

Some kids were troubled by the idea of their conversations being recorded. When one parent explained how the child’s conversation with the doll could end up being shared widely on the computer, the child responded: “That’s pretty scary.”

At minimum, toy designers should create a way for the devices to notify children when they are recording, the researchers said. Designers could consider recording notifications that are more humanlike, such as having Hello Barbie say, “I’ll remember everything you say to me” instead of a red recording light that might not make sense to a child in that context.

The study found that most parents were concerned about their child’s privacy when playing with the toys. They universally wanted parental controls such as the ability to disconnect Barbie from the internet or control the types of questions to which the toys will respond. The researchers recommend toy designers delete recordings after a week’s time, or give parents the ability to delete conversations permanently.

A recent UW study demonstrated that video recordings that are filtered to preserve privacy can still allow a tele-operated robot to perform useful tasks, such as organize objects on a table. This study also revealed that people are much less concerned about privacy ? even for sensitive items that could reveal financial or medical information ? when such filters are in place. Speech recordings on connected toys could similarly be filtered to remove identity information and encode the content of speech in less human-interpretable formats to preserve privacy, while still allowing the toy to respond intelligibly.

The researchers hope this initial look into the privacy concerns of parents and kids will continue to inform both privacy laws and toy designers, given that such devices will only continue to fill the market and home.

“It’s inevitable that kids’ toys, as with everything else in society, will have computers in them, so it’s important to design them with security measures in mind,” said co-lead author Franziska Roesner, a UW assistant professor at the Allen School. “I hope the security research community continues to study these specific user groups, like children, that we don’t necessarily study in-depth.”

Abstract of the study:

Hello Barbie, CogniToys Dino, and Amazon Echo are part of a new wave of connected toys and gadgets for the home that listen. Unlike the smartphone, these devices are always on, blending into the background until needed. We conducted interviews with parent-child pairs in which they interacted with Hello Barbie and CogniToys Dino, shedding light on children’s expectations of the toys’ “intelligence” and parents’ privacy concerns and expectations for parental controls. We find that children were often unaware that others might be able to hear what was said to the toy, and that some parents draw connections between the toys and similar tools not intended as toys (e.g., Siri, Alexa) with which their children already interact. Our findings illuminate people’s mental models and experiences with these emerging technologies and will help inform the future designs of interactive, connected toys and gadgets. We conclude with recommendations for designers and policy makers.

1 Comment

Filed under At home, Media literacy, Research, Review, Technology

What does the research say about arts education?

I picked a second study from Best Evidence in Brief but this is too interesting not to share, although e.g. an earlier OECD-report on the same theme was a bit more nuanced on the last element mentioned.

Child Trends has released a new research brief that identifies “five ways the arts are good for kids.” The author, David Murphey, presents existing research on the topic from several sources such as the University of Pennsylvania, the National Institutes of Health, and the National Endowment for the Arts, as well as several published articles. The conclusions are as follows:
  • Arts participation is associated with numerous positive academic and personal outcomes. According to the brief, these outcomes include higher grades and test scores, enrollment in post-secondary education, attainment of a bachelor’s degree, and higher levels of literacy and civic engagement.
  • The benefits of arts participation may be greatest for children who are economically disadvantaged. For example, the research shows that young people from poor communities tend to benefit from having one or more projects that strengthen their sense of self and connect them with peers who share their interests.
  • Arts organizations can positively influence children’s neighborhoods. According to the brief, there is some evidence that the presence of arts organizations (including performance facilities, galleries, and artists’ workspaces) helps reduce a neighborhood’s concentrated poverty and attract other creative and high-tech enterprises.
  • Children’s arts participation varies by age, gender, and educational status. For example, the research shows that students are more likely to participate in school arts activities if their parents have attained higher degrees, and if they plan to attend a 4-year college themselves.
  • Music, in particular, may give children a brain boost. According to the brief, young people who have had music training demonstrate higher cognitive skills across disciplines

1 Comment

Filed under Education, Research, Review

Best Evidence in Brief: a systematic review on mindfulness in school

There is a new Best Evidence in Brief and this time it includes a review study that will spur some attention and/or discussion. I’ve noticed that a lot of people will call mindfulness a fad, others will argue that there is proven effect. So…

This Campbell systematic review examines the effectiveness of mindfulness-based interventions (MBIs) implemented in school settings on cognition, behavior, socioemotional outcomes, and academic achievement. MBIs are interventions that use a mindfulness component, broadly defined as “paying attention in a particular way: on purpose, in the present moment, non-judgmentally,” and is often combined with yoga, cognitive-behavioral strategies, or relaxation-skills training.
A total of 61 studies are included in the review, but only the 35 randomized or quasi-experimental studies are used in the meta-analysis, with a total of 6,207 student participants. Most of the studies were carried out in schools in the U.S. (74%), with some in Asia (5%), Europe (16%), and Canada (5%). The interventions ranged in duration (4-28 weeks), number of sessions (6-125 sessions), and frequency of meetings (once every two weeks to five times a week).
The findings show that MBIs in schools had a small positive effect on cognitive outcomes and socioemotional outcomes, but did not improve behavior or academic achievement. There was little heterogeneity for all outcomes, apart from behavioral outcomes, suggesting that the interventions produced similar results across studies on cognitive, socioemotional, and academic outcomes, despite the interventions being quite diverse. Overall, Brandy Maynard and colleagues found a lack of support at post-test to indicate that the positive effects on cognitive and socioemotional outcomes then translate into positive outcomes on behavior and academic achievement.
Will this close the debate? For sure not, I’m afraid. Still, a good overview what to expect when you try mindfulness at school.

6 Comments

Filed under Education, Research, Review

Jordan Peterson puts it quite bluntly: The Theory of Multiple Intelligences Is Rubbish!

Well, maybe you’ll be surprised, but than you haven’t read this (or our book). H/t Carl Hendrick

1 Comment

Filed under Education, Myths, Psychology, Review

A very interesting rant on a study on the brain and grit & growth mindset

I like the work of Stuart Ritchie very much. This little twitter rant he shared with the world yesterday is actually very enlightening for anyone reading research:

1 Comment

Filed under Education, Research, Review

Best Evidence in Brief: What are the effects of a four-day school week?

There is a new Best Evidence in Brief and this time I already discussed several of the studies mentioned in this great news letter on this blog. But this study was new to me, and quite interesting:

A Brookings report by Paul T. Hill and Georgia Heyward examines the four-day school week in rural Idaho. According to the report, four-day weeks have been implemented in approximately 42 of Idaho’s 115 school districts, with a primary goal of cost savings (e.g., savings on transportation, heating, janitorial, and clerical costs). The revised schedule adds roughly 30 to 90 minutes to each day that students are in school, and then on the fifth day (usually Friday), the goal is to assign projects and encourage parent and community groups to organize study halls and enrichment activities.
The authors collected data by interviewing district and school leaders in Idaho communities that had moved to the four-day week. Key findings included:
  • Though cost cutting was the original motivation for the four-day week, savings have been elusive in most localities. This is because so many costs are fixed (e.g., teacher and administrator salaries).
  • Teachers reported difficulty restarting instruction and focusing children’s attention after a three-day weekend.
  • Teachers in many places now consider the fifth day an amenity, and some superintendents reported that the four-day week made the locality more attractive to teacher candidates.
  • Working parents found the longer hours in school more convenient as it meant that children’s days more nearly matched their own workdays. However, the fifth day presented new problems of child care and planning for positive uses of children’s time.
  • No district that had adopted the four-day week had rigorously assessed the effects on student achievement. Several district leaders said student and teacher attendance had improved in the first year of the four-day week, but they had not assessed whether these results persisted over time.
The authors discuss the limits of their evidence, and note that the long-term effects for rural students’ education are unknown.

1 Comment

Filed under At home, Education, Research, Review

Does successfully improving students’ achievement test scores lead to higher rates of national economic growth?

No, this isn’t a OECD or PISA-bashing post, but I found a new study by Komatsua and Rappleye via @cbokhove that raises an important question: does successfully improving students’ achievement test scores lead to higher rates of national economic growth. This is a claim based on research by Hanushek and Woessmann and is the basis for a lot of policy-influencing research and policy-advice by e.g. PISA or the World bank. But Komatsua and Rappleye argue now that this claim is maybe based on flawed statistics, as the abstract makes clear what it’s all about:

Several recent, highly influential comparative studies have made strong statistical claims that improvements on global learning assessments such as PISA will lead to higher GDP growth rates. These claims have provided the primary source of legitimation for policy reforms championed by leading international organisations, most notably the World Bank and OECD. To date there have been several critiques but these have been too limited to challenge the validity of the claims. The consequence is continued utilisation and citation of these strong claims, resulting in a growing aura of scientific truth and concrete policy reforms. In this piece we report findings from two original studies that invalidate these statistical claims. Our intent is to contribute to a more rigorous global discussion on education policy, as well as call attention to the fact that the new global policy regime is founded on flawed statistics.

They performed a replication on the same data Hanushek and Woessmann used and their conclusion sounds a bit damning:

Our primary purpose has been to report findings from two studies where our results invalidate H&W’s strong statistical claims that attempt to link student test scores and economic growth. In Study 1, we observed that the explanatory power of test scores was weak in subsequent periods: the relationship between scores and growth has been neither consistently strong nor strongly consistent. In Study 2, we observed that the relationship between changes in test scores in one period and changes in economic growth for subsequent periods were unclear at best, doubtful at worst. Combined, these two original studies do not simply challenge the key statistical claims advanced by H&W but invalidate them because they utilise the same sample, dataset and methods.

But I have to agree with Christian Bokhove in our tweet conversation about this article that both scientists are quite a bit firm in their statements. Still: I think it’s too important not to have further debate on this topic as it’s quite essential to present policy.

1 Comment

Filed under Education, Research, Review

Important new meta-analysis on the testing effect – with some surprises…

There is a new Best Evidence in Brief and one of the studies the newsletter discusses is all about the effect of testing:

Olusola O. Adesope and colleagues conducted a meta-analysis to summarize the learning benefits of taking a practice test versus other forms of non-testing learning conditions, such as re-studying, practice, filler activities, or no presentation of the material.

Results from 272 independent effects from 188 separate experiments demonstrated that the use of practice tests is associated with a moderate, statistically significant weighted mean effect size compared to re-studying (+0.51) and a much larger weighted mean effect size (+ 0.93) when compared to filler or no activities.

In addition, the format, number, and frequency of practice tests make a difference for the learning benefits on a final test. Practice tests with a multiple-choice option have a larger weighted mean effect size (+0.70) than short-answer tests (+0.48). A single practice test prior to the final test is more effective than when students take several practice tests. However, the timing should be carefully considered. A gap of less than a day between the practice and final tests showed a smaller weighted effect size than when there is a gap of one to six days (+0.56 and + 0.82, respectively).  

Are you as baffled by these results as I am? I checked in the original article to find out more.

Eg. MC-questions having a bigger effect size – while remembering is often harder than recognizing? Well, there are some studies who actually support the latter:

For instance, Kang et al. (2007) revealed that students who took a short-answer practice test outperformed students who took a multiple-choice practice test on the final test, regardless of whether the final test was short-answer or multiple-choice.

But:

On the other hand, C. D. Morris, Bransford, and Franks’s (1977) research on levels of processing suggests that retention is strongest when processing demands are less demanding. They reason that this is because less demanding retrieval practice activities allow participants to focus all of their cognitive energy on a simple task at hand, whereas deeper levels of processing require more cognitive energy and can distract participants from relevant aspects (C. D. Morris et al., 1977).

And looking at the meta-analysis, the second theory seems to be winning as “the differences between multiple-choice and short-answer practice test formats did emerge: g = 0.70 and g = 0.48, respectively” But it’s worth noting that the researchers do warn it’s not that simple:

…we found that multiple-choice testing was the most effective format; however, this should be interpreted with caution, since an educator’s decision to use any given format should be based on the content of the learning material and the expected learning outcomes. For example, multiple- choice tests may be especially useful for memorization and fact retention, while short-answer testing may require more higher order thinking skills that are useful for more conceptual and abstract learning content

And what about a single test being more effective than taking several practice tests? The meta-analysis does support this, but Adesope et al. can only guess why:

Thus, our findings suggest that although a single test prior to a final test may result in better performance, the timing of the test should be carefully considered. One plausible explanation is more time between the practice and final tests allows students to mentally recall and process information, leading to deeper learning. An alternative hypothesis is that multiple tests within a short time may result in test fatigue that affects performance, while retrieval practice over a distributed time period enables long-term storage.

I do think that this meta-analysis will invite other researcher do join the debate…

Abstract of the study:

The testing effect is a well-known concept referring to gains in learning and retention that can occur when students take a practice test on studied material before taking a final test on the same material. Research demonstrates that students who take practice tests often outperform students in nontesting learning conditions such as restudying, practice, filler activities, or no presentation of the material. However, evidence-based meta-analysis is needed to develop a comprehensive understanding of the conditions under which practice tests enhance or inhibit learning. This meta-analysis fills this gap by examining the effects of practice tests versus nontesting learning conditions. Results reveal that practice tests are more beneficial for learning than restudying and all other comparison conditions. Mean effect sizes were moderated by the features of practice tests, participant and study characteristics, outcome constructs, and methodological features of the studies. Findings may guide the use of practice tests to advance student learning, and inform students, teachers, researchers, and policymakers. This article concludes with the theoretical and practical implications of the meta-analysis.

2 Comments

Filed under Education, Research, Review

New website for Evidence-based education: Evidence for ESSA

Evidence for ESSA, is a free web-based resource that provides easy access to information on programs that meet the evidence standards defined in the American Every Student Succeeds Act (ESSA). The website is designed for education leaders at the state, district, and school levels, to provide information to state chiefs, superintendents, principals, teachers, parents, and anyone else interested in which programs meet the ESSA evidence standards.

While this website is aimed at educators in the United States, I do think it can be interesting to a global audience as the website reviews math and reading programs for grades K to 12 to determine which meet the strong, moderate, or promising levels of evidence defined in ESSA (additional subject areas will be added later). The site provides a one-page summary of each program, including a program description, brief research review, and practical information on costs, professional development, and technology requirements. It is easily searchable and searches can be refined for particular groups (such as African Americans, Hispanics, or English Learners); communities (urban, suburban, or rural); and program features (such as technology, cooperative learning, or tutoring). Evidence for ESSA directs users to the key studies that validate that a program meets a particular ESSA standard. 

The mission of Evidence for ESSA is to provide clear and authoritative information on programs that meet the ESSA evidence standards and enable educators and communities to select effective educational tools to improve student success.

1 Comment

Filed under Education, Research, Review

Infographic by Oliver Caviglioli and Paul Kirschner: the case for fully-guided instruction

Oliver Caviglioli The same guy who designed the great study method posters by the learning scientists worked now together with Paul Kirschner on this infographic which you can download here!

3 Comments

Filed under Education, Review