Good warning: One in ten student research participants don’t make an effort

Many – psychology – research uses students as respondents, but this is not without danger as BPS Digest describes in their post on this new study:

Jonathan DeRight and Randall Jorgensen at Syracuse University have investigated student effort in 90 minutes of computerised neuropsychology tests designed to measure attention, memory, verbal ability and more. The session, which took place either during a morning or afternoon late in the Spring semester, involved the students taking the same broad battery of tests twice, with a short gap in between. The students received course credits for their time.

To test whether the students were making a proper effort, the researchers embedded several measures – for example, performing worse than chance on a multiple-choice style verbal memory challenge was taken as a sign of low effort; so was performing more slowly on an easier version of a mental control task than on the more difficult version.

Among the 77 healthy student participants who took part (average age 19; 36 women), the researchers identified 12 per cent who failed at least one of the embedded measures of effort during the first battery of neuropsych tests; 11 per cent also failed one or more measures during the second battery. The vast majority of those who showed low effort had participated in the morning. In fact, focusing only on the morning participants, one in four displayed low effort.

Abstract of the study:

Although performance validity testing is becoming fairly routine in clinical settings, research protocols involving neuropsychological tests infrequently include assessments of performance validity. The current study utilized an embedded measure of effort over two administrations of CNS Vital Signs to determine the frequency of poor effort in non-clinical healthy undergraduate students participating in a research study for course credit. Results indicate that more than 1 in 10 college students participating in a cognitive test battery for research showed test scores consistent with inadequate effort, which was associated with poor performance on testing across many domains. This conclusion was supported by poor performance on many other subtests. Healthy college students with suboptimal effort (n = 11) had an overall score in the 15th percentile on average compared to the 48th percentile in the rest of the students (n = 66). Those who failed validity indicators on the baseline administration were more likely to fail validity indicators on the repeat administration. Those who were tested in the morning were also more likely to fail validity indicators. The current study provides evidence for the potential limitations of conducting research using neuropsychological tests with healthy college student volunteers in the absence of performance validity testing. Revised college-level cutoffs are proposed.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.