A little bomb that doesn’t surprise: Study reveals – again – that students aren’t ideal test subjects for behavioral research

Oh, it can be so handy: you need to do experiments, and you need volunteers. Maybe we can ask our students for a kind of trade-off for credits? It happens in a lot of psychology departments around the globe, but are students representative of the total population? This research about students in agricultural studies suggests – again – this is not the case.

From the press release:

Students are popular test subjects for many studies in behavioural sciences. However, using only students does not reveal the full picture about people in general. In fact, many of the students’ decisions in those experiments differ from those of other population groups. These are the findings of a new, extensive study consisting of 36 experiments which was conducted by a team of behavioural scientists from Martin Luther University Halle-Wittenberg (MLU). The study was published in the Australian Journal of Agricultural and Resource Economics.

Students are popular test subjects, especially in behavioural economics. “This makes sense because students are open to and interested in scientific studies, they are already in a university setting, and also they are receptive to financial incentives offered in those studies,” says Dr Sven Grüner from the Institute of Agricultural and Food Sciences at MLU. “However, it is unclear whether students are representative of other population groups — after all, they differ in important ways, such as age and income.” To answer this question, the behavioural economist conducted an elaborate study using 300 test subjects. The result: Only limited conclusions can be drawn from students about the behaviour of other people.

In a total of 36 sub-experiments, Grüner compared the decisions of agricultural science students to those of farmers. The team examined individual characteristics such as risk-taking, impatience, altruism, trust, punishing unfair and rewarding generous behaviour. “We used established economic experiments from decision and game theory,” Grüner explains. For example, when determining willingness to take risks, the test subjects were given the choice between a higher probability of winning a small amount of money and a lower probability of winning a higher amount of money. “We gradually increased the monetary incentives in all of the experiments to see how the expected sum influenced decisions,” says Grüner. Contrary to previous studies, the incentive was actually given to the subjects afterwards, because theoretical earnings could falsify the results: If participants knew they would not get any money, they might have shown a greater degree of socially desirable behaviour.

The results of the comprehensive study revealed a very mixed picture: for example, there are no clear differences between the groups in terms of risk-taking. “This contradicts earlier studies in which students were more risk averse than farmers,” says Grüner. The differences were also slight when it came to trust and rewarding generous behaviour. However, when testing for the groups’ patience there were bigger differences: farmers were much more likely to choose the option with the higher probability of a lower pay out, while students were consistently found to be more patient and wait longer for more money. At the same time, farmers turned down unfair offers more often, even when this meant they would not get any money themselves. These finding are not in line with earlier studies that showed similar behaviour among students and other population groups, says Grüner.

“Our study shows that it is really problematic to generalize the behaviour of students to other real actors. This could call into question a lot of the results of previous studies — not only in agricultural sciences, but across all disciplines,” adds Grüner. This is also a sensitive topic because surveys on individual decision-making examine important questions about the future: risk behaviour and patience, for example, are decisive criteria for investing in sustainable production structures that usually only pay off after many years. The new study helps to identify factors that can be used to weight the results.

Abstract of the study:

Experiments are often used to study individual decision-making under controlled circumstances. Due to their low opportunity costs and high availability, university students are frequently recruited as the study population. Even though they are rather untypical with regard to many characteristics (e.g. age and income) compared to the representatives of the social group of interest, the experimental behaviours of students are sometimes prematurely generalised to other social groups or even to humans in general. Given the widespread challenges in the agricultural and environmental sector, it is particularly interesting to address farmers’ decision-making. We analyse whether agricultural students can be used to approximate the behaviour of farmers in simple economic experiments, which are often used to measure risk aversion, impatience, positive reciprocity, negative reciprocity, altruism and trust. Moreover, we consider the role of systematically varied monetary incentives. We find no differences between agricultural students and farmers in their risk aversion; farmers’ positive reciprocity and trust are positively associated with the incentive level, which cannot be observed with agricultural students. Findings regarding altruism in the two populations are mixed and challenge the finding of earlier studies of students being less pro-social. Agricultural students are a lower boundary of impatience and negative reciprocity. These heterogeneous results suggest that scientific inference from agricultural students to farmers should be made cautiously. However, we do not deal with a representative sample of our target population (e.g. gender). Replication studies are required to evaluate the generalisability of our findings.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.