We've updated our Privacy Policy to make it clearer how we use your personal data.

We use cookies to provide you with a better experience. You can read our Cookie Policy here.

Advertisement
A New Study Finds That Many Published Psychology Experiments Lack Evidence of Validity
News

A New Study Finds That Many Published Psychology Experiments Lack Evidence of Validity

A New Study Finds That Many Published Psychology Experiments Lack Evidence of Validity
News

A New Study Finds That Many Published Psychology Experiments Lack Evidence of Validity

Pixabay
Read time:
 

Want a FREE PDF version of This News Story?

Complete the form below and we will email you a PDF version of "A New Study Finds That Many Published Psychology Experiments Lack Evidence of Validity"

First Name*
Last Name*
Email Address*
Country*
Company Type*
Job Function*
Would you like to receive further email communication from Technology Networks?

Technology Networks Ltd. needs the contact information you provide to us to contact you about our products and services. You may unsubscribe from these communications at any time. For information on how to unsubscribe, as well as our privacy practices and commitment to protecting your privacy, check out our Privacy Policy

An examination of nearly 350 published psychological experiments found that nearly half failed to show that they were based on a valid foundation of empirical evidence, suggesting that a wide swath of psychological science is based on an "untested foundation."

The study -- conducted by David Chester, Ph.D., a psychology professor at Virginia Commonwealth University, and Emily Lasko, a psychology doctoral student at VCU -- focuses on the practice of experimental manipulations, in which psychologists induce specific mental states, such as giving research participants insulting or complementary feedback to manipulate how angry they feel.

To conduct these experimental manipulations in a scientifically valid way, researchers must first establish whether their manipulations actually affect the intended psychological variable (for example: make people feel angry) and not other closely related variables (for example: make people feel sad). However, the extent to which psychologists actually examine the validity of their manipulations remains unknown.

Chester and Lasko investigated 348 psychological manipulations included in peer-reviewed studies. They found that roughly 42% of the experiments were paired with no validity evidence, and that the remaining psychological manipulations were validated in ways that were extremely limited.

"These findings call into question the accuracy of one of psychology's most common practices and suggest that the field needs to strongly improve its practices in this methodological domain," said Chester, an assistant professor in theDepartment of Psychologyin theCollege of Humanities and Sciences.

The forthcoming study, "Construct Validation of Experimental Manipulations in Social Psychology: Current Practices and Recommendations for the Future," will be published in the journal Perspectives on Psychological Science.

The study is the first of its kind to systematically document the extent to which psychology's experiments are based on a valid foundation of empirical evidence.

Importantly, Chester said, the study's findings do not suggest that the experimental psychologists' findings were necessarily wrong or invalid.

"We do not find such experiments are invalid, instead we simply don't have the evidence to know one way or another how valid they are," he said. "Almost all of the manipulations we examined failed to provide the necessary evidence that they were valid, which does not mean they are invalid -- their validity is just unknown."

As a result, he said, the study suggests that "the findings of experimental psychology likely rest on an untested foundation."

"This framework might be weak, it might be strong, it is more likely both of these things depending on many factors," he said. "We have outlined a prescribed series of recommendations for experimenters to ensure that this is not the case going forward -- that the validity of each experimental manipulation is tested in a systematic and accurate way."

Chester added that he and Lasko hope their findings encourage experimental psychologists to include validity evidence in future research.

"We hope our paper makes experimenters aware of this untested aspect of their research, motivates them to change their practices, and provides a road map of precisely what to do to make such changes," he said.

Reference:

David S. Chester, Emily N. Lasko. Construct Validation of Experimental Manipulations in Social Psychology: Current Practices and Recommendations for the Future. Perspectives on Psychological Science (in press), 2020 DOI: 10.31234/osf.io/t7ev9

This article has been republished from the following materials. Note: material may have been edited for length and content. For further information, please contact the cited source.

Advertisement