Eight years ago, a team of researchers started a project to carefully replicate early and influential laboratory experiments in cancer research. The Reproducibility Project, as it is called, recreated 50 experiments. It is now reported that about half of the experiments did not produce the reported results originally.
The reproducibility of experiments and the confirmation of the results are of central importance for the broad acceptance of scientific claims.
âThe truth is that we are deceiving ourselves. Most of what we claim is novel or significant doesn’t exist, “said Dr. Vinay Prasad. He is a cancer doctor and researcher at the University of California, San Francisco and was not involved in the project.
Most scientists believe that the strongest results come from experiments that can be repeated with similar results.
However, there is little reason for researchers to share methods and data for others to validate the work, said Marcia McNutt. She is President of the National Academy of Sciences. Researchers lose respect in the scientific community when their results don’t stand up to careful scrutiny, she added.
For the project, the researchers tried to repeat experiments from cancer biological papers. The papers appeared in major scientific publications from 2010 to 2012, including Nature and Cell.
Overall, 54 percent of the original results did not meet the requirements of the reproducibility project. The team’s study appears in diary eLife. The non-profit eLife is supported by the Howard Hughes Medical Institute, which also supports the Associated Press Health and Science Department.
Among the studies that failed was one that suggested a link between stomach bacteria and colon cancer. Another claimed a drug had shrunk tumors in mice. And a third was a mouse study of a potential prostate cancer drug.
This is the second major study in the reproducibility project. In 2015, they found similar problems when trying to repeat experiments in psychology.
Study co-author, Brian Nosek of the Center for Open Science, said it can be wasteful to move forward without doing the work first to replicate the results.
The researchers tried to limit the differences in how the cancer experiments were performed. Often times, they couldn’t get any help from the scientists who did the original work. They couldn’t get answers to questions about what kind of mice to use or where to find specially made tumor cells.
Michael Lauer is the Assistant Director of Research for the National Institutes of Health (NIH). He said, “I was not surprised, but what is worrying is that about a third of the scientists have not been helpful, and in some cases have been more than helpful.”
Lauer added that the NIH plans to mandate data sharing between organizations it supports in 2023.
Dr. Glenn Begley is a biotechnology consultant and former head of cancer research at drug company Amgen. Ten years ago, he and other scientists at Amgen reported even lower confirmation rates when trying to replicate published cancer experiments.
Cancer research is difficult, Begley said. He added that “it is very easy for researchers to be drawn to results that look exciting and” provocative“Findings that seem to further support her favorite idea of ââhow cancer should work, but they are just wrong.”
I am Jonathan Evans.
Carla K. Johnson covered the story for the Associated Press. Jonathan Evans adapted this story for Learning English. Caty Weaver was the editor.
Words in this story
originally – Adult at the beginning; when something first happened or started
novel – adj. new and different than previously known
significant – adj. big enough to be noticed or to have an effect
diary – n. a magazine that reports on things that are of particular interest to a certain group of people
provocative – adj. Discussions, thoughts, arguments, etc.