Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology

Gould, Elliot , Fraser, Hannah S , Parker, Timothy H , Nakagawa, Shinichi , Griffith, Simon C , Vesk, Peter A , Fidler, Fiona , Hamilton, Daniel G , Abbey-Lee, Robin N , Abbott, Jessica K , Caravaggi, Anthony , others (2025). BMC biology. 35.

DOI | PDF | Data & Code

Summary

Even when using the same data, researchers can reach very different conclusions because of the choices they make in analyzing it. In a study with 174 teams, analyses of Blue Tits and Eucalyptus trees showed wide variation in results. Some effects were strong, some weak, and some opposite to the average trend. These differences weren’t explained by the methods or peer reviews used. The study highlights that analytical decisions can strongly shape outcomes, so ecologists and evolutionary biologists need to interpret results carefully and make analyses more transparent.

BibTeX

@article{gould2025same,
 author = {Gould, Elliot and Fraser, Hannah S and Parker, Timothy H and Nakagawa, Shinichi and Griffith, Simon C and Vesk, Peter A and Fidler, Fiona and Hamilton, Daniel G and Abbey-Lee, Robin N and Abbott, Jessica K, Anthony Caravaggi and others},
 journal = {BMC biology},
 number = {1},
 pages = {35},
 publisher = {BioMed Central London},
 title = {Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology},
 volume = {23},
 year = {2025}
}