Published: Dec. 3, 2015 By

After three years, Brandon Stell reveals role in establishing controversial website


The mastermind behind a formerly anonymously run website that serves as a crowd-sourced watchdog of peer-reviewed scientific journal articles turns out to be Brandon Stell, a 1997 graduate of the University of Colorado Boulder.

Stell recently disclosed that he is the architect ofĢż, the controversial online public forum where scientists can anonymously critique each otherā€™s published research. The siteā€™s users have exposed many errors in published works.

While a student at CU-Boulder, Brandon Stell joined discussions of published, peer-reviewed research. He noted a high frequency of errors or other problems in the work, and he thought it was a shame that such feedback could not be shared widely. Eventually, this experience helped him hatch the idea for what became PubPeer. Photo courtesy of Brandon Stell.

While a student at CU-Boulder, Brandon Stell joined discussions of published, peer-reviewed research. He noted a high frequency of errors or other problems in the work, and he thought it was a shame that such feedback could not be shared widely. Eventually, this experience helped him hatch the idea for what became PubPeer. Photo courtesy of Brandon Stell.

The idea for PubPeer originated when Stell was an undergraduate at CU-Boulder studying ecology, populations and organismal biology. Stell regularly attended meetings to discuss recently published research.

The discussion groups dissected papers, scrutinizing each experiment and conclusion. This in-depth review of peer-reviewed research always revealed a few flaws.

ā€œIā€™ve never been to a journal club meeting that didnā€™t turn up one or two issues about a paper,ā€ says Stell.

Stell enjoyed the occasionally brutal discussions, but he left the meetings disappointed. The insights, remaining questions and possible flaws regarding the research hung in the air with no resolution.

ā€œI always thought that the authors could maybe clarify an issue if they had a voice at the journal club, but they didnā€™t. And, when issues were brought up in the meeting, no other scientists in that field ever heard about it,ā€ says Stell.

ā€œThat is really important because scientists build off of each otherā€™s work.ā€

We thought, if we could create this community where people are discussing science that will take the focus off of where science is published and put it on the content or the data that weā€™re publishing.ā€

Stell longed for a centralized, public forum for scientists to discuss and critique each otherā€™s published research.

The idea that grew into PubPeer percolated for years. Throughout his graduate studies in neuroscience at the University of California Los Angeles and a post-doctoral fellowship in Paris, Stell ranted about this idea in hallways, perpetually asking colleagues why someone else hadnā€™t already launched such a forum.

He even contacted Google Scholar, a specialized search engine for academic and scholarly literature, hoping to encourage it to establish such a site. His email went unanswered.

He caught a break when Richard Smith came to work in his lab in Paris. Richard and his brother George, who provides technical expertise, helped Stell launch the site in 2012. (PubPeer has revealed the brothersā€™ names but not their professional affiliations.)

The site published anonymous critiques for an obvious reason: Criticism is rarely complimentary. And because peers in the same area of expertise review oneā€™s research, any critique carries the risk of offending a colleague. This could have significant consequences for the criticā€™s career.

ā€œIf some of my colleaguesā€™ papers were critiqued, people who were reviewing my grants and my papers and basically deciding my career, if their papers were being critiqued on the site and they knew that I was behind the site, they probably wouldnā€™t be very happy about that,ā€ Stell explains.

ā€œIt would also put pressure on me to remove comments that people were unhappy with. So, we were anonymous, and because we were anonymous, it made it difficult to promote the site. You canā€™t really do that anonymously.ā€

Despite the anonymity and lack of promotion, the site gained popularity. In fact, once anonymous commenting was enabled, the siteā€™s use more than tripled in less than two years (). One success of PubPeer is that it allows other scientists in a field, who are building on that published research, to learn about other researchersā€™ critiques and concerns.

ā€œWhen there are problems that are noticed, then other scientists [in that field] are aware of it¾±³¾³¾±š»å¾±²¹³Ł±š±ō²āĢżbecause thereā€™s now this forum,ā€ explains Stell.

Stell and colleagues saw that this was going to have broader implications.

ā€œIf scientists donā€™t know that there are problems with a research article, this has major cost,ā€ says Stell.

ā€œThereā€™s an economic cost ā€“ weā€™re wasting taxpayer money trying to reproduce or build on [research] that canā€™t be built upon. It also has a human cost because graduate students and post-docs could be wasting their careers trying to build on something that canā€™t be built upon. If a student spends three years trying to build on something thatā€™s not very solid, and in the end canā€™t do it, it makes it very difficult for their scientific career.ā€

With a system like PubPeer, as soon as thereā€™s a problem with a paper, people who use the research as a foundation for future initiatives, including policymakers, health-care providers, and pharmaceutical companies know about it.

Additionally, PubPeer provides an avenue for focusing scientistsā€™ attention on the fundamental issue: science.

ā€œWe thought, if we could create this community where people are discussing science that will take the focus off of where science is published and put it on the content or the data that weā€™re publishing,ā€ explains Stell.

ā€œI feel that scientists are on the edge of an interesting moment in science where weā€™re collectively questioning the role of journals.ā€

ā€œPublication is not the end of the road,ā€ according to Stell. ā€œItā€™s not, ā€˜OK Iā€™ve been through the review and publication process, the paperā€™s now in a coveted research journal, I can cash it in for jobs, or grant money.ā€™ā€

ā€œIf the data arenā€™t solid, thereā€™s a conversation thatā€™s going to happen about that.ā€

Roni Dengler is a writer for CUā€™s Science Buffs blog and a Ph.D. candidate in the Molecular, Cellular and Developmental Biology Department.

Ģż