The integrity battlefield: where science meets policy

4 03 2022

Professor Ross Thompson, University of Canberra


On the whole, I am inclined to conclude that my experience of academia and publishing my work has been largely benign. Despite having published 120-odd peer-reviewed papers, I can count the number of major disputes on one hand. Where there have been disagreements, they have centred on issues of content, and despite the odd grumble, things have rarely escalated to the ad hominem. I have certainly never experienced concerted attacks on my work.

But that changed recently. I work in water science, participating in and leading multi-disciplinary teams that do research directly relevant to water policy and management. My colleagues and I work closely with state and federal governments and are often funded by them through a variety of mechanisms. Our teams are a complex blend of scientists from universities, state and federal research agencies, and private-sector consultancies. Water is big business in Australia, and its management is particularly pertinent as the world’s driest inhabited continent struggles to come to terms with the impacts of climate change.

In the last 10 years, Australia has undergone a AU$16 billion program of water reform that has highlighted the extreme pressure on ecosystems, rural communities, and water-dependent industries. In 2019, two documentaries (Cash Splash and Pumped) broadcast by the Australian Broadcasting Corporation were highly critical of the  outcomes of water reform. A group of scientists involved in working on the Murray-Darling Basin were concerned enough about the accuracy of aspects of those stories to support Professor Rob Vertessy from the University of Melbourne in drafting an Open Letter in response. I was a co-author on that letter, and something into which I did not enter lightly. We were very concerned about being seen to advocate for any particular policy position, but were simultaneously committed to contributing to an informed public debate. A later investigation by the Australian Communications and Media Authority also highlighted concerns with the Cash Splash documentary.

Fast forward to 2021 and the publication of a paper by Colloff et al. (2021) in the Australasian Journal of Water Resources. In that paper, the authors were critical of the scientists that had contributed to the Open Letter and claimed they had been subject to “administrative capture” and “issue advocacy”. Administrative capture is defined here as:

“… when some scientists act as self-interested Issue Advocates (Pielke 2007) and, thus, deliberately seek to constrain either the scientific issues addressed, the decision options explored or to constrain the evidence, data and views that are considered within scientific-policy debates.”

At the time, I was working on a draft of a paper with a colleague and friend of mine, the late Professor Sue Briggs, trying to define a set of ‘principles’ for science-policy-management engagement. We were really interested in the question of how scientists who are funded by government can protect their independence while at the same time producing timely science relevant to management and policy. I broadened the principles paper to include a range of local and international experts and used the Open Letter as a case study. 

In parallel, another colleague (Michael Stewardson) from the University of Melbourne led a response piece to the Colloff et al. (2021) paper. That paper (Stewardson et al. 2021) identified a set of concerns over the logic, use of evidence, and interpretations therein. 

Meanwhile, I struggled along with preparing the principles manuscript, which now included 15 authors. I was a little bit paranoid about ensuring that all authors were comfortable with the text, so there were literally hundreds of emails and dozens of drafts circulated. Fortunately, I retained all that correspondence. The whole process was complicated still further by the fact that my co-author Sue Briggs had passed away after a short illness in December 2020. Finally in May 2021, we submitted the paper to the journal River Research and Applications. During the submission process, the automatic system asked me if I wanted to make the preprint available via the online Authorea platform. “Sure”, I thought. “Why not?”

The first sign of trouble was when one of my co-authors on the principles paper rang me and described a phone call from one of the authors of the Colloff et al. (2021) paper. Regarding the preprint we had just published, my colleague conveyed several allegations of misconduct labelled against us, such as the inclusion of authors without permission and without sufficient contribution. Those comments were repeated in letters to another co-author on the principlespaper (who kindly forwarded the comments on to me).

I found this all quite remarkable, particularly because as lead author, I had not been approached. I duly reached out to the Colloff et al. team. The response doubled down on accusations of misconduct — specifically accusing me of including a deceased author who had not contributed, and listing other authors who had not contributed (“guest authors”).

“Shit”, I thought. “What do I do here?” These kinds of accusations can end careers, and are taken very seriously. They also make you ask yourself, “Am I behaving with integrity?” “Am I being rigorous enough about consulting with and including co-authors?”

What I first chose to do was to brief the journal editors (our manuscript was still under review at the time) and my co-authors. The second step was an extreme one — I made the decision to submit all my materials to my University Research Office for a review of the claims of misconduct (thank goodness they found no case to answer; otherwise, I would have been feeling VERY silly). Thankfully, I had retained all my email correspondence, manuscript drafts, and comments (note to self: always do this). 

Ultimately, the principles paper was published earlier this year (minus material that had already appeared in the Stewardson et al. 2021). Understandably, the journal, aware of the controversy, had been extremely diligent in the peer-review and revision process. Hopefully, the final paper makes a useful contribution to the literature and helps others navigate this contested space.

So, what to take from all of this? 

1. Be wary. There are certainly researchers who are willing to go outside of the usual peer-review processes. In some cases, they will pursue this to a point that can genuinely damage reputations and careers. Be warned. Be rigorous about keeping emails and drafts. Ask questions over your own conduct and seek advice when you are concerned.

2. Be cautious with pre-prints. Releasing the submission on Authorea created an environment that made it much more challenging to get the paper published because of the heightened attention the draft received. I wouldn’t do it again, although I understand why some people like the idea of making pre-prints available. 

3. Check yourself. In an area that seemingly everyone has a vested interest, it is easy and tempting to get ‘hot under the collar’ and flick off emails and letters. I’ve done it myself. Most of the time, this comes from genuine convictions and a real passion for the topic. We have expressed our concerns over the politicisation of water science in Stewardson et al. (2021), and those concerns are very real ones. Heated exchanges that target individuals, particularly when they are played out in the public arena, don’t help the scientific process — they damage it. 

4. Ask questions of your own motives and conduct. Researchers are increasingly asked to work more closely with industry and government partners. In general, I think this is a good thing. But it does bring challenges in terms of maintaining scientific independence that we describe in Thompson et al. (2022). Asking questions over the integrity of data and interpretations is valid and should happen through the peer-reviewed literature. Asking questions of the integrity of individuals needs to be done with much more care. In this case, it has helped me ask questions of myself and the arrangements put in place around my work that support my scientific independence (we describe an example of those in Thompson et al. 2022). 

It’s been a bit of a brutal experience. I’m not a person that enjoys conflict and I have never learned how to let this stuff wash over me. It has kept me awake at night and meant my eight-year old has had to cope with a grumpy dad at times. People who I respected have behaved in ways that make me uncomfortable. 

It all makes me wonder if we have become a little bit too remote from the consequences of our actions. Perhaps we just need to remember that there is a person on the other end of that scientific paper or op ed, that it is unlikely that they are a truly ‘bad’ person, and try and keep the discussions respectful. This whole process has certainly encouraged me to hold myself to higher standards when participating in these kinds of exchanges. 


References


Actions

Information

One response

4 03 2022
David Jay

This is a brave, thoughtful and important piece – thank you.

Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s




%d bloggers like this: