Fish and Wildlife Study Reminds Us That Informing Policy Doesn’t Mean Dictating Policy
Here’s a link to David Bruggeman’s post on his blog. I subscribe to, and recommmend, David’s blog to keep up with science policy issues.
David cites the Nature blog, here it is and below is an excerpt:
The researchers claim that this is the first study to examine government peer review. “Peer review is fundamentally different in a government setting than at a scientific journal,” says Noah Greenwald, a co-author and endangered species program director at the Center for Biological Diversity in Portland, Oregon. “There’s no editor looking at whether they followed the advice of reviewers and whether they have a good explanation for not following it.” Most of the time, the final habitat boundaries did not match the drafts set by the internal scientists. In 81% of the cases, the researchers found, the habitats were cut by more 40% between the draft and final policies.
The FWS says it did not have time to review the study, but issued a short statement, arguing that the agency, in setting boundaries, must consider economic and national security concerns, and not just science. “Scientists may not always agree on the conclusions of a scientific analysis, especially in analyses as complex and challenging as critical habitat designations. In some cases, peer reviewers may disagree; in others, our biologists may not agree with the conclusions of individual peer reviewers.”
Karen Hodges, an ecologist at the University of British Columbia in Kenowa, agrees that designating critical habitat can be a tricky business. “Here’s a species that you need to protect, and you need to pick how much habitat to protect so it doesn’t go extinct. Good luck with that! That’s hard for species we know about, let alone species with limited data,” she says.
Note from Sharon: Karen’s statement sounds a bit like my previous comment. Also, there is a substantial literature on peer review in the sciences and the results are not all rosy. It would be too easy to go after some of the climate peer review debates; but I was cleaning out my office this week prior to retirement and found some older literature (95 ish) about gender bias in peer review. Scientists are people, after all, with the inherent tribal tendencies, and reviewing manuscripts fall under “other duties as assigned.” For you non-scientists, when scientists go on about the wonders of peer review, you might want to look closely at what else they are trying to communicate. In fact, their statements about peer review don’t seem to be informed by the best science..
However, this was not”peer review” of a scientific publication, it really sounds like what David says here in his post, it’s a policy decision:
This drew my attention because of a comment in the Nature piece by one of the co-authors of the BioScience piece. He described the FWS actions as ‘a scientific integrity issue.’ Which raises the perennial disconnect for many scientists. Having scientific and technical information inform policymaking is not the same thing as having it dictate policy. As Nature describes the FWS response to the study,
“the agency, in setting boundaries, must consider economic and national security concerns, and not just science. ‘Scientists may not always agree on the conclusions of a scientific analysis, especially in analyses as complex and challenging as critical habitat designations. In some cases, peer reviewers may disagree; in others, our biologists may not agree with the conclusions of individual peer reviewers.’”
Absent additional evidence, or further explanations of how the study authors see the political interference they refer to in the BioScience piece, I don’t think they have made the case that FWS action in this case qualifies as a scientific integrity matter. While they acknowledge that other influences factor into agency decisions, they don’t seem to like it. Near the conclusion of the article, they state:
“scientists within the USFWS need to be given leeway and clear direction in order to base their decisions solely on the best available scientific information. The loss of biodiversity is too serious a problem to let short-term political interests intrude.”
The authors aren’t wrong to want these things. But they should recognize that they are advocating for changing how policy decisions are made, rather than restoring the role of science in decisionmaking. They aren’t alone in this.
The discussion of informing versus dictating reminds me of all the discussion on the planning rule about considering versus some other word. Which could have been informed by research on science technology policy studies; or perhaps dictated ?
I also find it kind of bizarre that scientists feel like they can show a fundamental misunderstanding and/or disrespect of legitimate (and democratic) political processes in, of all places, a scientific journal. I wonder what the editors were thinking.