A brief exploration of how unsubstantiated statements can enter the scientific literature and rapidly take on an air of reliability, in only three generations.


In 1675, Isaac Newton famously said “If I have seen further it is by standing on the shoulders of Giants”. The modern peer reviewed publication process is designed to help make this process rapid and transparent.  Searchable databases help us quickly find the work on which we seek to build, while citations make the pedigree of knowledge clear.

It is a system that supports trust in scientific advances. It has become so well accepted that, at times, we take it for granted. As flawed humans, with busy lives to lead, we generally accept that a citation to a peer-reviewed article, used as evidence or justification for an assertion, is trustworthy. Every now and then, it is a little frightening to be reminded how quickly ‘knowledge’ can acquire the stamp of scientific approval.

The child

In search of information on the prevalence of piscine reovirus, I found the following promising reference:

Kibenge MJT, Wang Y, Morton A, Routledge R, Kibenge FSB (2017) Formal comment on: Piscine reovirus: Genomic and molecular phylogenetic analysis from farmed and wild salmonids collected on the Canada/US Pacific Coast. PLoS ONE 12(11): e0188690. https://doi.org/10.1371/journal.pone.0188690

PRV is now considered ubiquitous in farmed Atlantic salmon (Haatveit et al. 2017 [4]) and has an estimated 80% prevalence rate among BC farmed salmon (Kibenge et al. 2013 [8]).

The parent

The figure I was after is qualified by the word ‘estimated’, but almost all our measures of prevalence are estimates, so that shouldn’t be a problem. However, the lack of confidence interval worried me, so I checked the citation:

Molly JT Kibenge, Tokinori Iwamoto, Yingwei Wang, Alexandra Morton, Marcos G Godoy and Frederick SB Kibenge (2013) Whole-genome analysis of piscine reovirus (PRV) shows PRV represents a new genus in family Reoviridae and its genome segment S1 sequences group it into two separate sub-genotypes. Virology Journal 2013, 10:230 https://www.virologyj.com/content/10/1/230

The virus has been detected in marine-farmed Atlantic salmon in Chile [25,26]. There is anecdotal evidence that it is also present in farmed Atlantic salmon and wild Pacific salmon in British Columbia-Canada [27], where 75% of 300 farm salmon reportedly tested positive for PRV [27] but no sequence information was reported.

An estimate of 80% has been transformed into anecdotal evidence of 75%, but at least we have a denominator now (300 farms) so it is possible to calculate a confidence interval. However, I naturally wanted to verify whether the sampling was representative:

The grandparent

The citation for the evidence was:

Feinberg J: Science divided on fish virus found in Cultus Lake trout. https://www.theprogress.com/news/163480656.html.

This is a newspaper article from The Chilliwak Progress. The author is Jennifer Feinberg who “covers news, city hall, business, First Nations and environment.” She quotes:

“I’m not surprised that the virus sequence is present because we find it in 60 to 75 per cent of farmed salmon,” said Gary Marty from the Animal Health Centre in Abbotsford. “In Norway, they call PRV ‘ubiquitous’ because every time they sample fish, they find it.”

“Based on my knowledge of testing hundreds for the virus and not linking it to heart disease, I think that risk would be negligible,” Marty said.

For example he said 300 salmon farm fish were tested and although 75 per cent were positive for PRV, less than two per cent died of heart disease.

Distant relatives

A quick search showed that Gary Marty has presented the information elsewhere (Piscine Reovirus Information Sheet)

The samples were from 76 government audits of recently dead fish (n = 402) on the salmon farms and 137 fish that had been submitted directly from farm veterinarians, most of which were for health screening before transfer. The PRV prevalence was about 80% in all three groups of interest: audits with fish that died of unexplained heart disease (4 of 5 audits), audits with no fish that died of unexplained heart disease (56 of 71 audits), and direct submissions from salmon farms, none of which died of unexplained heart disease.

The citation for this was:

Marty, G.D., and J. Bidulka. 2013. Piscine reovirus (PRV) is common but unrelated to disease among farmed Atlantic salmon in British Columbia. Annual Meeting of the Fish Health Section of the American Fisheries Society. Port Townsend, Washington, USA. June 2013. Abstract.

A quick web search failed to find a copy of the abstract, and that is where I decided to give up.

Conclusion

Ultimately the source of the estimates appear to have a reasonable scientific foundation. However, in the process of shifting from peer-reviewed literature to citations for newspapers, fact sheets and non-peer-reviewed conference presentations, it is not clear if we are talking about the same evidence, and the reference population has become obscured. Originally, it was prevalence amongst unexplained mortalities, but this morphs into the entire farmed salmon population. The numerator and unit of interest is rather unstable as well (are we talking fish or audits?).

When we stand on the shoulders of giants, we should try to make sure that they themselves have a firm footing.