Industry-funded research IFfy?

In his column Bad Science, in The Guardian on Saturday 14 February, Ben Goldacre drew attention to an article in the British Medical Journal by Tom Jefferson et al in which the observation was reported that...

"Publication in prestigious journals is associated with partial or total industry funding, and this association is not explained by study quality or size."

The Impact Factor (IF) of the journals in which research funded by the public sector was published averaged 3.74 and the IF of the journals in which industry-funded research was published averaged 8.78. As Impact Factors go, that is a substantial difference. And, as Jefferson et al indicate, there was no discernable difference in terms of quality, methodological rigour, sample size, et cetera between the articles in question. Goldacre doesn't have an explanation. The suggestion is given in his column (he admits it is an "unkind suggestion") that it may have to do with journals' interest in advertisements and reprint orders – which can indeed be massive – from the very same industry that funds the research these journals publish. He doesn't say it, but this could mean, of course, that the journals accept articles based on research funded by industry, particularly the pharmaceutical industry, more readily than articles based on publicly-funded research.

I don't have an explanation for the phenomenon, either, but I doubt that journals accept industry-funded articles more easily than public sector articles. For a start, most publishers do not have in-house Editors-in-Chief who decide what's published and what not. That doesn't mean the publishers cannot have an influence on those Editors, but often it is already so difficult for them to get Editors to comply with everyday, sensible wishes, that I think this would be rather far-fetched. For publishers that do have in-house Editors-in-Chief, such influence may be more easily exerted.

A hypothesis I can imagine, however, is different and less sinister, although also to do with the massive numbers of reprints disseminated by the pharmaceutical industry. But this hypothesis would reverse cause and effect. Might it be that because of the wide dissemination, availability, and visibility of these reprints, the industry-funded articles are cited more often? After all, we know that articles are not only cited because they are the most appropriate ones, but also simply because they are the appropriate ones known to the author. (Sort of like when you ask a 'randomer' – a word I learnt from my 18-year old daughter and that I guess means random person – for the best restaurant in town, you are likely to get the best restaurant he or she knows, which is not necessarily the best restaurant in town). If articles based on industry-funded research are cited more often, the journals in which they appear get a higher Impact Factor.

If this hypothesis holds water, it would mean that wide availability is one of the important factors – with dissemination and visibility, and of course relevance – for being cited. In other words, could the results described in the BMJ article constitute evidence that open access could have a similar effect on Impact Factors as that – still hypothetically – caused by the massive numbers of reprints that the pharmaceutical industry purchases and disseminates?

Food for further study, I would think.

Jan Velterop

Deploring or exploring?

When Homo sapiens was still in the early stages of his evolutionary development, he hadn't yet figured out many other uses for water than to drink it. And perhaps to bath and swim in it. This is conjecture, of course, but the earliest evidence of the use of boats, or even just rafts, dates from much later than the emergence of Homo sapiens, so assuming that he was just using water to drink may be an acceptable point of departure for my story.

Water is one of the most abundant resources on earth, but if you're just using it to drink, you don't quite get much of its potential out of it. When people invented rafts, and developed boats – probably in the form of dug-out logs – a whole new world, literally, opened up to them. They all of a sudden didn’t have to see expanses of water as impediments to getting to the other side, and once navigation was thus discovered, waterways and seas became the most important transportation routes upon eventually empires were built. The rest is history, to use a cliché.

There is something similar going on with the way we use information. The image that I have in mind is that there are virtually oceans of information available to humans, but that the only use we make of that information is ‘by the drink’ – by reading articles or bits of articles. That way, the knowledge contained in the ever growing seas of information (just think of the amounts of information coming out of, say, microarray experiments), is unlikely to come out in full. There remains an enormous amount of “unknown knowns” (apologies for using a Rumsfeldism) if we do not find a way to do more with information than read articles and books, or consult databases. We have to develop ways of extracting knowledge out of large amounts of information. Thousands of papers, and thousands of database entries. Or hundreds of thousands. We can’t read those. We have to invent the equivalents of rafts and boats to navigate information. And still read, but manageable amounts (after all, we still drink, too).

In whatever information navigation we already do, we stay very close to the coast, and only to the coasts we know. We search. And we pretend that we are navigating the vast expanse of knowledge that search capabilities on the internet have opened up. But are we? Is searching not a retrograde step in terms of knowledge discovery? Aren’t we inclined to search for knowledge and relations between bits of information we already know to exist? And so foster more homophily in the process than before, when large-scale search wasn’t yet possible? And stay in our knowledge comfort-zone. Look for confirmation rather than for falsification. We should give chance more of a chance. Serendipitous discoveries are, after all, the 'stuff' of which breakthroughs are made.

Some people deplore the fact that more and more information becomes available. They talk of information overload or overabundance. And if the only thing you can imagine doing with it is read (‘drink’), then you may have reason to be negative about it. If you think like this you may seek solutions in selection, in limiting access, in having the choices made for you. But if you can imagine truly navigating the ever growing seas of information, you will not deplore the abundance, but instead, start exploring it.

Jan Velterop