Our brains respond differently to a painting if we are told it is genuine according to a study by academics at Oxford University.
Fourteen participants were placed in a brain scanner and shown images of works by 'Rembrandt' -- some were genuine, others were convincing imitations painted by different artists. Neither the participants nor their brain signals could distinguish between genuine and fake paintings. However, advice about whether or not an artwork is authentic alters the brain's response; this advice is equally effective, regardless of whether the artwork is genuine or not.
I wonder if academics’ brains would undergo the same process if told that an article was published in Nature (or whatever the ‘top’ journal in your discipline might be) as opposed to being posted on some random website or published in a low ranking journal (however defined). For the sake of argument I am assuming that the academics would only be looking at work which was good (the imitation Rembrandts were good paintings by all accounts.) I‘m not a scientist, but I know that Nature is good – at least that is where to publish if you want Radio 4 to notice your work.
Last week I attended a workshop for Islamic Studies PhD students in my capacity as Acting Academic for the HEA Network. A business academic told me about the Association for Business Schools’ journal guide. Each journal has been classified as 1 to 4 star (in parallel to the UK’s Research Evaluation Framework). It isn’t my place to comment on the policies of disciplines in which I do not have expertise, but this strikes me as a highly transparent way of assessing the quality of research—if you publish in a 4 star journal the article must be good, if a 3 star not so good etc. etc. No arguments- the publication is the judge.
However this puts some topics off limits to academics wanting to publish in the top journals. I understand the top business journals publish little about Islamic Finance—if this is your topic then you cannot publish in the top journals. A humanities academic from an Eastern European country recently informed me that research impact in his country involved publishing in ‘top journals’, in short journals written in English. Linguistic issues aside, one of the consequences is that he and his fellow academics have to write about the sorts of topic Anglophones (or more accurately Americans) think are important —therefore fewer academics are writing about their own country – they write about the USA.
In science PLOS One is an open access venue which is unrestricted by topic or by what editors think would be expedient to publish (important and popular not being the same thing).
This is one of the great advantages of the internet—we can have peer review, open access research which is not restricted to certain topics.
Under the current system the journal an article is published in is our equivalent of a genuine Rembrandt. It would be interesting if all inputs to the REF had to be submitted as plain text files to see if the efforts of the Rembrandts and artists of lesser reputation can really be distinguished. Brain responses might be the fairest method of evaluation we have available to us.