If we were to rank critics...

Benjamin Disraeli once said, \"How much easier it is to be critical, than to be correct\" and the recent interchange QS has enjoyed with one of its foremost critics in the pages of The Australian brings forth an idea. Not an idea that I would expect to see-through but an idea that represents an interesting framework for answering the question on the value of critics.

Critics are valuable, not only as an essential call to account, but also because they give their subjects oxygen. In the university rankings space, it is because there is no ultimate solution and no correct answer that subjective choice emerges as to which is more suitable to any given purpose. Critics give voice to all exercises in the category, encourage people to visit them and make up their own mind, their comments are posted in mainstream media sources, usually depicted as independent and, perhaps perversely, they contribute to the popularity of the rankings of which they are critical.

But what if someone were to develop a ranking of critics - not just rankings critics, but literature, art, car and restaurant critics. What would the hypothetical indicators be? Perhaps they would fall under the following headline categories:

BACKGROUND

- What are the critics' qualifications? Does it add value for an art critic to be a trained artist, a literature reviewer to be a writer or a car critic to be a mechanic? Or perhaps it is more oblique than that, substitute artist/writer/mechanic for collector/reader/driver?

Rankings Example: Ellen Hazelkorn has conducted research at DIT on the effects of rankings on higher education policy and is a published author on the subject

TRACK-RECORD

- How long have they been a critic, have previous reviews been balanced and informative, have they taken the same approach to different subjects in a given category, have their previous reviews been, on average negative or positive?

INDEPENDENCE

- Heston Blumenthal might be a fantastic restaurant critic, but even if he wrote a perfectly balanced review of a rival's restaurant, could it be taken at face value? The best critics ought to avoid official alignment with any particular subject in the field they choose to scrutinize as even if it doesn't affect their independence, it could be perceived to.

Rankings Example: Following a long career at the World Bank, Jamil Salmi is a highly respected consultant on higher education policy and whilst he covers a much broader perspective on higher education is occasionally drawn to comment on rankings.

BALANCE

- If, over time, a book reviewer praises every new Stephen King novel but assassinates every new James Herbert, despite genre, plot and concept parallels a degree of favouritism would begin to be perceived. The value of future reviews of these authors would quickly fade, with the reviewer becoming perceived as pursuing some form of personal agenda, even if there is none. Edgar Allan Poe once said \"In criticism, I will be bold, and as sternly, absolutely just with friend and foe. From this purpose nothing shall turn me.\" Such is the central tenet to which a respectable critic must adhere.

Rankings example: Andrejs Rauhvargers has compiled two reports on rankings for the European Universities Association, these are fairly comprehensive and clinical analyses of the key strengths and weaknesses of all the major existing global rankings

EVIDENCE-BASED

- Jeremy Clarkson is an influential UK-based car-critic. Largely because he is, or at least strives to be, funny and entertaining, that he makes sweeping generalisations and is a little outrageous. He frequently gets into trouble for saying something he shouldn't and in general is a polarising character - people tend to think he's great or awful. However, whenever he claims a car is great or terrible, it is generally backed up with fairly detailed evidence as to why that is the case. He might say, \"this is the worst car ever made\", but he will go on to qualify the statement with evidence.

Rankings Example: Richard Holmes on his University Rankings Watch blog finds the time to go into more depth and detail than any other rankings commentator, over the years he has been broadly critical to a similar degree of all major rankings.

POPULARITY

- Not much point in a critic if nobody reads their stuff, no matter how well it measures up to the above aspects. Very few people worry about bad reviews in publications that very few people read.

Rankings Example: Simon Marginson and Philip Altbach, who both serve in advisory capacity to THE and ARWU are probably the most frequently referred to and quoted critics of rankings.

So if a group of indicators were compiled in these categories to rank critics of rankings where would the principal protagonists sit? Criticism, however valid, should always be taken for what it is - someone else's viewpoint on the matter - ultimately the diner, collector, driver, reader has to form their own view.