Back to top

CAUT Bulletin Archives

December 2005

What Counts Cannot Be Counted

Stephen Wexler
University is done for its own sake, not for the sake of something else. It has no product. One point of university is to stand against the idea that everything is done for money. This makes university different from much of the rest of the world.
Metricization is very popular in business and government right now. Everything has to be counted. The motto of the 21st century seems to be: if you can’t count it, it doesn’t count. Because the University of British Columbia wants to look good to the government and the public it has bought into this attitude. We want to explain ourselves and prove we are doing something worthwhile. We want to defend ourselves against the charge that we get five months off in the summer and only work six hours a week.

Metricization of our research is supposed to enable us to account to the public for the money spent on universities and increase our productivity. It is said that when lawyers metricized, they discovered that it increased their productivity. Why would it not have the same result at the law school? What lawyers increased by metricizing were billable hours. Is that what we are trying to do? Are we not, in the name of improving and defending the university, committing a travesty of university values? As an American general said during the Vietnam War, in order to save the village, they had to destroy it.

The first thing that is wrong with the metricization of research to increase our productivity is that it assumes we are not now being productive enough. My research seems quite productive to me. I hope I will be able to add a grain of sand to the research on Aristotle. I try as hard as I can not to metricize my research. I’m doing the best I can and I do not feel that any of my colleagues are doing less than a fair share. Some concentrate on teaching and do not do much research. Others do more research and the type of research varies. There is no more and less, which is what numbers inevitably indicate.

Could we be more productive either as individuals or as a faculty? Probably. There might be some marginal gains that could be made, but metricization will not increase research, it will only increase the number put on whatever is deemed to be the output of research.

There is no measure of research. The idea that research can be measured is all wrong. Everyone would agree, for instance, that reading a book is research, but what is the number to be assigned to reading a book? Does one book equal another or are some books worth more than others? Do we metricize on the number of pages or is there some other test for the worth of a book? If there is another test is it the number of other people reading the book? Does one get a higher number for reading a book everyone else is reading or does one, perhaps, get a higher number still for reading a book no one else has read? What number should be assigned to reading a book for the second or third time? Does reading a book again indicate that you didn’t understand it the first time and hence are no good at your job, inefficient, unproductive?

Of course, our reading is not the research we will be metricizing. We will metricize our writing, our output. One problem with metricizing output is that all research does not result in an output. Much research leads down blind alleys. Ideas that seem like they might go somewhere turn out not to go anywhere. What numbers will we give to reading a useless book or pursuing a research strategy that does not succeed? And of course, one never knows when one starts whether a line of research will be “productive” or not. A new idea can lead nowhere or to the best work of one’s career.

How are numbers to be assigned to thinking about things, to having ideas or discussing them with one’s colleagues? Dr. Michael Smith, the Nobel Laureate who was a professor at UBC and for whom a new $30 million UBC laboratory is named, said the research that led to his Nobel Prize began in a conversation with another scientist over a cup of coffee. What numerical value is to be assigned to that conversation and when is it to be assigned? At the time it takes place or when it results in the Nobel Prize? Suppose it does not result in a Nobel Prize. Does it get a lower number? Does it get any number at all?

The idea that numbers can be assigned to research is ridiculous, but more important, it is pernicious. Assigning numbers leads to the mistaken view that more is better. Metricizing research is directed not just at faculty research but at student research as well. In my courses, I require students to do 10 to 12-page research papers, which they then read to me and the other students. Many professors require far more pages but allow students to slip the paper in under the door. It takes 45 minutes to read a 10 to 12-page paper out loud and discuss it. If I assigned longer papers, I could not have them read aloud. I do not wish to say that the way I do things is “right” and the way others do them is “wrong,” but nor do I wish to be told that the way I do them is “wrong” because the number of pages I require is too low.

More pages do not mean more research for either students or faculty. Since the most convenient measure of research is the number of pages in published works, we use it, but no one takes this measure seriously. That is not to say there can be no evaluation of academic research. The quantity and value of research is, and must be, evaluated before a professor is hired or granted tenure. This process requires subtlety and intelligence. It sometimes comes down to counting published works, but everyone realizes this is arbitrary.

Once a professor has tenure, academic freedom requires that there be no further arbitrary assessment. A professor’s research should always be subjected to intelligent and subtle critique and evaluation, but a tenured professor must be able to work free from concerns about whether the work will be seen as important by other scholars. You cannot write in order to be published. If others see your work as valuable that is gravy, but professors must be allowed to go down garden paths.

The value of research is the process, not the product. Processes cannot be metricized, only results. Universities are not like businesses. They do not have a product. Attempting to increase their productivity is misguided at root. The purpose of being at a university is precisely to not be subject to the requirements of productivity.

Metricizing research in order to increase productivity is analogous to metricizing love to increase its productivity. Aristotle distinguished between doing something for its own sake and doing something for the sake of something else. There is no product from love or art or sport and there is no product from university research. One does it for its own sake. That is the point of being a scholar. Our English word “scholar” comes from the Greek skholazein, which means to do things for their own sake. Add what is called “an alpha-privative of negation” and you get askholia, which means “business.”

University and the research and teaching which go on there are valuable for their own sake, not for what they produce. Like love, art or sport, university and research can become businesses, but that is not something to seek. It is something to deplore and avoid. The purpose of university is being at one. Students are already too prone to think the purpose of university is the degree or qualification they receive at the end. This is profoundly and tragically wrong and metricizing can only increase this mistake.

The UBC Law Faculty is looking for new ways to metricize in order to compete with science for university funds. We are trying to show that our research is as valuable as theirs. The Arts Faculty has the same problem to an even higher degree. Metricization might almost be said to fit scientific research because so much of science is about measuring things. For lack of any better measure, many scientists have come to treat number of pages published, number of times an article is cited, amount of grants received and number of patents issued as a valid measure of their work. All scientists, even those who succeed at getting grants and patents, know this is not a proper measure of science, but because much scientific research is very expensive, scientists must seek grants; scientific research has become grant-dependent. Because they and the granting agencies are familiar with quantification, many scientists have accepted metricization as inevitable. They have allowed their disciplines to be taken over by the world of business.

To apply metricization outside of science makes even less sense. A scientist tries to discover new things and in some weird sense one might be able to say that any new discovery is worth as much as any other. From there, one might jump to saying that the number of published pages describing a discovery is somehow a measure of something. Yet it still is not clear why more is better. The ultimate products of scientific research are not long. E=MC2, Boyle’s Law and The Periodic Table of the Elements could all be published on one page.

And even if we were to allow that scientific research could be metricized, outside of science people are not trying to discover new things. They are thinking again about things that have been thought about many times before. In no meaningful sense can an idea about Homer be said to be equal to an idea about Dickens and neither could be said to be equal to an idea about the Wills Act or negligence.

How does one value scientific research against other university research? Some scientists are already asking this question. We in Law should ally ourselves with them and with those in Arts to resist the “scientification” of the university and the metricization of research. Our eventual goal should be to get all the scientists to join us, and jointly we should attempt to explain that a university is not a business and should not be treated as one.

The prime thing for the sake of which things are done is of course money and when it comes to money, metricization does make a kind of sense. With money, there is a bottom line. There is no bottom line at a university. University is done for its own sake, not for the sake of something else. It has no product. This makes university different from much of the rest of the world. People used to put university down as an “ivory tower” but that is not a put-down of university. University is an ivory tower. We should try to remember that.

The purpose of university is precisely to be different from the rest of the world. One point of university is to stand against the idea that everything is done for money. Metricizing university research is a step on the road to destroying the values associated with universities, not a step on the road to preserving them.

Stephen Wexler is a law professor at the University of British Columbia. He is in his 35th and antepenultimate year of teaching. He can be reached at 604-822-2194 or 604-228-8953. He does not do e-mail and says “I do not do e-mail because what I do as an intellectual requires me to read things very carefully and think about them very hard. My mind is like a samurai’s sword. If I read e-mail, I will dull my blade. I am very selective about what I read and would much rather talk to people than read. ‘Chatting’ on-line is not what I mean by talking.”A version of this essay was originally published in the November 2004 issue of Faculty Focus, the newsletter of the Faculty Association of the University of British Columbia.

The views expressed are those of the author and not necessarily those of CAUT.