By Rick Gooding, Sandra Hoenle, Peter Little, Catherine Christie, George Davison, Kelly MacFarlane, Ronda Ward & Leslie Jermyn

Our post-secondary education system is moving to ever greater reli­ance on student opinion surveys (mislabelled as teaching or course evaluations) to judge, reward and punish instructors. We are all vulnerable, but contract academics are particularly at risk of having their livelihoods depend on these flawed instruments of assessment. We need to address the growing impact of student opinions on our working lives through bargaining, public education on campus, and by rejecting or tempering the use of these scores in peer-review processes like hiring and promotion.
     
A comparison with the medical profession may help to illuminate the critical flaws inherent in the use of opinion surveys to measure teaching ability. Imagine if physicians were judged by opinion surveys taken as their patients exited their waiting rooms. Then imagine if their employment and their pay depended on high scores on these surveys. What kind of doctors would be well paid with robust job security?
     
If you consider what makes a phy­sician “effective,” most of what they do would not be captured in patients’ opinions upon leaving their offices the first time, or, as is typical for instructors, two-thirds of the way through a course of treatment. A good physician takes time to listen to us so they can determine what ails us and what we’re doing to make our problem better or worse.
     
A good physician uses all the tools of assessment at her disposal to develop an accurate diagnosis. She pokes and prods, physically and metaphorically, even when it makes us uncomfortable. We may leave feeling negatively judged and found wanting; we may leave feeling like our dignity has been compromised. If we were asked at that moment if we liked the doctor or enjoyed the experience, we would be hard pressed to be laudatory or even neutral. Our assessment would not be positive.
     
Physicians who worked primarily toward high scores, on the other hand, would not ask us to do things we didn’t like; would not chastise us for failing to change harmful behaviours; would not listen for the real problem but would try to give us what we seemed to want rather than what we need. We would leave satisfied, but not necessarily on the road to better health. We would give ineffective physicians high scores on the exit surveys and only realize they were “useless” when our health failed to improve.
     
Good teachers sometimes make their students uncomfortable: we poke, prod and assess, and we may ask our students to reconsider unproductive and unhealthy attitudes and behaviours. We ask them to alter habits of mind and to think critically about ideas that are not always comfortable. Students who are thoroughly challenged and exercised by their teachers may find this disconcerting and they may not yet have the long term perspective, at the conclusion of the course, to know whether their instructor’s overall impact was positive or negative.
     
Students who struggle with meeting expectations and who receive lower grades are even less likely to have a balanced assessment of their teachers’ abilities. The end result is that really good teachers may receive low scores on student opinion surveys. This negatively impacts the teacher’s ability to get hired, promoted or rewarded.
     
This is not to suggest teachers who score well on opinion surveys are not good. But the evaluation instrument is flawed and is not designed to capture elements of effective teaching which may not be evident in the short term or to students struggling academically. Student opinion surveys are snapshots of students’ emotional dispositions toward courses and instructors taken before the full impact of the material or method of delivery can be known.
     
These are then fairly arbitrary measures of something other than teaching ability and should not stand as determinants of careers, salaries or discipline for tenure-track, tenured or contract faculty. Where our colleagues are being assessed solely on teaching, divorced from service and research contributions, as are many contract academics, the danger is multiplied.
     
Academic staff associations must give critical attention to the use of student teaching assessments for all faculty, and pay close attention to the impact student opinions have on contract academic job security. We must encourage colleagues and administrators who evaluate teaching for hiring and promotion to move beyond reliance on student opinions to more sophisticated and appropriate measures of effective teaching such as in-class peer review and polling of students well after the conclusion of the course.
     
The most effective encouragement is strong contract language. In an econo­mic climate where wage gains are proving difficult, protection from arbitrary performance assessment is a cost-neutral goal that can benefit all members.

---------------------------------------------------------------
Rick Gooding, Sandra Hoenle, Peter Little, Catherine Christie, George Davison, Kelly MacFarlane, Ronda Ward and Leslie Jermyn are members of CAUT’s contract academic staff committee.

The views expressed are those of the authors and not necessarily CAUT.

Comment
CAUT welcomes articles between 800 and 1,500 words on contemporary issues directly related to post-secondary education. Articles should not deal with personal grievance cases nor with purely local issues. They should not be libellous or defamatory, abusive of individuals or groups, and should not make unsubstantiated allegations. They should be objective and on a political rather than a personal subject. A commentary is an opinion and not a “life story.” First person is not normally used. Articles may be in English or French, but will not be translated. Publication is at the sole discretion of CAUT. Commentary authors will be contacted only if their articles are accepted for publication. Commentary submissions should be sent to (Liza Duhaime).