If Alex Kerr, ArtSci ’08, is unhappy with a course, he lets his professor know when he fills out teaching evaluation surveys, the University Survey of Student Assessment of Teaching (USAT) forms, distributed at the end of term.
“If I’m dissatisfied with a course, I write the specific sections of material or course issue, teaching style or whatever it was that bothered me,” he said. “I hate the idea that people will suffer through the same thing, and to me it’s better to offer suggestions for change than to tell everyone I know not to take the course.”
Still, Kerr said he’s not sure professors get the message because of how the questionnaires are designed.
“It’s problematic that they just have you check off the level to which you agree with prepared statements,” he said. “Very often my answers to those ‘check the box’ questions only reflect how much I like or dislike the teacher.”
Soon, students dissatisfied with their classroom instruction may have a new and improved way to make their voices heard.
This term, Kerr is one of 8,400 students who will fill out an extra teaching evaluation form as well as the current USAT survey.
The new questionnaires are a pilot project introduced by the t Committee to ister the Collective Agreement (JCAA) teaching assessment committee.
The committee was started in the spring of 2005 by the University and Queen’s University Faculty Association (QUFA).
John Holmes, QUFA president, said the committee was created because of concern about yearly faculty evaluations, which determine promotions, bonuses and contract renewal for University teaching staff.
“Each year, the head of department does a review of faculty after each member submits a report,” he said. “There was concern from our point of view that USAT was the only mechanism being used in these evaluations.”
Patrick Deane, vice principal (academic), said professors are evaluated annually on research, teaching and service to the community.
USAT surveys are the primary way successful teaching is evaluated.
“Right now USAT forms play a primary role in teaching evaluation,” he said. “In promotions and tenure, very, very serious consideration is given to USAT forms.”
Deane said some Canadian universities use evaluation dossiers consisting of USAT forms, colleague evaluations and other student .
“For some universities, the larger dossier is mandatory, but we are fairly representative of most Canadian schools,” he said. “Still, I would like to see the system augmented and extended into an entire evaluation dossier. The sooner the better.”
The JCAA committee is co-chaired by professor Caroline Baillie and Joy Mighty, director at the Centre for Teaching and Learning.
Baillie said although she can’t give specific information about other initiatives, new questionnaires are only part of larger overall changes to professor evaluations.
“This project is the first of many. We’re basing it on the idea of a matrix, with several inputs from different sources,” she said, adding that the committee wants departmental teaching evaluations to include more than just student input.
“Students can give about their own experience, but they aren’t qualified to objectively evaluate teaching and curriculum,” she said.
Baillie said the new forms were designed using the Course Experience Questionnaire, an international database for universities to create student evaluations of educational experiences.
The current USAT forms consist of four university-wide questions and up to six default department-wide questions. Professors can then choose up to 10 additional questions from a list of 25 sections.
She said the questionnaire is meant to dig deeper into the motivations behind student opinion.
“Someone might do a great song-and-dance routine, and be ranked very highly by students because of it,” she said. “But that doesn’t mean students are learning, which is the most important thing.
The forms will be tested by 80 volunteer faculty this term, although every class will still fill out traditional USAT evaluations.
Baillie said the test pilot forms will take longer to process and return to professors.
“We haven’t worked out how long processing the data will take, but it will certainly take longer to compile all the written comments,” she said. “Still, that more personal will be invaluable to professors.”
Baillie said faculty take the questionnaire evaluations very seriously.
“This is a delicate issue that is incredibly important to teachers here, and has a big influence on their careers,” she said.
Baillie said she predicts the committee will present their findings on the success of the pilot project to the University and QUFA in May or June.
Charles Sumbler, AMS academic affairs commissioner, represents undergraduate students on the JCAA committee.
Sumbler said he worries student dissatisfaction with a course might be misinterpreted as frustration with a professor.
“This dissatisfaction is coming across as being related to the professor rather than the course itself,” he said. “I don’t think these forms accurately reflect student impressions of their professors.” Sumbler said the pilot questionnaires consist of 30 questions designed to get more specific information about faculty performance.
“The forms look at whether professors are meeting student expectations in more precise areas than USAT evaluates,” he said. “Not just overall whether a professor is effective, but more specific strengths and weaknesses.”
Sumbler said he hopes this will allow professors to make effective changes.
“The goal is that faculty can better grow from more specific , and find mechanisms to improve in very particular areas where students might want to see changes,” he said.
The forms also ask students if the course is mandatory or elective, and provide space for personal comments after each question.
Sumbler said the changes should give a better sense of what students expected from the course, and whether these expectations were met.
Professor Kim Nossall, head of the politics department, is participating in the trial.
Nossall said although he’s satisfied with the current surveys, there is room for improvement. “One persistent complaint is there’s no way of assessing how a student evaluation links up to performance in the class,” he said. “Is a student who evaluates a professor as absolutely bad getting absolutely bad marks as well?”
Nossall said he tries to respond to student whenever possible.
“There are certain things you can do, and things you can’t,” he said. “Complaints about a textbook, sure, but comments about the essence of yourself as an individual are what you can’t change.”
Nossall added that he doesn’t think students know how much their matters.
“I do wish students would take the evaluations more seriously as a means to communicate with their professors,” he said.
Students deciding on classes can access some professor ratings on the academic affairs section of the AMS website.
Professors must agree to publicize the ratings when customizing their USAT forms.
According to the University registrar website, in 2005-06, 52 per cent of professors authorized the release of their evaluations. The website says that the figure decreased from 96 per cent in 1995-96 because “the [form] default was switched from release-to-students to withhold-from-students if the instructor did not provide specific instructions.” Sumbler said although ratings will continue to be available online [FOLLOWING THE NEW SYSTEM’S INTRODUCTION?], he didn’t think they were very helpful.
“I have problems with USAT, clearly, and I’m not sure how useful the information would be for students in of course selection,” he said. “They don’t offer specific information on professors, and they don’t dig very deep.”
Sumbler said he hopes students will benefit from teaching evaluations that are more reflective of what goes on in the classroom.
“We want to put things in a broad perspective of quality teaching, and not just focus on a number or score communicating how a professor is doing,” he said. “There is a lot more to it, and we hope these changes reflect that.”
All final editorial decisions are made by the Editor(s) in Chief and/or the Managing Editor. Authors should not be ed, targeted, or harassed under any circumstances. If you have any grievances with this article, please direct your comments to [email protected].