Arbitration award concerning Faculty Course Survey

Dear colleagues,

We are writing further to the recent arbitration award concerning Faculty Course Survey (FCS). You can view the  (PDF file) full arbitration award online (external link) .

Our goal is to ensure that faculty are fairly and accurately assessed, as it’s of critical importance for the success of our faculty and for the mission of the university. We want to elicit useful information free from bias. We’re committed to implementing the results of the arbitration decision and to working with the RFA to solidify processes that are transparent.

Summary and next steps

Below is a summary of the decisions from the arbitration award, as well as next steps the university will take to put them into action, effective for the Fall 2018 term.

1. “The collective agreement is to be amended to ensure that FCS results are not used to measure teaching effectiveness for promotion or tenure. A necessary corollary of this direction is that the numerical weighting system in the FCS be replaced with an alphabetical one. Question 15 should be struck.”

1. “The parties are directed to meet and agree upon an appropriate, user-friendly, intelligible and easily accessible mode of presentation of FCS data in the form of a frequency distribution together with response rates.”

1. “The parties are also directed to ensure that Deans and DECs, FTCs, FPCs and any others charged with evaluating faculty are educated – a process already set out in the collective agreement – in inherent and systemic biases in SETs so that FCS results can be considered in light of their actual and inherent limitations and with necessary context.”

1. “The parties are directed to meet and try to agree upon governing guidelines to be incorporated into the collective agreement.”

1. “The parties are further directed to establish a properly resourced joint committee with hard time lines to consider the current FCS and the possible revision/addition to the questions that are asked, the provision and use of comments, the process and methodology for FCS administration including paper, online, in class, scheduled time, and data tracking of response rates.”

1. “Until and unless the parties agree on a substitution for the online system for non-online courses, it is to be discontinued for probationary faculty.”

FAQ

Set out below are questions and answers to issues that we expect will arise for faculty members as they proceed with year-end assessments, teaching awards, tenure reviews and promotion applications. Currently, there are differences of opinion between the university administration and the RFA with respect to the interpretation and implementation of the award. We are committed to further discussion with the RFA to resolve these differences. Should there be a change as a result of those discussions, the university administration will provide you with further updates and clear direction. If you have any questions about this communication, please contact the Office of Vice-Provost, Faculty Affairs.

1. What can’t we rely upon FCS to measure?

We cannot rely on Question 15 results nor on certain previous results, conclusions drawn or Teaching Awards which may have, in part, been granted based upon Question 15.

We also cannot rely on any FCS results as evidence of teaching effectiveness. In assessments any conclusions on teaching effectiveness for the purposes of meeting the required standard for tenure or promotion instead ought to be based on a range of data inputs from peer assessments, feedback from other sources and from a comprehensive Teaching Dossier. In the words of the arbitrator:

It is important for those on committees also to be aware that there must be some scrutiny and consideration of the possibility of bias influencing peer and other assessments.

Finally, we cannot rely on averages and comparison with departmental averages and must instead consider and present results using frequency distribution.

2. For what purposes can we rely upon FCS?

The Arbitrator was clear that FCS results provide information about the student experience: He wrote that:

With the proper contextualization, FCS results may be used to assess the following aspects of teaching:

(a)  Student Learning Experience: FCS ratings reflect the way students feel about their learning. This includes students' assessment of the manner in which they received course content from their teacher through oral instruction and course materials as well as the difficulty or ease of the class. It also includes the students’ opinions on engagement with, and ability to apply new concepts within the learning objectives of the course.

(b)  Student Engagement with the Subject Matter: FCS ratings may provide a measure of students' interest in the course, engagement in the course and whether learning concepts were effectively communicated to them through the use of examples.

(c)  Coverage of Course Materials: FCS provides an opportunity for students to report on whether class discussions, assignments, and one-on-one interactions with their teachers adequately addressed the topics set out in the course syllabus.

(d)  Maintenance of a Fair and Respectful Learning Environment:  FCS ratings allow students to report on whether the classroom environment was one in which the contributions of its diverse students were valued. It is a faculty member's obligation to make every attempt to create an equitable, diverse and inclusive atmosphere of mutual respect where students can learn.

(e)  Availability of Faculty: FCS ratings allow students to report satisfaction with their ability to contact and receive timely responses from their teacher that enhanced their learning.

(f)   Contribution of Faculty Feedback to the Student Learning Experience: FCS tool provides students with an opportunity to report on whether they are receiving timely and comprehensive teacher feedback so they know, as early as possible, whether they are meeting the expectations of the course. The FCS measures students' perception of the quality of their teachers' feedback on their progress.

(g)  Impact of Teaching Methodology on the Student Learning Experience: FCS allows students to assess whether the types of assignments, course materials, and in class learning tools improved their understanding of the course material and allowed them to demonstrate their learning.

3. What is meant by the proper contextualization of FCS results?

In the presentation, review and consideration of FCS results, there must be an active and thoughtful consideration as to whether or not the personal characteristics of the faculty member such as race, gender, accent, age or “attractiveness” may impact on the results either in a positive or negative manner. Consideration ought also to be given as to the impact of grade expectations, and course characteristics including elective versus required, class size, graduate or undergraduate, quantitative versus humanities and traditional teaching versus innovative.  

4. How do I demonstrate teaching capacity for, achievements in and commitment to teaching under the collective agreement if I cannot rely upon my FCS results?  

We understand that for those of you who are seeking a transfer to tenure or a promotion and/or are on a committee undertaking a review, the arbitration decision may raise a number of questions and concerns.

Traditionally, faculty members have demonstrated capacity for, achievements in and commitment to teaching by relying on the results of Question 15, as well as comparisons to averages and/or on teaching awards which may have been granted based in part upon Question 15 results. Since we can no longer rely on averages or on Question 15, faculty members should ensure that they provide a comprehensive teaching dossier that includes a description and record of your major teaching accomplishments and strengths in a manner that conveys the scope and quality of your teaching. The key areas to comment on are outlined in Article 5A./5B.8.C., as well as Article 10.12.A of the RFA Collective Agreement. Some suggestions of how to build this dossier include the following:

• As the arbitrator mentions in his award, “FCS results can continue to be used in tenure and promotion, when the results are presented as frequency distributions.” 
• You can also provide the evaluation committees with information about how you have overcome any weaknesses in your teaching as identified by previous assessments/reviews.
• You may also utilize peer assessments and other sources including any external teaching assessments or assistance you have sought or training, education, teaching/pedagogy focused conference attendance or other activities undertaken to demonstrate a commitment to teaching.  
• Pre-tenure faculty members may wish to exercise the option of requesting an additional in-class teaching assessment in any given semester. If electing to do so, they must give the chair of their DEC at least two weeks’ notice of their request before the last two weeks of the semester so the assessment can be scheduled within the allowable timeframe.

5. What if my Year End Assessments refer to FCS results including, for example, results on Question 15?

The university will advise evaluation committee members to ignore any references to Question 15 and/or FCS averages in any past reports or year-end assessments.

We are mindful of the impact that this will have on committees and individual faculty members as we work to operationalize the recommendations from the arbitrator and we appreciate your patience as we work on implementing these changes.