Procedures Governing the Learner Assessment of Clinical Teaching and Supervision

● Related Documents: Policy for the assessment of Clinical Teaching and Supervision

● Owner: School of Medicine Council

● Approval Date: 2023

These Procedures set out the process for the collection and amalgamation of results of Learner assessments of Clinical Faculty.  

All capitalized terms set out in these Procedures that are not otherwise defined herein, have the definitions set out in the Learner Assessment of Clinical Teaching and Supervision Policy (“Policy”).

II. Responsible Office

The Clinical Faculty Affairs Section shall develop, maintain, and implement the Clinical Faculty Learner assessment framework for the School.  .

The Clinical Faculty Affairs Section shall seek regular feedback from all relevant Departmental leaders, Divisional leadership, Learners, and Clinical Faculty on the Clinical Faculty Learner assessment framework.

The Clinical Faculty Affairs Section will approve and issue a common package  of teaching and supervision Learner assessment  forms (each an “Assessment Form”) to be  utilized within the School for all teaching and supervision contexts including but not limited to the following:

a. Ward

b. Outpatient Clinic

c. Emergency Department

d. Non-surgical procedures

e. Surgical procedures

f. Resuscitation

g. Sedation and/or anesthesia

h. Home visit

i. Small group teaching/session

j. Large group teaching/session

k. Simulation

l. Other

No department or program shall develop or deploy a Clinical Faculty Learner assessment form to be completed by Learners independently of the Clinical Faculty Affairs Section.

III. Collecting Teaching and Supervision Learner Assessments

Learner assessments of Clinical Faculty teaching and supervision (“Assessments”) will be triggered in three ways:

a. Whenever a Learner requests an assessment from a Clinical Faculty  they will be prompted to complete an Assessment Form for the Clinical Faculty. 

b. While working on a clinical rotation Learners may elect to complete an Assessment Form about the clinical supervisor in an on-demand fashion.

c. Learners may be assigned to complete an Assessment Form based upon their participation in a learning activity

Learners will be expected to complete at least one (1) Assessment Form per week.

All Assessment Forms shall be submitted by Learners to the Clinical Faculty Affairs Section via the Learning Assessment Management System (LAMS) and/or email to their rotation supervisor.

Screening Assessment Forms

A. The Clinical Faculty Affairs Section shall undertake an initial review of all submitted Assessment Forms and will categorize each Assessment Form as follows:   

a. Cleared

i. Assessments without concerning Clinical Faculty behavior or inappropriate feedback.

ii. Assessment becomes part of Clinical Faculty record.

iii. Aggregated assessment reports will be generated annually.

iv. Once minimum of three separate Learners from either UGME or PGME have completed assessments on that Clinical Faculty within a given context (Appendix A) those assessments will be distributed to individual Clinical Faculty.

b. Concerning Clinical Faculty behaviour

i. Clinical Faculty Affairs will inform the Department Head (or delegate) of the Department where the clinical faculty member is primarily affiliated.

ii. The process to be followed for reviewing and addressing concerns with the clinical faculty member will be defined by the Department. General guidelines for Departmental approaches are outlined under the section ‘Departmental Procedures for Reviewing Concerns.

iii. Departments may elect to contact the Learner to get additional information in collaboration with the Learner Affairs Unit. The impacted Learner(s) will be supported during this period by the Learner Affairs Unit and their identity will not be disclosed to the involved clinical faculty member without their consent.

c. Inappropriate feedback

i. Assessments containing offensive or unprofessional language and/or ratings that are perceived as unsubstantiated will be flagged and forwarded to the Learner’s Program Director (PGME) or Phase Lead (UGME). The process to be followed for reviewing and addressing concerns with the Learner will be defined by the program.

Procedures for Amalgamating Teaching and Supervision Assessments

A. Following screening, teaching and supervision assessments will be de-identified by an employee working under the Vice Dean of Clinical Faculty Affairs that has no involvement in the assessment or promotion of medical Learners. They will replace information about the Learner including their name and program with codes that only they have access to. This deidentified data will be submitted to the Continuous Quality Improvement and Accreditation Unit where they will be amalgamated and analyzed to create reports to inform the following use cases:

i. System Development

1. Annual normative reports of aggregate teaching and supervision assessments organized by department, division, clinical rotation, and learning experience will be forwarded to the relevant clinical faculty leaders in addition to the Vice Dean of Education, and Assistant Dean of Clinical Faculty Development. These reports will investigate trends and will be used to inform the design of clinical faculty development interventions with the goal of improving teaching and supervision throughout the TMU SoM.

2. Over time, de-identified teaching and supervision analytics will be integrated with Learner and clinical faculty social identity data to identify discrimination against clinical faculty from systematically marginalized communities with the goal of identifying, quantifying, and addressing such discrimination in a systematic fashion. These reports will be forwarded to the relevant clinical faculty leaders as well as the Vice Dean of Education and Vice Dean of Community, Culture, and Belonging.

ii. Clinical Faculty Development

1. To preserve Learner anonymity, teaching and learning assessments will be aggregated and delivered at least once annually to individual Clinical Faculty provided there is sufficient volume of Learner data to protect Learner identity. Usually, the minimum aggregate data would be that at least three separate Learners from any specific level of Learner (e.g. UGME, PGME, or CPD) have completed assessments on that Clinical Faculty - regardless of context.

2. Teaching and supervision assessments that have been delivered to Clinical Faculty will be amalgamated annually and delivered to their Department Head and/or Division Head prior to the annual clinical faculty review period.

iii. Learner Development

1. Every six months the Director of each training program will receive an aggregate report outlining aggregated metrics summarizing the teaching and supervision assessments completed by Learners in their programs. These reports will not specify which Clinical Faculty were evaluated or contain any narrative information, but they will provide an overview that highlights the number of assessment forms completed on each rotation and/or clinical experience as well as metrics quantifying the quality of their assessments, when available.

2. Every six months Learners will receive a report that normatively compares their completion of their teaching and supervision assessments relative to peers in their program and/or comparable programs. All metrics but their own will be presented in an anonymized, aggregate format.

iv. Promotion

1. Clinical Faculty who are applying for promotion will be provided with a report visualizing all of their teaching and supervision assessments completed within the application period that provides normative comparisons to clinical faculty in their Division, Department, and overall.

Departmental Procedures for Reviewing Concerns

A general process for the assessment of Clinical Faculty teaching is outlined below. Departments may modify the general process depending on the size of the department, number of divisions, and/or number of  postgraduate programs within the department. It is recognized that processes may differ for full-time  Clinical Faculty and those with part-time appointments.

Concern(s) may arise during the screening and/or review of teaching and supervision assessments about Clinical Faculty teaching and/or supervision of Learners and/or professionalism.

A. When identified during screening procedures, the Associate Dean of CFA will inform the Clinical Faculty member’s Department Head of the concerns.

B. Once the information is received and reviewed, the process to be followed will be determined based on the nature of  concerns and previous performance assessment data (e.g., repeated concerns, previous interventions  etc.).

C. The process of review and debriefing concerns with the Clinical Faculty member must be defined by the Department.

D. Following review of the concerns, including previous performance information, the Department Head should identify the areas requiring support including but not limited to:

a. Creation of a positive learning environment

b. Effective teaching and assessment strategies

c. Appropriate supervision 

d. Professionalism

E. If the concerns are considered egregious in nature, including but not limited to:

a. Serious professionalism breaches

b. Issues with supervision, resulting in potential risks to patients or Learners.

c. Concerns with clinical competence

The Departmental Representative should seek input from others, including but not limited to the Associate Deans, UGME and PGME, Vice Deans, and Hospital leads, as appropriate. 

F. Departmental Representative is advised to consult with an intake office under the Discrimination and Harassment Policy and/or the Director of the Sexual Violence Prevention  and Response Office if issues fall outside the scope of these guidelines.

G. Concerns of Clinical Faculty performance raised outside of the written Clinical Faculty assessment process requiring  further action must be shared with the Head of the Department, depending on the nature of the concerns.

a. Concerns brought forward by Learners outside the written assessment process must consider confidentiality if maintenance of anonymity is not possible.

b. Where appropriate, consent should be sought from the Learner to extend the circle of  confidentiality.

c. Learners may seek additional support such as from the Learner Affairs Unit (LAU).

H.. Discussion with Clinical Faculty members by Department Heads (or delegates) should include the following:

a. Relay the concerns outlined by the documentation and/or verbal feedback from Learners and the impact on the Learners.

b. Discuss the Clinical Faculty member’s interpretation of the concerns and give opportunity for self reflection and sharing of contributing factors that may influence performance. 

c. Shared development of an individualized educational plan and identification of required resources and support. 

d. Shared plan for monitoring for improvement and timelines for review. 

e. Discussion of consequences if no significant improvement.

I. In some instances, discussing the details of feedback may result in unintentionally revealing the  identity of the individual raising concerns. The Department Head (or delegate) should be aware of the risk of revealing the identity of people reporting concern(s) and mitigate this risk where appropriate, including:

a. Clarify with the individual whether their identity can be disclosed;

b. Only share enough information to adequately explore the concern(s).

J. In some instances, upon reviewing the concerns, it may be determined that the concern(s) are best  addressed by moving immediately to a School of Medicine and/or TMU intervention. This includes but is not limited to:

a. An assessment of whether the concerns are within the scope of these guidelines or within the jurisdiction of other University policies.

b. Prior attempts at program interventions around the same or similar concern(s) were ineffective  or poorly received.

c. The circumstances of the concern(s) are egregious in nature and/or require emergent or urgent  attention (e.g., patient, Learner, other(s)’ safety potentially affected; institutional reputational issues).

d. Issues related to clinical competence brought forward by Learners that need to be investigated/addressed by hospital leads, in collaboration with the University.

K. The departmental intervention may result in the following outcomes after consideration of the  evidence, including weight and relevance. Written documentation is required to outline the areas of  concerns, discussion with the Clinical Faculty member, and the outcome:

a. No further action is required if:

i. the information provided by the Clinical Faculty member clarifies the situation; or

ii. the Clinical Faculty member has derived insight from the discussion alone; or

iii. the events were not felt by the Departmental Representative to require further intervention.

b. A developmental/remediation plan is developed in collaboration with the Clinical Faculty member which clearly outlines:

i. areas of concern;

ii. educational resources (e.g., coaching, Clinical Faculty development sessions);

iii. monitoring plan and timelines; and

iv. consequences if concerns are not appropriately addressed.

c. The Department Head believes the concerns are addressed through existing University policies that are applied (e.g. Cliincal Faculty Code of Conduct, Discrimination & Harassment  Policy, Sexual Violence Policy, Research Integrity). This recommendation is made for the following reasons, including but not limited to:

i. The clinical faculty member does not demonstrate adequate understanding of the concern(s) or insight into the concern(s).

ii. The clinical faculty member fails to engage in the developmental plan.

iii. The clinical faculty member identifies other issues or concerns that would necessitate a School of Medicine intervention.

L. Clinical faculty members may choose to seek additional support, such as a discussion with the Department Head, Office of Clinical Faculty Relations.

M. In some cases, information shared in the Teaching and Supervision assessments may fall under the jurisdiction of other policies of the  University and in rare circumstances, regulatory bodies or laws not otherwise reflected in university policy which may obligate external reporting. In those cases, subject to limitations at law, the information will be shared with the relevant governing body as per policy.

N. Legal counsel at TMU must be consulted if a clinical faculty member retains legal counsel to represent them in the process and/or decisions following review of performance concerns. 

O. School of Medicine intervention is required when: 

a. Prior attempts at departmental interventions around the same or similar concern(s) have been  unsuccessful.

b. The clinical faculty member refuses to engage in the departmental Intervention. 

c. Removal of Learners from the supervision of the clinical faculty member is being considered or has  taken place.

d. The clinical faculty member identifies other issues or concerns necessitating a SoM intervention.

e. The circumstances of the concern(s) are egregious in nature and/or require emergent or urgent attention (e.g., patient, Learner, other(s)’ safety potentially affected, institutional reputational  issues).

f. Other issues at the discretion of the Head of the Department (e.g., social media).

P. The Head of the Department will lead/participate in the SoM Intervention and formally involve appropriate leadership as required, including but not limited to:

a. Associate Deans of UGME and PGME;

b. Vice Dean of Education;

c. Vice Dean of Clinical Faculty Affairs;

d. Employee and Labour Relations;

e. University Legal Counsel; and/or

f. Clinical Leaders of Hospitals, as appropriate.

Procedures for the Development and Implementation of a Remediation Plan

A formal documented remediation plan may be required after either a departmental or SoM process. A  remediation plan is required for significant concerns that may or has affected the ability of a faculty member to discharge their teaching responsibilities. 

A. The faculty member is expected to be actively involved in the development of a remediation plan. The Department Head (or delegate) may consult, as appropriate, to develop an appropriate plan for remediation to address identified concerns.

B. The remediation plan must include the following information:

a. Identification of the areas of deficiency 

b. Components/nature of the remediation

c. Suggested timelines for monitoring and review

d. Expected outcomes of the remediation plan

e. Specification of how the faculty member’s performance in the remediation plan will be evaluated

f. Consequences of receiving an unsatisfactory outcome of the remediation plan

g. Any modifications to current duties/responsibilities and conditions for their resumption

h. A specific plan for implementation and follow-up, including a process to ensure the effective and timely implementation of the remediation

i. Confirmation of individuals responsible for assessment of specific remediation activities

C. The faculty member will have the opportunity to review and respond to the remediation plan within a timely manner, regardless of their involvement in the development of the plan. The faculty member’s response will be taken into consideration and may be incorporated into the remediation  plan at the discretion of the Department Head or Delegate (e.g. a Deputy Head or Division Director).

D. The remediation plan will have final review/approval by the Department Head. The Department Head must notify all educational programs and teaching sites where the faculty member has responsibilities of any amendments or restrictions in a timely manner that would preclude them from fully discharging  their teaching responsibilities.

E. Once finalized, the Department Head will provide a copy of the remediation plan to the faculty member and any other individuals, as appropriate for timely final review and acceptance.

F. A copy of the remediation plan and any other relevant documentation, including documentation of  the satisfactory resolution of the issue through remediation, will be maintained in the faculty member’s departmental file for a minimum of ten (10) years. This period may be extended at the discretion of the Chair. If the faculty member has a cross- or joint- appointment, that department  will be notified as appropriate. The relevant educational programs must also be notified.

G. After completion of the remediation, documentation will be reviewed by the Department Chair or delegate and a decision will be made regarding the status of the remediation, which may include:

a. Satisfactory completion of the remediation 

b. Provisional satisfactory completion of the remediation – some minor items outstanding,  potentially triggering the need for further remediation and/or continued monitoring of  performance

c. Unsatisfactory completion of remediation

H. After consideration of all available information, including but not limited to the status of the remediation plan and any other relevant information provided by the faculty member, the  Department Head will make the final determination for next steps.

I. If a situation arises that requires suspension or temporary withdrawal of learners from the supervision of a clinical faculty member, procedures from the Clinical Faculty School of Medicine Clinical Faculty Appointment, Renewal, & Promotion Policy and Procedures will be followed.

Appeals

A. Ad hoc Committee for Consideration of Appeals

In the event an appeal from a Clinical Faculty Member is received arising from a negative Learner assessment (or set of assessments), an ad hoc committee will be struck and be chaired by Vice Dean of Clinical Faculty Affairs. The committee will comprise the following individuals:

a. Chair - Vice Dean of Clinical Faculty Affairs (non-voting)

b. One Associate Dean - One of the UGME or PGME Associate Dean

i. If the appeal concerns medical student assessments the Associate Dean of PGME will serve on the committee.

ii. If the appeal concerns PGME Learner assessments the Associate Dean of UGME will serve on the committee.

iii. If the appeal concerns both medical student and PGME Learner assessments the Associate Dean of Clinical Faculty Relations will serve in this role.

c. Two (2) elected Clinical Faculty from the School of Medicine Council who are not within the Division of the appealing Clinical Faculty.

d. Two (2) Learner representatives - one from UGME and one from PGME, who shall be drawn from the School of Medicine Council and/or the elected representatives pool as described in the Appointment of Clinical Academic Administrators (ACAA) policy and procedures. Both Learners must be free from a conflict of interest and therefore should not be in the same class that originates the assessments that are being appealed and/or program as the Clinical Faculty member.

B. Ad hoc committee’s recommendation process

a. Upon the appellant’s submission to the Vice Dean of Clinical Faculty Affairs, the ad hoc committee will need to convene within three months (3) of this date.

b. The ad hoc committee will convene at least once synchronously and may choose to draft their decision asynchronously via password-protected file-sharing.

c. Prior to the meeting, the ad hoc committee will be able to request source materials from the relevant educational Office (e.g. PGME Office, or UGME Office) beyond the submission of the appellant. The Office of Clinical Faculty Relations will be charged with aggregating this material on behalf of the committee.

d. The appellant will be able to address the committee and they will be encouraged to bring an advocate on their behalf. This advocate can be a member of their Departmental leadership or Clinical/Hospital leadership.

e. If the Clinical Faculty member retains legal counsel, the Vice Dean of Clinical Faculty Affairs must be notified and TMU will provide legal representation to the ad hoc committee.

f. Other concerned parties may only address the ad hoc committee upon invitation via the committee and/or the Chair.

g. Once all the materials and statements have been rendered, the committees deliberations will be had to generate a recommendation to the chair of the committee:

i. Members of this ad hoc committee should strive to reach consensus in respect of its recommendation to the chair of the committee.

ii. Where consensus cannot be reached, such decisions and recommendations will be arrived at by majority vote, which shall be 3 of 5 (three of five) voting members.

iii. The recommendation about the appeal will then be forwarded formally in writing to the Chair, who shall then decide whether they endorse the recommendation.

C. Communicating the Recommendation to the Dean for Decision

a. If the committee and the Chair are in agreement, the recommendation will be forwarded by the Vice Dean to the Dean for final ratification.

b. If the committee and the Chair (usually the Vice Dean of Clinical Faculty Affairs) are in disagreement, the investigation, documents, and appellate statement will be forwarded to the Dean/VPMA for consideration.

D. The Dean/VPMA’s Decision

a. Upon receipt of either forms of the above package, the Dean/VPMA will make a decision about this appeal within one (1) month. The Dean/VPMA reserves the right to consult the TMU General Counsel on the matter. The Dean/VPMA’s decision will be final. The ad hoc appeals committee cannot overrule the decisions of the Dean/VPMA.

In addition to the questions for each form that are outlined below, metadata will also be collected including the date, trainee type (type of medical student, resident, or fellow), clinical rotation, and assessment form identification number. The forms have been designed to be completed rapidly with a minimal number of questions requiring Likert scales and narrative elaboration required for lower ratings. The following are sample forms that the Clinical Faculty Affairs Section may supercede and redesign at their discretion.

A1: The Ward

A2: Outpatient Clinic

A3: Emergency Department (including On Call consultation)

A4: Non-Surgical Procedures

A5: Surgical and Perioperative Care

A6: Resuscitation

A7: Sedation and/or Anesthesia

A8: Small Group Teaching/Session

A9: Large Group Teaching/Session

A10: Simulation

A10: Other