Documenting Teaching Effectiveness
One component of your teaching portfolio should be how you document your teaching effectiveness. For many of us, that will usually include the use of student evaluations. But this can also include feedback from peers, advisors, consultants, and other constituents. A significant component of this section is your own reflection about your effectiveness based on the use of data gathered from these various sources. Even more importantly, it is not just to show a potential employer how good your evaluations are, but to demonstrate how you used the feedback in your development as a teacher.
Your students are the most obvious source of feedback on your instruction. Research has shown that students provide valuable information about your teaching if the questions are structured in a useful way. Michael Theall (2002) has written a concise, insightful article in which he debunks several of the myths about student evaluations. For Ohio State’s policy on evaluation of instruction, see the section entitled “Evaluation of Instruction” in the Office of Academic Affairs Handbook (45 ff.).
Typically, your portfolio will have a separate section devoted to discussing student feedback. There are three main elements of this section: numeric (quantitative) evaluations, discursive (qualitative) evaluations, and reflection about and interpretation of the evaluations themselves. Various types of student feedback options at Ohio State are discussed below.
Types of Student Feedback
There are many ways to assess your teaching; using an end-of-term survey is the most popular (and mandatory for most instructors at Ohio State). End-of-term surveys provide instructors with valuable information to help shape the course and teaching strategies for future course offerings. But how can an instructor collect data that could potentially help enhance the course for the current students? Here are a few tools for getting both end-of-term and mid-term student feedback.
Student evaluation of instruction (numerical/quantitative ratings)
SEI (Student Evaluation of Instruction) forms are typically filled out at the end of the quarter. Students are asked to assign a numerical value to various aspects of the course and the instructor, on a scale of 1 (poor) – 5 (excellent). Ohio State faculty and TAs can download their cumulative SEIs at Faculty and Staff. Log into the Faculty Center. Click on the link to SEI Info and choose “Generate New SEI Cumulative Report.” For more information about SEIs, see the online SEI Handbook from Ohio State’s Office of Academic Affairs. Individual departments may have their own unique surveys for instructors to use in addition to or in place of the SEIs.
Discursive student evaluations (qualitative)
Often conducted in conjunction with SEIs, discursive student evaluations are written comments that the students offer in addition to the numerical ratings. These allow the students to add more information about issues evaluated on SEIs and to address issues that do not appear on the SEI forms. For example, you might ask them what about the course or instructor helped them (or didn’t help them) learn; their most (or least) favorite part of the course; how valuable certain assignments were; what they thought about the readings, etc. Some departmental forms use a similar system for collecting open-ended feedback. Discursive feedback does not go to the Registrar’s office, but instead will either go to the department (and may be typed up for an instructor in some cases) or will be returned directly to you as raw data.
Mid-quarter evaluations can be conducted at any time (and several times) during the quarter. The advantage to collecting mid-quarter feedback is that you can act on it immediately, by the next class.
CATs (Classroom Assessment Techniques)
The Classroom Assessment Techniques described briefly here are taken from Angelo and Cross (1993). These are typically very brief and focused questions or tasks related to the current content of your course. For example, at any point during a class, you may ask your students to write a “minute paper” addressing something specific, e.g., “What were the two most important points covered in class thus far?” Give your students a minute to write their answers, collect them, and use their responses to guide you in constructing the next class period. Another useful question is, “What is the muddiest (most confusing) point from today’s class?” For more information on CATs, see Angelo and Cross (1993) or go to the following links for examples (many web-based resources are available — these are just a few).
FYI (Feedback on Your Instruction)
A useful tool in putting together a longer questionnaire is FYI, one of Ohio State’s web resources (available for Ohio State instructors only). Using this website, you can design a questionnaire for students to answer about your course. Several questions are listed from which you may choose a relevant subset, depending on your course and your teaching goals. The questions are either scaled or open-ended. You simply select the questions (you may want to consider 8-12 questions to include at any one time, depending on the ratio of scaled versus open-ended questions) and print it from the website.
Please note that this is not a web-based questionnaire that your students may take online. Instead, you must print out the questionnaire, and copy it for your students. You can either interpret the results on your own or ask a consultant from UCAT to assist you. FYI questions may also be used to supplement the survey that you provide at the end of the term.
SGID (Small Group Instructional Diagnosis)
SGIDs are focus groups conducted by UCAT consultants during class time and with the instructor absent. The consultant will put the students in small groups and ask them to talk about and write down answers to three questions: (1) What about the course/instructor is helping you learn? (2) What about the course/instructor is not helping you learn? and (3) What specific suggestions do you have for improvement? After the questions’ responses are transcribed, the instructor and UCAT consultant meet, to discuss the feedback and identify constructive ways to respond to it. Instructors receive a document, that can be printed on UCAT letterhead, providing a short description of the process as well as the transcribed student responses. This document is shared only with the instructor, but could be a great source of data to include in a teaching portfolio.
Other teaching professionals — such as your peers, professors or university teaching consultants — can also provide you with evaluations of your teaching. If you are thinking about getting documentation on your teaching for summative and/or formative purposes, you may want to consider which individuals are appropriate sources to give you written evaluations on your teaching: peers, outside consultants, faculty, an advisor, or others who know your work and the field. Below are some kinds of documents from others that could be included.
Peers, advisors, and other faculty
- Written feedback from a classroom observation that details judgment on teaching
- Written feedback that details judgment on course materials, such as handouts, exams, and syllabi
- Written documentation that details teaching contribution to the department
Documentation from outside consultants
- Written feedback from a classroom observation that details strengths as well as areas for improvement
- Written summary from a classroom videotaping that details strengths as well as areas for improvement
- Written summary of open-ended comments from student evaluations of instruction that details strengths as well as areas for improvement
- Written summary from midcourse feedback that details areas of strength as well as areas for improvement
- Written summary that details the teaching improvement work that you did with the consultant
Be inclusive. Find an efficient way to document as many courses as possible, if appropriate (there is no need to go beyond five years in most cases). If the courses were for different student populations — for example, freshman vs. upperclassmen vs. adult learners — you might want to point this out, and organize it accordingly.
Be descriptive. Explain when and how these evaluations were collected. For example, if the evalations are provided in the form of SEI results, clearly explain what the term “SEI” means, describe how it is used on your campus (mandatory vs. elective), etc. Include course names, quarters/years taught, number of students in class, number of students providing reponses to the survey, etc. Be sure to include a description of the scale (e.g., if the scale is 1-5, state whether 5 is “excellent” or “poor”). For qualitative data, explain when and how these evaluations were collected. If you asked your students to address specific topics, mention them.
Graphic displays. You may decide to display SEI or other quantitative data graphically — which is useful if your evaluations have improved over a period of several quarters, for example, or if you are showing data for a few different classes. Be sure that the graph is easy to read and interpret. Be careful not to assemble graphs of numbers with no explanation. Somewhere on the graph, include number of students, dates the courses were taught, qualities that are being evaluated, etc. Include a figure legend that describes what the graph is showing, in case the reader does not have time to decipher the graph. A main feature of graphs is that they should not be too cluttered; clarity is key. Depending on your audience, you may choose to include all courses taught over a certain period of time (promotion and tenure purposes) or only a select number (applying for a faculty job) in order to showcase your “best” work or the courses you’ve actually designed. Either way, it will be important to include an explanation of which courses you’ve chosen to include, and why.
Include student comments. If you have copies of written student feedback, it can be submitted as supporting data. However, as discussed for graphic displays above, it is inadequate to provide a long list of student comments without any kind of explanation. One way to summarize student comments is by category. Within each category, it is important to include representative student comments; there is no need to include a large number of comments that target the same issue. You may also want to include a few comments that address a teaching weakness. It is important, however, to provide some commentary on what you have done or plan to do with this kind of feedback. It is also important to let your reader know that these are “representative” comments and not every one you’ve ever received. Choose the number of comments to include carefully. Too many can be overwhelming; too little can look sparse.
See an example of how someone has summarized end-of-term discursive feedback by organizing representative comments around common themes or categories.
Be reflective: Even if you follow the tips listed above, they are inadequate without a paragraph that describes your interpretation of the evaluations and how you have used the feedback to change or enhance your instruction or course design. The following questions are designed to help you think about how student feedback has influenced you as a teacher.
- At what point during the quarter do you collect feedback?
- How often do you collect feedback?
- For what purposes do you collect feedback?
- How have you integrated this feedback into your teaching?
- What would you still like to improve?
- What will you continue to do?
Once you choose how you will summarize discursive and quantitative feedback, you will want to consider how you will lay this out in one document for your portfolio.There is no right or wrong way to do this.
The following samples, written by winners of the Graduate Associate Teaching Award at OSU, represent different ways of addressing student feedback and teaching effectiveness. All of them show how the instructors have summarized end-of-term discursive feedback by organizing representative comments around common themes or categories.
Robert M. Anthony, Sociology
Bora Bosna, Mathematics
Monali Chowdhury, Psychology
Joshua Eckroth, Computer Science and Engineering
Kristin Edwards Supe, Psychology
Szu-Hui Lee, Psychology
Mahesh Iyer, Chemical & Biomolecular Engineering
Tim Jensen, English
Elizabeth Riter, Civil Engineering
Spencer Robinson, Slavic and East European Languages
Diana Ruggiero, Spanish & Portuguese
Leslie Wade, Psychology
Dawn Walts, English
You will need to make some decisions on how you will organize this section. If this is for a formative portfolio, how much information you include depends on what you want to know and think about. If this is for a summative portfolio submitted for a job application, how much you choose to say and how much detailed data you include will depend on how much information you think an institution to which you are applying wants to know.
You can either include this information in your portfolio as a separate section, include relevant pieces of this document in other sections of your portfolio, or do both. For example, if the rationale for changing how you taught a course mid-quarter, or from one quarter to the next, was a result of student feedback, you will want to include a few sentences describing the feedback in the teaching responsibilities section of the portfolio. You might also mention it as part of your teaching philosophy statement if your philosophy of teaching was influenced by student evaluations. It might even be possible to quote a student comment in your cover letter. Mentioning your response to student and non-student feedback in various locations within your portfolio serves to connect the different documents that make up your portfolio.
Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass.
Theall, M. (2002, Fall). Student Ratings: Myths vs. Research. Focus on Faculty, 10(3), 2-3. Brigham Young University Faculty Center.