Online Course Evaluations

Based on user feedback, implementing uniform procedures across campus will increase the response rate, ensure data integrity, and improve reporting. Please adhere to the following procedures when utilizing the Online Course Evaluation (OCE) system:

UNIFORM PROCEDURES

1. Standard Template

Do not alter the standard template questions, 1-29. Departments can add department-specific questions at the end of the survey beginning with question 30. Altering the template will result in data not being archived properly. Any modifications made to the standard template will jeopardize data integrity over time. Additionally, note that there are 25 questions plus 4 section headings (for a total of 29 "questions"), which is why department-specific questions begin with the number 30.

http://its.ucsc.edu/ecommons/evaluation-system/ucsc-evaluation-template.pdf


2. Teaching Assistant (TA) Template

While there have been some requests for a uniform Teaching Assistant template, it is not within the purview of Academic Affairs to establish a uniform template. Below are examples from Course Sponsoring Agencies (CSAs) that can serve as a starting point for CSAs that do not yet have their own. CSAs are advised to consult with Graduate Division and Graduate Council before issuing their own templates.


3. 5th-Week Roster

During a recent evaluation cycle, some students who officially dropped a course were emailed an evaluation for a course in which they were no longer enrolled. This occurred because the evaluation was created prior to the 5th-week roster. The concept is thus: rosters remain dynamic (i.e., are updated according to add/drop/swaps) until they are attached to an evaluation. Once they are attached to the evaluation they become static and will not change to reflect enrollment in the course. Therefore, it is recommended that Course Sponsoring Agencies (CSAs) not attach a roster to an evaluation until after the Add/Drop/Swap deadline. This will ensure the most accurate roster for the course.


4. Communication Plan

At UCSC, faculty engagement appears to be the single most important factor to increase student participation and response rates. Department Managers, Chairs, and Provosts can increase response rates by alerting faculty of the dates that evaluations are to be completed and requesting that they encourage their students to complete the online review process even in courses that do not use eCommons for any other purpose.

  • A comprehensive communication plan includes:
  • A message from the Department Chair or Provost to students and faculty communicating the importance of course evaluations and noting the ease of doing so online.
  • Information and instructions for faculty to give to their students about online evaluations (e.g., passwords used, anonymous, confidential, value of constructive criticism and feedback) and how the data will be used.
  • Department-specific student notification and reminder text should be included in each survey (evaluation), specifying open and close dates. For the Winter 2017 quarter, it is important the evaluation period close on Sunday, March 19 at 11:59 p.m. prior to the start of final exams.
  • Information for instructors about when the results will be available to them and in what format.

Click here for Sample Communications.


5. Deployment Period

Historically, paper evaluations were handed out to students in the final week of classes, prior to finals week. With the online process this practice should not change. Evaluations need to be collected prior to finals week. This has been, and will continue to be, standard campus practice. Please set up Winter 2017 evaluations as follows: 

  • Evaluation period: online evaluations should be available from Sunday, March 5 at 12:00 a.m. to Sunday, March 19 at 11:59 p.m. Evaluations should cease being available by Monday, March 20 at 12:00 a.m.
  • Student Notification:  until such time that there is a technical solution to aggregate notification (i.e., students receive only one email that includes information on all evaluations they have yet to submit), choose the notification reminder setting ‘every-other-day’ for each evaluation instrument.

6. File Naming

To maintain the integrity of our archival process, it is imperative for departments to appropriately name both the evaluation instrument and the report data file. The following are examples of appropriately named files:

  • Best practice for naming an evaluation for a single course:

TERM / COURSE TITLE / COURSE NUMBER / INSTRUCTOR

E.g., W12 SOCY 177G 12345 Smith, John

  • Best practice for naming a group or batch of evaluations:

TERM / DEPARTMENT

E.g., W12 Music Dept Course Evaluations


7. Archiving OCE Data

Since the Course Sponsoring Agency (CSA) is the office of record for instructor evaluations [not the Faculty Instructional Technology Center (FITC)], it is imperative that CSAs download both the PDF and excel spreadsheet for archiving purposes. A memo recently sent out by Academic Personnel Office (APO) advised departments to have the excel file on hand should it be required during the faculty personnel review process. It is not enough to only download the PDF. During the faculty review process, the Committee on Academic Personnel (CAP) prefers to look at student comments within the context of all feedback provided by an individual student. It provides additional context for interpreting the individual remarks. Without the excel file, CAP has no way of connecting free response comments to multiple-choice comments for an individual. 

Prior to the availability of an archival solution, the permanent archiving of OCE evaluations within units must be given special consideration as follows:

  1. Ensure that the files you have downloaded from the OCE (.pdf and .xls) are stored on a backed-up server that is accessible to current and future departmental staff.  The evaluations may be needed 10 or more years from now.
  2. To add redundancy, it is highly recommended that you print out and store paper copies of the evaluations using your standard paper storage process.
  3. Please discuss your archiving processes with the appropriate divisional academic HR coordinator.

8. Faculty Review of Evaluations

Faculty may see their evaluations once the grade posting deadline has passed and it has been verified that they have submitted grades for their course. Until all grades for a course are submitted, faculty should not review evaluations. This standard campus practice is not changing.

  • All units are responsible for the distribution of the evaluation reports directly to their own faculty, just as they have done when using Scantron or other evaluation processing methods.  It is important that the administrator communicate with instructors their schedule for the release of this information. It is not currently available online for instructors.

9. Faculty Communication

Communication from faculty to their students is of the utmost importance to increase student response rates. Some methods to increase student response rates include:

  • Remind students of the deadline date of the evaluation and the importance of results after each class.
  • Explain in the course syllabus the importance of student input for course improvement and that their evaluations are taken seriously and do effect change.
  • Reinforce the message that instructors will not have access to individual student evaluations. Summary reports will be provided only after grades are submitted.
  • Remind students that their response will be used to improve the course for future students. Students should know that a professor will take their feedback seriously and specific actions will be taken to resolve any issues raised.
  • Help students understand how to give constructive criticism. This will help students give valuable feedback. Further, this action will help convince them that their comments will be heard.
  • Direct students to a computer laboratory where they can submit their evaluations.
  • Consider the use of department- or course-level incentives.  Departments have access to the list of students who have completed evaluations; some faculty have used this information within their classes.  Of course, such information should not be shared with fewer than 10 respondents, as it may make responses identifiable.  Evaluations can be submitted without answering any questions.

SEE ALSO:

May 2012 OCE Procedures

October 2012 OCE Update and Additional Procedures

April 2013 OCE Update

May 23, 2013 OCE Urgent Update

OCE Sample Communications

ITS OCE Service Page

Evaluation System Enhancements (FITC)

Online vs. Paper Evaluations (external research)