Rubric Scoring: A Measurement Tool for Student Learning Reformas a Result of QA Process

Praphan Chanthanapode
Kasem Vejakupta
Chuchai Durongkasaenee
and Suwichote Chiawchan
Thailand

Orthopedic Surgery one of 13 specialty units of Buddhachinaraj hospital, has participated in instruction and learning in a M.D. program that the hospital cooperated with Naresuan University since 1995. After self-assessment by staff in the unit and internal assessment by the internal assessor team comprising of hospital staff (Details will appear in Dr. Yongyos et al's paper), Orthopedics Surgery unit came up with key question of how to improve learning and instruction by means of student assessment. The assessment should achieve 3 purposes; i.e., 1) enhancement of student learning in such that they understand clearer body of knowledge and are more skillful on clinical performances with the right and safety of patients are prime concerns,2) reflection of the real performances of students with patients and their relatives, and 3) alignment of the learning experiences to the educational objectives. This paper describes how the Orthopedic Surgery employed the Rubric Scoring to encourage student to improve their own learning, and to strengthen staff understanding and involvement in the learning and assessment processes.

The modifications of assessment initiated during discussion in one of regular meetings of the staffs. The staffs were encouraged to express their ideas regarding advantages, disadvantages, and limitations of the modifications. In fact, the staff argued whether it would really add more workloads on top of already heavy patient-loads. Finally, they agreed to try Rubric Scoring - a scoring scale consisting of a set of criteria that describe what expectations are being assessed/evaluated and descriptions of levels of quality used to evaluate student works or to guide students to desired performance levels. Each staff then was charged with 2-3 items of Rubric construction, which later were assembled to cover learning activities; e.g., ward works, OPD works, case conference, etc. It was agreed to construct the Rubric Scoring, for each learning activity in the way that the Rubrics would 1) define all key performances needed to be assessed, 2) assign "full point" for each key performance by weighing from its significant against the intended performance outcomes of students, and 3) for each key performance, define 5 different quality levels gradiently; i.e., level 1 signifies the poorest performance and level 5 the best. It was also agreed that the level 3 - performance is "acceptable" or "passing level." Moreover, these dimensions of each key performance must meet the standards of general practitioner who 1) are qualified by Medical Council Standard for Licensing, and 2) are ready to practice in a community hospital in the rural areas, as stated in curriculum objectives.

Start implementing in this 2002 academic year, staff of the Orthopedic Surgery unit have been surprised by changes in student performances. A group of students who just pass the Orthopedic Surgery rotation came up to the staff and informed that Rubric Scoring criteria had guided them in self-directed learning before attending the classes and they were satisfied with that. Students concluded that the Rubric Scoring essentially provided clear indications of what should be learned and how, and when should they perform the skills.

Currently, staff of Orthopedic Surgery keep improving their Rubric Scoring Criteria with the hope that the newly developed set would be clearer, more valid and reliable and less time-consuming for the staff.