Assessment of Surgical Trainees Utilising Work-based Assessment with Behaviourally Anchored Markers
Date
2024
Authors
Pan, Oliver
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Introduction
The Royal Australasian College of Surgeons (RACS) created its competency framework in 2003 which initially consisted of nine competencies each regarded as equally important for a practising surgeon. Work-based assessment has gained increasing acceptance and adoption, but there remains a need for an approach that aligns the surgical curriculum to the evaluation instrument for surgical trainees. This thesis will develop a behaviourally anchored assessment instrument specific to surgical trainees, pilot the instrument, comprehensively evaluate the responses, and identify the barriers and perceptions around the deployment of this instrument.
Methods
The primary focus of this project is to develop a novel behaviourally anchored instrument of assessment that is centred on the Royal Australasian College of Surgeons core competencies and integrates self-assessment with supervisor assessment to formulate guided feedback for trainees. This novel instrument will be piloted at a tertiary centre and its associated health network and we will evaluate the suitability of such an instrument for use by surgical trainees.
Results
A novel behaviourally anchored instrument was developed which contains 48-items as part of the tool. 36 of 59 (61%) trainees in the Health Network participated in the pilot assessment. From 25 nominated supervisors, 68 assessments were completed. Overall, there is good reliability for each competency for both trainees and supervisors and a statistically significant difference in self-rating scores compared to those of supervisors. A multiple regression model demonstrated that for both self-ratings and supervisor ratings, the most significant correlation is with the post-graduate year of the trainee. The perceived barriers in the implementation of our instrument suggests significant difference in the perspectives surrounding trainee-supervisor relationships, how a trainee manages negative feedback, the consequences of poor scores and the perceived importance of anonymity and input from allied health and nursing staff
Conclusion
This project has provided an opportunity to explore the use of a new assessment and feedback instrument for surgical trainees that is aligned to the Royal Australasian College of Surgeons competency framework using behaviourally anchored questions and rating scales to maximise relevance and trainee reflection upon their skills. A better understanding of this instrument will potentially be helpful in early identification of underperforming trainees in order to facilitate early intervention, or even its use as a selection tool in formal training programs. The associations identified between trainee and supervisor ratings, and between trainee level of training, allude to the subtleties that our instrument can identify. There is potential for this instrument to be used in trainee selection, formative assessment, and early identification of struggling trainees. These findings demonstrate the acceptability of a digital format in the distribution of self-assessment and supervisor assessment forms. It also provides reassurance that the perception of supervisors who felt that negative ratings may affect trainee-supervisor relationships is unfounded. Surgical units should feel that comprehensive work-based assessment tools involving self-ratings and colleague ratings are likely to be received well by trainees. This method of assessment should provide future trainees an additional source of feedback, and an opportunity of self-reflection to incorporate into their training.
Description
Keywords
Citation
Collections
Source
Type
Thesis (PhD)
Book Title
Entity type
Access Statement
License Rights
Restricted until
Downloads
File
Description
Thesis Material