Digital Objective Structured Clinical Examination (dOSCE) Pilot

What is the dOSCE project?

The University of Queensland has a capability gap with regards to the technologies and practices that support Objective Structured Clinical Examinations (OSCEs). This project seeks to fill that gap by procuring and implementing a digital technology solution to manage and report on OSCEs.

How was this project initiated? 

This project was initiated on request from the Faculty of Medicine who currently use a free web application. The current solution was highlighted during the AMC Accreditation review in May 2022 as having a high number of risks associated with it that need to be addressed. These risks include the lack of security of student and assessment data, the application not being centrally supported and the fact that no service levels are in place. It is also labour intensive, requiring a number of manual tasks and double handling of data in order to make the system work. 

What is the objective of the pilot?

The pilot's objective is to procure and implement a technology solution for Medicine to manage and report on OSCEs for release in Semester 1 2023 in readiness for commencement of the new Medicine Program.

The new digital OSCE system may also address UQ’s needs for Oral and Demonstrative Exams more broadly (e.g. where the context is not clinical and where video interviewing is required) but the purpose of this procurement is a digital OSCE system suitable for use by faculties across the University, with the Faculty of Medicine as the key stakeholder.

What is happening? 

After initial requirements gathering and procurement planning, requests for tenders were released to the open market. Once all tenders were collected, evaluation planning occured, followed by a shortlisting process and a final solution recommendation. A representative from each faculty was appointed to the evaluation panel to ensure equitable representation in choosing a solution that meets UQ’s needs. The Working Group approved the evaluation panel’s recommendation, therefore a contract has been finalised with the chosen vendor based on the final technology option.

After intensive training, familiarisation with the risr/assess platform, producing training resources, a closed small-scale pilot was held followed by a successful larger pilot. The dicision was made to move support of risr/assess to the Faculty of Medicine. To support the decision a handover of project resources and training materials was conducted.

Staff communications

• 22 August 2022: This project page was created to keep stakeholders informed
• Articles regarding this project will feature in the Teaching and eLearning News newsletter
• Updates at the eLearning Operational Forum (eLOF)
• Updates at eLearning Service Meetings
• Reports to the Digital Learning Sub Committee
• Working Party meetings

Project timeline

• 20 July 2022: Business requirements gathered
• 19 Aug 2022: Working Party Meeting
• 22 Aug 2022: Procurement plan and Request for Quote (RFQ) finalised
• 22 Aug - 12 Sep 2022: Tender period
• 13 -16 Sep 2022: Initial offer review and shortlisting (Evaluation Advisors)
• 19 - 23 Sep 2022: Individual scoring (Evaluation Panel)
• 26 - 30 Sep 2022: Scoring moderation (Evaluation Panel/Advisors)
• 17- 21 Oct 2022: Vendor demonstrations (Evaluation Panel/Advisors)
• 17 - 4 Nov 2022: Solution recommendation (Evaluation Panel/Advisors)
• 8 Nov 2022: Recommendation presented to Working Party
• 5 Dec 2022: Working Party Meeting
• 13 Jan 2023: Working Party Meeting
• 20 March 2023: the vendor and the solution was determined and the contact proceedings started
• July 2023: initial training with the vendor
• 24 July 2023: dOSCE pilot (3 stations, 3 examiners, 3 simulated patients)
• August - September 2023: preparatiion for the first dOSCE exam on risr/assess platform
• 23 September 2023: first year 500 students dOSCE exam.
• 15 October 2023: handover meeting.

Project Team

Working Party Membership

Evaluation Panel

Evaluation Advisors