Digital Objective Structured Clinical Examination (dOSCE) Pilot

July 2022October 2023

What is the dOSCE project?

The University of Queensland has a capability gap with regards to the technologies and practices that support Objective Structured Clinical Examinations (OSCEs). This project seeks to fill that gap by procuring and implementing a digital technology solution to manage and report on OSCEs.

How was this project initiated? 

This project was initiated on request from the Faculty of Medicine who currently use a free web application. The current solution was highlighted during the AMC Accreditation review in May 2022 as having a high number of risks associated with it that need to be addressed. These risks include the lack of security of student and assessment data, the application not being centrally supported and the fact that no service levels are in place. It is also labour intensive, requiring a number of manual tasks and double handling of data in order to make the system work. 

What is the objective of the pilot?

The pilot's objective is to procure and implement a technology solution for Medicine to manage and report on OSCEs for release in Semester 1 2023 in readiness for commencement of the new Medicine Program.

The new digital OSCE system may also address UQ’s needs for Oral and Demonstrative Exams more broadly (e.g. where the context is not clinical and where video interviewing is required) but the purpose of this procurement is a digital OSCE system suitable for use by faculties across the University, with the Faculty of Medicine as the key stakeholder.

What is happening? 

After initial requirements gathering and procurement planning, requests for tenders were released to the open market. Once all tenders were collected, evaluation planning occured, followed by a shortlisting process and a final solution recommendation. A representative from each faculty was appointed to the evaluation panel to ensure equitable representation in choosing a solution that meets UQ’s needs. The Working Group approved the evaluation panel’s recommendation, therefore a contract has been finalised with the chosen vendor based on the final technology option.

After intensive training, familiarisation with the risr/assess platform, producing training resources, a closed small-scale pilot was held followed by a successful larger pilot. The dicision was made to move support of risr/assess to the Faculty of Medicine. To support the decision a handover of project resources and training materials was conducted.

Staff communications

Project timeline

  • 20 July 2022: Business requirements gathered
  • 19 Aug 2022: Working Party Meeting
  • 22 Aug 2022: Procurement plan and Request for Quote (RFQ) finalised
  • 22 Aug - 12 Sep 2022: Tender period
  • 13 -16 Sep 2022: Initial offer review and shortlisting (Evaluation Advisors)
  • 19 - 23 Sep 2022: Individual scoring (Evaluation Panel)
  • 26 - 30 Sep 2022: Scoring moderation (Evaluation Panel/Advisors)
  • 17- 21 Oct 2022: Vendor demonstrations (Evaluation Panel/Advisors)
  • 17 - 4 Nov 2022: Solution recommendation (Evaluation Panel/Advisors)
  • 8 Nov 2022: Recommendation presented to Working Party
  • 5 Dec 2022: Working Party Meeting
  • 13 Jan 2023: Working Party Meeting
  • 20 March 2023: the vendor and the solution was determined and the contact proceedings started
  • July 2023: initial training with the vendor
  • 24 July 2023: dOSCE pilot (3 stations, 3 examiners, 3 simulated patients)
  • August - September 2023: preparatiion for the first dOSCE exam on risr/assess platform
  • 23 September 2023: first year 500 students dOSCE exam.
  • 15 October 2023: handover meeting.

Project Team

NameRoleOrganisational Unit / team
Daniel GoninonDLCR Program Manager / dOSCE Project ManagerITS
Iliria StenningSenior eLearning AdviserITaLI, eLearning Systems and Support Team
Helen RatnykovaSenior eLearning AdviserITaLI, eLearning Systems and Support Team
Lynn KooBusiness AnalystITS
Gary HealySolution ArchitectITS
Adam MassinghamSenior ICT Procurement SpecialistITS
Kellie BradyManager, Academic Programs and PracticesFaculty of Medicine
Dr Christy NobleClinical Learning and Assessment LeadFaculty of Medicine

Working Party Membership

NameRoleOrganisational Unit / team
Professor Pauline FordProject Sponsor and ChairPro-Vice-Chancellor (Teaching & Learning) (Acting) 
Dr Greg WinslettDeputy Director (Digital Learning), Institute for Teaching & Learning InnovationITaLI, Executive Team
Jacqui LynaghManager, Digital AssessmentITaLI, Digital Assessment Team
Dr Simon CollyerManager, eLearning Systems and SupportITaLI, eLearning Systems and Support Team
Iliria StenningSenior eLearning AdviserITaLI, eLearning Systems and Support Team
Kellie BradyManager, Academic Programs and PracticesFaculty of Medicine
Dr Christy NobleClinical Learning and Assessment LeadFaculty of Medicine
Professor Lydia KavanaghAcademic Lead, AssessmentFaculty of Science
Tabitha BozinDeputy Director, Applications Delivery and SupportITS
Daniel GoninonDigital Learning Capability Roadmap (DLCR) Program ManagerITS
Luke GaiterSenior Manager, Digital Capability SupportLearning and Research Services, UQ Library
Loretta Atkinson

Senior Manager Outreach & Service Delivery

Learning and Research Services, UQ Library
Paul MalanStudent RepresentativeUQ

Evaluation Panel

MemberRoleOrganisational Unit / team
Hari Jitindar SinghdOSCE Project Manager (Chair)ITS
Sam HarrisTeaching & Learning ManagerFaculty of Health and Behavioural Sciences
John RatiLearning DesignerFaculty of Business, Economics and Law
Kim HenvilleLearning DesignerFaculty of Engineering, Architecture and Information Technology
Dr Aaron Herndon LecturerFaculty of Science
Kellie BradyManager, Academic Programs and PracticesFaculty of Medicine

Evaluation Advisors

NameRoleOrganisational Unit / team
Dipak BhandariSenior ICT Procurement SpecialistProcurement Advisor
Stephen RossSenior Legal CounselLegal Advisor
Marc BlumICT Security ArchitectICT Cyber Security Advisor
Vern BawdenICT for IntegrationICT Integration Advisor
Sam MatuschekBusiness AnalystITS
Iliria StenningSenior eLearning AdviserITaLI, eLearning Systems and Support Team
Vidya BharatBusiness AnalystITS