eExams pilot in the Faculty of Medicine

Throughout 2016, the Teaching and Learning Committee and Faculty of Medicine co-funded a proof-of-concept pilot within the medical program. This pilot was designed to test the concept and viability of invigilated, on campus, bring-your-own-device (BYOD) based online exams (e-exams). The Faculty of Medicine partnered with ITaLI, ITS, and UQ Examinations for governance and practical support where required.

The pilot was a small opt-in trial in a first year medicine course, Clinical Science 2 (MEDI7112): Participants undertook one formative exam before sitting both the mid-semester and end-of semester exams on the e-exam platform.

A well-known e-exam platform was identified for the proof-of-concept. This was selected based on a brief comparison of three similar applications using broad functionality identified within the school of medicine as key for a successful proof-of-concept. These were:

  • Blueprinting and item banking;
  • Multiple question types, including rubrics for OSCEs;
  • Secure multi-platform exam delivery;
  • Integration with Blackboard;
  • Provision of timely and detailed student feedback;
  • Cost effectiveness;
  • Offline exam delivery (allowing for lock-down of network and device applications during exam);
  • Customisable analytics;
  • Collaborative online question submission, review and marking;
  • Ability to link assessment to program-level outcomes.

​​A summary of the overall process is:

  1. Existing exam questions are imported into or new questions directly created within the platform;
  2. Questions are tagged with one or more categories to facilitate feedback and analytics;
  3. Questions are assembled into exams within the platform;
  4. The exam is moderated by the same academic staff who would usually moderate exams;
  5. The exam is finalised and approved;
  6. An email is deployed to students involved in the pilot from within the application notifying them that they can download the exam;
  7. Students download the exam prior to the exam date - the exam file itself is password protected and students cannot access the exam content;
  8. Students attend the exam venue, are provided with the password and complete the exam;
  9. Feedback is provided through the application regarding student results including strengths and opportunities report across the categories tested.

The project discovered a range of policy and process challenges for introducing e-exams at UQ.  

Examples include:

  1. The development and implementation of a school/faculty BYOD policy;
  2. Identification of suitable physical spaces for online exams (power outlets, seating arrangements etc);
  3. Changes to standard examination procedures – how they change, how to prepare staff and students for those changes;
  4. Developing a back-up plan in case of device or system failure – i.e. available loan devices, and paper copies of exams.

Pilot in 2017

The Project Team is applying for additional funds to continue the proof-of-concept throughout 2017 with a view to investigating the level of interest in a broader university-wide deployment.  If broad interest exists, stakeholders will be identified to contribute to the development of a full and representative list of functional requirements.

For UQ to consider broad deployment of any application, a full and proper technology evaluation is required.

UQ staff interested in updates or specifics regarding the pilot should contact Dr Tammy Smith, Faculty of Medicine.

UQ staff interested in contributing to a broader user needs analysis should contact Elizabeth Wardrop, eLearning Systems and Support.