UQ ePortfolio Product Selection

The ePortfolio project was initiated by Professor Sarah-Roberts Thomson, Associate Dean Academic of the Faculty of Health and Behavioural Sciences (HABS), funded by the Technology-Enhanced Learning Sub-Committee.

The project aims to investigate and address the need for a sustainable ePortfolio solution(s) that will enable faculties and schools to facilitate and capture students’ learning that occurs outside the normal lecture or didactic settings across the following faculties: Faculty of Health and Behavioural Sciences (HABS), Faculty of Humanities and Social Sciences (HASS) and Faculty of Medicine and Biomedical Sciences (M+BS).

The project seeks to rationalise several bespoke systems as well as numerous spreadsheets and ad-hoc methods currently used across UQ to manage existing processes, which are unsustainable in their current form, and are embedded in a wide range of disciplines across schools within the three Faculties.

The acquisition of a single commercial product that is centrally supported and meets this broad spectrum of requirements is the solution preferred by the project sponsor, the steering committee, and ITS.

Product Selection Method

A business user needs analysis was undertaken across the 14 schools using the bespoke and ad-hoc methods described above. All high level business processes were captured and documented, then carefully translated into a list of required functionality/product criteria to use in the product selection process. The project Steering Committee rated the functionality requirements according to priority, and a broad market scan was undertaken.

35 off-the-shelf products were identified as potential solutions, and 12 products were shortlisted as likely to meet the prioritised functional requirements.

Image showing functional requirements categories

The 12 shortlisted vendors were invited to respond in writing to describe the way in which their products would meet the criteria. The project team analysed and rated each response to each criterion. A panel of stakeholders reviewed the analysis and 3 products were further shortlisted as the most compatible with the user requirements. A closed eTender invitation was published and the 3 vendors invited to participate. All 3 vendors submitted a response to the eTender.

Evaluation phase

The evaluation phase is guided by the tender process. ITS conducted a full analysis of the 3 potential products, including technical analysis, potential integration into existing enterprise systems (such as Blackboard, SI-net, TRIM), licensing and implementation options, and contract negotiations. The project team analysed the systems against the functional requirements, and actively sought stakeholder feedback.

Vendor led product demonstrations were scheduled for project stakeholders to view and provide feedback on the 3 shortlisted products. These demonstrations took place on 28 July and 10 August. The presentations were part of the evaluation process, and as such were carefully designed to align to the project scope. Feedback was sought from the audience through surveys. This feedback was incorporated into the final product selection.

A test environment was established for a small scale proof of concept test, during which project stakeholders evaluated and provided feedback on the products. A number of workshops were been scheduled during which stakeholders were introduced to the main functionality in each of the products, and asked to provide feedback, this also contributed to the final selection.

The project then delivered a report making recommendations on a suitable solution, including identification of any additional custom functionality required to support the decommissioning of the existing bespoke systems. The final report included a Pilot Plan for wider school engagement.

Project Contacts: Elizabeth Wardrop (ITS); and Jessica Tsai (Faculty of Health and Behavioural Sciences)

Evaluation Process update

25/09/2015

The official evaluation period of the project has now finished. A summary of the evaluation completed is below:

Stakeholder Input

During the evaluation each vendor held product demonstrations for project stakeholders to view and provide feedback on the 3 shortlisted products. The recommended presentation content was designed to align to the project scope. Approximately 35 staff attended each presentation. There was an opportunity for questions at the end. Each presentation was recorded. Feedback was sought from the audience through the use of a survey.

A test environment/sandpit was established for each product and integrated into the Blackboard staging environment. Using this environment, the project team facilitated a number of workshops, and stakeholders were offered the opportunity to personally evaluate and provide written feedback on the products. 35 staff and 6 students attended.

This feedback has been incorporated into the final recommendation.

Functional requirements

Each of the identified functional requirements has been tested within each product by the project team, and other ITS staff. Clarifications were sought where necessary from the vendor, including online meetings held for further demonstrations in many cases.

The project team has individually rated each requirement to determine which product can most adequately complete the tasks and deliver the outcomes deemed essential by the end-users of the existing bespoke systems and ad-hoc processes.

Technical/financial/contractual analysis

Each vendor was asked to respond to a number of technical/financial and contractual specifications. An ITS panel has reviewed the evaluation outcomes. The rating system used by ITS in product selection is now being completed by each panel member.

Selection

The next Steering Committee meeting is scheduled for October 6^th, 2015. The Steering Committee will consider the following information:

Project Team summary of testing (functional requirements);
Stakeholder feedback summary;
ITS evaluation review;
Vendor references and qualifications/experience;
Licensing and consultancy fees.

It is expected that during this meeting the Steering Committee will decide on the solution for pilot in 2016.

Status Update

16/10/2015

The Steering Committee reached a unanimous decision on the outcome of the evaluation process on October 6th, as planned. UQ has now contracted to pilot the Chalk and Wire solution in two undergraduate programs throughout 2016. For more information about the solution and the pilot, please visit the ePortfolio Pilot Project page.