Before you start

Review the learning objectives

Check the assessment tasks you have designed provide adequate opportunity for students to demonstrate intended learning objectives.

Clarify your understanding of the terms "criteria" and "standards"

A clear understanding of these terms will make the development task easier. Salder (1987) defines these terms as:
Criterion: A property or characteristic by which the quality of something may be judged. Specifying criteria nominates qualities of interest and utility but does not have anything to offer or make any assumptions about actual quality.

e.g. “Fluency of expression” which does not specify a quality. Avoid terms such as “fluent expression” or “expression is fluent”  or “expresses ideas fluently” which imply a level or standard.

Standard: A definition level of achievement aspired to or attained.

e.g. The standards described in the below example illustrate three distinct levels of quality, achievement or performance.

standard rubric

Locate useful resources

UQ resources

Personal resources

  • Course Profile
  • Course Assessment program
  • Examplars of student learning at different levels.

Other resources

Workshops

Investigate possibilities for collaboration

Developing criteria and standards in collaboration with colleagues is a good opportunity to share the work load and ensure consistency across courses.

Getting going

Select a rubric type

Standard: Criteria are weighted using percentages.

A total score is automatically calculated, out of the mark for highest standard (i.e. for the below example a mark out of 4 would be calculated). The total score is also calculated as a percentage.

standard rubric scoring

Custom: A score is allocated to each criterion – standard “cell”. Each criterion can be set to a different number of standards by setting extra “cells” to zero.

A total score is automatically calculated (i.e. for the below example a mark out of 14 would be calculated). The total score is also calculated as a percentage.

custom rubric

Quantitative: no scoring is included in the rubric, though an overall mark can still be entered.

Select / develop and organise criteria

Criteria are intended to help make it clear to students what factors will be considered when marking assessments. Criteria should be:

  • Linked to learning objectives
  • Observable: Describes a quality that can be perceived (seen or heard usually)
  • Descripting qualities: (critical appraisal, structure and organisation, use of writing conventions) not assessment components (introduction, body, conclusion).
  • Complete: If “creativity” or “spelling and punctuation” are considered important they should be included in the criteria.
  • Distinct: Each criterion identifies a separate aspect of the learning objectives being assessed.

To be effective, criteria should be manageable in number. Criteria should include significant elements of the task but remain feasible for both students and markers.

Note: the number of characters for the criteria “title” is limited in TurnItIn, but a description can also be added underneath.

criteria scoring

Number of standards

The number of standards is a balance between using fewer levels that ensure a greater chance of consistency and agreement between markers and include enough discrimination for accuracy. Most markers are able to make only 3 or 4 valid distinctions among passing standards.

TurnItIn does not technically allow you to use mark ranges, so practically you may need to include extra standards to account for this.

Each assessment item does not have to be marked using the same number of standards as the final grade. If you use standards that correspond to UQ grades, ensure it is possible for a student to demonstrate a “UQ 7”.

Scoring

Scoring of a rubric should not be designed around the particular weighting of an assessment item. i.e. An assessment item worth 30% does not have to have scoring that totals to 30.
Note: TurnItIn allows scores with two decimal places (i.e. 2.5, 3.75) though the final total mark returned to the Grade Centre is rounded to a whole number.

Practically, if you need the pass mark to be at least 50%, 3, 4 or 7 levels of standards are the easiest to implement.

Examples (Standard rubric)

Examples (Custom rubric)

custom rubric example

The criteria are weighed by multiplying the base scale (i.e. 4,3,2,1 in the example above) by a number, e.g. 0.5 that halves to 2, 1.5, 1, 0.5 or 2 that doubles to 8, 6, 4, 2.

Note: Not all criteria needs to use the full scale, e.g. References in Pass (2) or Fail (1).

Standard labels

TurnItIn requires that each standard has a text label. The labels need to be linked in some way to the model to be used to calculate final grades so that it describes and supports this process.

Note: Do not use UQ grade labels unless the standard descriptors are directly linked to the UQ grade descriptors.

Examples

UQ grades

Grade 7

Grade 6

Grade 5

Grade 4

Grade 3

Grade 2

Grade1

Labels

High Distinction

Distinction

Credit

Pass

Fail

Low Fail

Non-attempt

Labels

Excellent

Advanced

Competent

Fail

 Standard descriptors

Standards must be described in brief, clear, specific language that is accessible to students. Accept that standards will never be able to carry all the detail of the explicit and implicit understandings students are to develop.

Four levels of standards could be described by using the following questions:

  • What is the best possible standard that can be anticipated in this learning environment?
  • What is the least standard that will be considered acceptable?
  • What standard is between these two?
  • What standard can be anticipated as unacceptable?

When describing standards

Use…

Rather than…

Specify demonstrable behaviours

Rephrases problems in own words and identifies major issues

Understands and interprets problems

Describe the behaviour – not the student

The ideas of other are not appropriately acknowledged.

You don’t know how to reference properly

Express behaviours in positive terms whenever possible

Argument consists of a series of assertions only

No supporting evidence provided for arguments

Avoid vague terms which are open to a wide range of subjective interpretation such as “critical”, “appropriate”, “excellent”, “analytical”

Evidence of familiarity with recommended course reading

Analysis demonstrates an awareness of the implications of significant detail

Evidence of appropriate reading

 

Sophisticated analysis

 

Use terms likely to be understood by students – avoid the obscure or esoteric

Demonstrates comprehensive and detailed knowledge of major facts, concepts and procedures addressed in course materials

Secure and pronounced knowledge

 

Avoid relative terms - comparatives are rarely helpful without a benchmark standard

Major issues are identified with discrimination and without  the distraction of irrelevant material

Solutions to problems are original and/or innovative without losing feasibility

Analysis is more analytical

 

More creative solutions offered to problems presented

Ensure a balance between validity and reliability i.e. don’t seek precision through quantitative statements which can trivialise complex learning outcomes.

References included have limited relevance to the problem (low standard).

Discerning selection of references from within and beyond recommended course materials (high standard).

Includes two references (low standard)

 

Includes more than six references (high standard)

 

Determining "out of" mark

TurnItIn allows you to set Points value, (the mark the assignment is out of) that is returned automatically to the Grade Centre. i.e. the raw mark will be converted so it is out of the Points value you set.

Note: TurnItIn will only return whole numbers to the Grade Centre so the marks are rounded.

Some options for the Points value are:

  • The mark the rubric is out of.
  • Use a Points value of 100 so the mark is stored in the Grade Centre as a percentage.
  • The weight of the assessment item i.e. Use a Points value of 30 if the assessment item is worth 30% overall.
  • Use the Points value of 7, so the mark is stored as a UQ grade.

If you require a mark that is not rounded:

  1. Use a custom rubric and only use whole numbers.
  2. Set Points value to the same as the rubric total.
  3. Download the Grade Centre to a spreadsheet and convert the mark so it is out of the correct total.

After your've finished

Check, test, review and revise

Check for:

  • Grammatical consistency.
  • Alignment with UQ grade descriptors to maintain standards.

Test:

  • If possible, mark old assignments (or ones on a similar topic) using the rubric and adjust as needed.

Regularly review your assessment criteria and standards to:

  • Improve the rubric based on you developing expertise and that of your peers.
  • Respond to feedback from students.
  • Ensure they fit adjusted assessment tasks.

Maintain currency with changing university policy and regulations.

Using criteria and standards (students)

Provision of a rubric alone does not guarantee students will understand the quality of work required for an assessment task. Rubrics should be used in conjunction with exemplars if possible.

Rubrics can also be used as a teaching and learning tool not just for evaluation of assessment i.e. Self and peer assessment.

When marking

Standard descriptions will not perfectly match student achievement, you need to select the “most like” description. Give further feedback on how the work meets the criterion in your general comment.

The use of a rubric does not guarantee the reliability of marking. To ensure assessment judgements are defensible, consistent and transparent rubrics should be used in conjunction with:

  • Exemplars of different standards
  • Marker training
  • A moderation process

Tip: While marking make notes of how to improve the rubric for the following semester.

Developed by Dr Clair Hughes (TEDI, The University of Queensland)

Additions by Ailsa Dickie (ITS, The University of Queensland)