Practical Guidelines for Designing Rubrics (Original)
Before you start
Review the learning objectives
Check the assessment tasks you have designed provide adequate opportunity for students to demonstrate intended learning objectives.
Clarify your understanding of the terms "criteria" and "standards"
A clear understanding of these terms will make the development task easier. Salder (1987) defines these terms as:
Criterion: A property or characteristic by which the quality of something may be judged. Specifying criteria nominates qualities of interest and utility but does not have anything to offer or make any assumptions about actual quality.
e.g. “Fluency of expression” which does not specify a quality. Avoid terms such as “fluent expression” or “expression is fluent” or “expresses ideas fluently” which imply a level or standard.
Standard: A definition level of achievement aspired to or attained.
e.g. The standards described in the below example illustrate three distinct levels of quality, achievement or performance.
Locate useful resources
UQ resources
- ITaLI Assessment website
- Part H Grading System
- Graduate Statement and Graduate Attributes Policy
- Faculty / school resources (e.g. guidelines, models and exemplars)
Personal resources
- Course Profile
- Course Assessment program
- Examplars of student learning at different levels.
Other resources
- Brookhart, S. M. (2013). How to Create and Use Rubrics for Formative Assessment and Grading. ASCD.
- Exemplar rubrics: http://www.aacu.org/value/index.cfm
Workshops
Investigate possibilities for collaboration
Developing criteria and standards in collaboration with colleagues is a good opportunity to share the workload and ensure consistency across courses.
Getting going
Select a rubric type
Standard: Criteria are weighted using percentages.
A total score is automatically calculated, out of the mark for highest standard (i.e. for the below example a mark out of 4 would be calculated). The total score is also calculated as a percentage.
Custom: A score is allocated to each criterion – standard “cell”. Each criterion can be set to a different number of standards by setting extra “cells” to zero.
A total score is automatically calculated (i.e. for the below example a mark out of 14 would be calculated). The total score is also calculated as a percentage.
Quantitative: no scoring is included in the rubric, though an overall mark can still be entered.
Select / develop and organise criteria
Criteria are intended to help make it clear to students what factors will be considered when marking assessments. Criteria should be:
- Linked to learning objectives
- Observable: Describes a quality that can be perceived (seen or heard usually)
- Descripting qualities: (critical appraisal, structure and organisation, use of writing conventions) not assessment components (introduction, body, conclusion).
- Complete: If “creativity” or “spelling and punctuation” are considered important they should be included in the criteria.
- Distinct: Each criterion identifies a separate aspect of the learning objectives being assessed.
To be effective, criteria should be manageable in number. Criteria should include significant elements of the task but remain feasible for both students and markers.
Note: the number of characters for the criteria “title” is limited in TurnItIn, but a description can also be added underneath.
Number of standards
The number of standards is a balance between using fewer levels that ensure a greater chance of consistency and agreement between markers and include enough discrimination for accuracy. Most markers are able to make only 3 or 4 valid distinctions among passing standards.
TurnItIn does not technically allow you to use mark ranges, so practically you may need to include extra standards to account for this.
Each assessment item does not have to be marked using the same number of standards as the final grade. If you use standards that correspond to UQ grades, ensure it is possible for a student to demonstrate a “UQ 7”.
Scoring
Scoring of a rubric should not be designed around the particular weighting of an assessment item. i.e. An assessment item worth 30% does not have to have scoring that totals to 30.
Note: TurnItIn allows scores with two decimal places (i.e. 2.5, 3.75) although the final total mark returned to the Grade Centre is rounded to a whole number.
Practically, if you need the pass mark to be at least 50%, 3, 4 or 7 levels of standards are the easiest to implement.
Examples (Standard rubric)
Examples (Custom rubric)
The criteria are weighted by multiplying the base scale (i.e. 4,3,2,1 in the example above) by a number, e.g. 0.5 that halves to 2, 1.5, 1, 0.5 or 2 that doubles to 8, 6, 4, 2.
Note: Not all criteria need to use the full scale, e.g. References in Pass (2) or Fail (1).
Standard labels
TurnItIn requires that each standard has a text label. The labels need to be linked in some way to the model to be used to calculate final grades so that it describes and supports this process.
Note: Do not use UQ grade labels unless the standard descriptors are directly linked to the UQ grade descriptors.
Examples
UQ grades | Grade 7 | Grade 6 | Grade 5 | Grade 4 | Grade 3 | Grade 2 | Grade1 |
Labels | High Distinction | Distinction | Credit | Pass | Fail | Low Fail | Non-attempt |
Labels | Excellent | Advanced | Competent | Fail | |||
Standard descriptors
Standards must be described in brief, clear, specific language that is accessible to students. Accept that standards will never be able to carry all the detail of the explicit and implicit understandings students are to develop.
Four levels of standards could be described by using the following questions:
- What is the best possible standard that can be anticipated in this learning environment?
- What is the least standard that will be considered acceptable?
- What standard is between these two?
- What standard can be anticipated as unacceptable?
When describing standards | Use… | Rather than… |
|---|---|---|
Specify demonstrable behaviours | Rephrases problems in own words and identifies major issues | Understands and interprets problems |
Describe the behaviour – not the student | The ideas of others are not appropriately acknowledged. | You don’t know how to reference properly |
Express behaviours in positive terms whenever possible | Argument consists of a series of assertions only | No supporting evidence provided for arguments |
Avoid vague terms which are open to a wide range of subjective interpretation such as “critical”, “appropriate”, “excellent”, “analytical” | Evidence of familiarity with recommended course reading Analysis demonstrates an awareness of the implications of significant detail | Evidence of appropriate reading
Sophisticated analysis
|
Use terms likely to be understood by students – avoid the obscure or esoteric | Demonstrates comprehensive and detailed knowledge of major facts, concepts and procedures addressed in course materials | Secure and pronounced knowledge
|
Avoid relative terms - comparatives are rarely helpful without a benchmark standard | Major issues are identified with discrimination and without the distraction of irrelevant material Solutions to problems are original and/or innovative without losing feasibility | Analysis is more analytical
More creative solutions offered to problems presented |
Ensure a balance between validity and reliability i.e. don’t seek precision through quantitative statements which can trivialise complex learning outcomes. | References included have limited relevance to the problem (low standard). Discerning selection of references from within and beyond recommended course materials (high standard). | Includes two references (low standard)
Includes more than six references (high standard)
|
Determining "out of" mark
TurnItIn allows you to set Points value, (the mark the assignment is out of) that is returned automatically to the Grade Centre. i.e. the raw mark will be converted so it is out of the Points value you set.
Note: TurnItIn will only return whole numbers to the Grade Centre so the marks are rounded.
Some options for the Points value are:
- The mark the rubric is out of.
- Use a Points value of 100 so the mark is stored in the Grade Centre as a percentage.
- The weight of the assessment item i.e. Use a Points value of 30 if the assessment item is worth 30% overall.
- Use the Points value of 7, so the mark is stored as a UQ grade.
If you require a mark that is not rounded:
- Use a custom rubric and only use whole numbers.
- Set Points value to the same as the rubric total.
- Download the Grade Centre to a spreadsheet and convert the mark so it is out of the correct total.
After you have finished
Check, test, review and revise
Check for:
- Grammatical consistency.
- Alignment with UQ grade descriptors to maintain standards.
Test:
- If possible, mark old assignments (or ones on a similar topic) using the rubric and adjust as needed.
Regularly review your assessment criteria and standards to:
- Improve the rubric based on you developing expertise and that of your peers.
- Respond to feedback from students.
- Ensure they fit adjusted assessment tasks.
Maintain currency with changing university policy and regulations.
Using criteria and standards (students)
Provision of a rubric alone does not guarantee students will understand the quality of work required for an assessment task. Rubrics should be used in conjunction with exemplars if possible.
Rubrics can also be used as a teaching and learning tool not just for evaluation of assessment i.e. Self and peer assessment.
When marking
Standard descriptions will not perfectly match student achievement, you need to select the “most like” description. Give further feedback on how the work meets the criterion in your general comment.
The use of a rubric does not guarantee the reliability of marking. To ensure assessment judgements are defensible, consistent and transparent rubrics should be used in conjunction with:
- Exemplars of different standards
- Marker training
- A moderation process
Tip: While marking make notes of how to improve the rubric for the following semester.
Developed by Dr Clair Hughes, additions by Ailsa Dickie