With the PAF option students asked to evaluate peers individual contributions to group work by splitting 100 marks between the members of the group, including themselves for course co-ordinator set criteria.

For each student a PAF (Peer Assessment Factor) is calculated. After the student responses have been moderated the PAF can be used to calculate individual student assignment marks based on their contribution to the group assignment.

i.e. If the group assignment receives a mark of 80 out of 100 and students receive PAFs of

  • Student 1: 0.85
  • Student 2: 1.00
  • Student 3: 1.05
  • Student 4: 0.95
     
  • Student 1’s assignment mark = 0.85 x 80 = 68
  • Student 2’s assignment mark = 1.00 x 80 = 80
  • Student 3’s assignment mark = 1.10 x 80 = 88
  • Student 4’s assignment mark = 0.95 x 80 = 76

Warning: The PAF should only be used to adjust individual students’ results if the process has been clearly explained to students and included in the ECP. Moderation of the student responses is essential to assure PAFs assignment to individual students represent their performance. A small minority of students will try to “game” system to give themselves a higher PAF than is warranted or potentially, to penalise other group members. 

Calculation of the Peer Assessment Factor (PAF)

The PAF is calculated by the sum of all scores attributed to a student individual by the number of criteria multiplied by 100.

PAF formula

The above formula assumes all students have completed the assessment. If a student has not completed the assessment the equation becomes:

PAF formula - not all students have submitted

Interpretation of the Peer Assessment Factor (PAF)

A student who pulls their own weight in a team and who does similar amounts of work to everyone else will achieve a peer assessment factor (PAF) of 1.0. Students who lead the team and who are recognised as putting in extra work will achieve a PAF of above 1.0. It should be noted that in order for students to reward someone that they believe is doing extra work, they have to take points from another student who may not be ‘free riding’ but who is perceived to not be doing quite as much as the rest of the team. Therefore it is quite common for students to receive PAFs slightly below 1.0 but not to be in danger of failing as their group members have ‘robbed Peter to pay Paul’.

PAF Interpretation Comments

>1.5

Alarm! Team failure Something has gone wrong – either there is a student who is not participating at all or this student has taken all the work home and done it by themselves. Either way, learning objectives are probably not being achieved.
1.15 - 1.5 Super Leader The team balance probably needs to be addressed as to achieve such a high score other students must not be participating or this student is doing far too much.
1.05 - 1.15

Leader

The student is showing definite leadership qualities and/or has been putting in significant extra effort.
1.00 - 1.05 Good teamwork The student is working well with the group and has been recognised as pulling their weight (1.00) and perhaps a little more (>1.00).
0.95 - 1.00 Acceptable teamwork

This student has probably only been penalised because another team member has shown leadership and put in extra effort.

0.85 - 0.95 Social Loafer Any PAF below 0.95 is unacceptable. Social loafers who lie in this band can usually be mentored with the group’s help and become productive members of the group.
0.75 - 0.85 Super Social Loafer As above and below.
< 0.75 Alarm! Individual failure! The individual is in grave danger of failing the course. Much work is required for this student to be accepted back into the group and there will be trust issues with allocating this student any work.

Use of SAPA (self assessment over peer assessment)

The SAPA (self assessment over peer assessment) gives an indication of how realistically students judge their individual contribution to a group project. It compares the score a student allocates themselves against the average that all the other students have given them. The SAPA can also be used to highlight students who are trying to “game” the system.

The SAFA is the score that the student has given themselves over the average that all the other students have given them.

SAPA

PAF Interpretation
> 1.0 Indicates that the student believes they do more in the group assignment / project than their group members think.
The student may also be trying to “game” the system to improve their PAF.
1.0 Student has a realistic judgement of their contribution to the group assignment / project.
<1.0 Indicates the student undervalues their contribution to the group assignment / project.

Ratio

All individual scores assigned to a student are also compared those assigned by the rest of the group. The Ratio is calculated the same way as the SAPA.

Ratio

A ratio of over 1.0 indicates that Student A values the works of the individual more than the rest of the group and this is common between friends. A ratio less than 1.0 indicates that Student A may have a personal conflict with the individual and undervalues their contribution.

Limit

The limit is used to highlight groups where there is disagreement between members with the scores allocated to individuals.

The limit is set to highlight groups where there is deviation from the average (1.0) of the individual SAPAs and Ratios.

The default is Limit = 15%. i.e. Highlights in yellow groups where there are SAPAs and Ratios great then 1.15 and less than 0.85.

The deeper the yellow colour the more members of the group have SAPAs and Ratios that fall outside the limit set.

Moderation

It is recommended during the moderation process that all student results are checked.

The use yellow highlights (based on the Limit) to identify groups where there is disagreement between members with the scores allocated to individuals.

Check unusual PAFs by taking into account all of the contributing factors including the comments that students gave for their division of points. Tutors may also be able to provide supporting knowledge for unusual PAFs. In some cases it may be necessary to meet with groups or individual students to check on the contributions of group members to the assignment / project.

It is important that you can justify PAFs if they are contested.

The tool allows you to remove individual score students have allocated or all the scores given by a particular student. The tool will automatically recalculate the new PAFs. i.e. Students who try to “game” the system can have scores they allocated removed from the calculation. Recommendation: It is good practice to inform student that scores that are outside the Limit set (default 15%) can be moderated.

Based on: A peer assessment tool for teams, Associate Professor Lydia Kavanagh