Anonymous marking- the truth behind the candidate number
The UK government is very keen to get people to go to university. During my search for information for this article, I have inspected several university prospectuses. For some reason, no university prospectus mentions the issues discussed on this site. No prospectus mentions that they may (or may not) have staff who can abuse the power to hinder your progress and if they do the university takes no action against them.
There is information that I hope all bodies representing students can use to gain fairer marking and a better appeals system.
The National Union of Students is campaigning on the issue of unfair marking. (1)
What is anonymous marking?
Under a system of anonymous marking candidates’ names are supposedly not meant to be visible to examiners. Candidates are meant to put an anonymous candidate number on them so in theory, the examiners don’t know who the candidate is. This website shows the flaws in the system.
Why is there a need for anonymous marking?
In the absence of anonymous marking those with non-white names are likely to get up to 12% lower marks according to the National Union of Students (1) this may well occur at GCSE and A level as well. I am sure that there may well be such bias against female students. When I was a medical student I remember one consultant openly telling us that he hated female doctors and most of all female medical students. I dread to think what would happen if he was an examiner.
The system where students are marked by name is not even in the least bit remotely transparent and is open to abuse. Yet anonymous marking has all the defects of the marking by name system..
Bias doesn’t exist, does it?
Sadly it does.
I can see students of white origin- especially in medicine is “what evidence do you have of racism or examiner bias? It doesn‘t exist.”
Undergraduate education in courses allied to medicine takes place in the NHS.
Well consider the words of Sir John Blofield QC the high court judge “the NHS is riddled with institutional racism” is damning.
It is also pertinent to note that these very same people will be examining candidates at all levels. A student in such an environment will be the most vulnerable given that they are the most junior.
Is it any wonder that ethnic minority students in courses allied to medicine there are disproportionate failure rates? When I was a student at Sheffield this was common.
It isn’t in a university’s interests to fail people on purpose.
I can also see students, especially in medicine, saying that it isn’t in the interests of a university to fail a student on purpose. It is always, without exception, in the interest of the university that students should pass exams. People who engage in discriminatory practices are contravening the university’s interests and add their colleagues' workload. Unfortunately, the link between knowledge and behaviour is not always so obvious. If it were so, then no person would ever smoke, take drugs etc. For example how many doctors smoke?
What is written in this site is partially based on my experiences at Sheffield university. It may (or may not) be possible that similar things have happened at other universities. In July 2003 the Times Higher ran a story called Almost 40% fail to comply with race laws
It is not just in medicine where racism/bias can occur. On 24 November 2000 the Steel Press Issue 36, 24th November 2000 ran a story called Racist Slur exposed (2). This article described how Sheffield university law lecturer Margaret Wilkie was accused of marking derogatory and offensive reference to black people twice in lectures. The Steel Press had heard a tape in which she made a remark about a “nigger” and referred to Nigeria as a White man’s grave.
Note that people like Wilkie may assess students.
Embarrassment if universities don’t follow it
The fact that some institutions have it is an admission of bias. Firstly there is the issue of the Commission for Racial Equality investigation into Sheffield University School of medicine- click and I have detailed it elsewhere on this website.
The factors affecting student exam performance.
The universities academics will all say that there is only one factor that determines a student’s performance- the amount of work that they have done.
(From Medical Student Stress- Sushant Varma)
Most of these are obvious
The Candidate
Knowledge.
Understanding
Handwriting- clearly if your handwriting is unreadable the examiners can’t mark it.
Exam technique
Ability to think logically
Mitigating circumstances
Stress
Confidence
Support from friends and family
The Teaching body
Teaching, both quality and quantity
Formative assessment (3).
Pastoral care — tutorial system and back up support (This is also related to the way the examining body give back results)
Feedback on performance
The examining body
Examiner prejudice
Examiner subjectivity
Procedural errors
Mistakes
Organisation
(Both Organisation and mix ups have been highlighted by the Scottish exam boards GCSE exam fiasco.)
Fixed marking scheme
The method of giving results. Traditionally in medicine pass and fail lists are put on up a notice board by name. This humiliates the candidate who has failed. This can lead to a self-fulfilling prophecy of failure.
Formative assessment is essential for education to occur. Here are further benefits.
The Benefits of formative assessment (From Rolfe and McPherson April 1995)
It assists students to evaluate their knowledge and understanding, practice their skills and consolidate their learning.
It assists students to define the expectations of their teachers, including the level of competence required.
It assists students to identify individual strengths and weakness without incurring academic penalty.
It provides opportunities for rapid feedback and action to remedy student deficiencies.
It assists teachers to evaluate and modify their coursework and teaching.
It assists teachers to recognise student progress and achievement.
It encourages interaction between staff and students.
In addition it can even help identify students who are having problems whether they be personal or academic if a “blip” occurs in the standard of work submitted
If the marks from formative assessment are recorded then there are two additional benefits.
It can be used to measure the progress of the student throughout the academic year.
In cases of where students have mitigating circumstances at the end of year exams then it is easier to for the examining department to grant a pass mark knowing how the student has done throughout the year.
In my view any university department that doesn’t have sufficient formative assessment is being negligent.
This is an extract from an email from David Newble- Professor of Medical Education at Sheffield to me “Students need good feedback and this is best done using ways which are specifically designed for this purpose and at a time which enables them to do something about it. Such assessments we call formative and their main value is to the student. We do not have enough of
them and I hope we will do more in this regard during our curriculum
changes.”
The Universities’ arguments
Our examining processes are fair
Firstly let me rebut the universities arguments saying that their marking is fair.
We have external examiners
The Universities will no doubt say “we have independent external examiners
As part of a national system, all institutions use External Examiners to assist them in monitoring the standards of all of their degrees except those granted on an honorary basis. External Examiners are so called because they act as Examiners (alongside other examiners appointed from among the staff of the University) and, because they are from outside the University, can provide an objective view to the nature and standards of the assessment of students. They are appointed to act as independent and impartial advisors providing informed comment on the standards set and student achievement in relation to those standards and to standards of comparable institutions elsewhere.
The main duties of an External Examiner are: to verify that standards are appropriate for the course(s) concerned; to assist institutions in the comparison of academic standards across higher education degrees and parts of degrees; and to ensure that their assessment processes are fair and are fairly operated and are in line with the institution’s regulations.
Elsewhere on the internetI will show the flaws of the external examiner system. Were these external examiners- Professor Mortimer from Hull University, Professor Mindham from Leeds and Connell from Glasgow acting independently in my case?
Students can always appeal
The universities will also say that there are appeals processes.
This is the process by which a student who has done less well in an assessment than he/she feels he/she deserves. When I was a student you would have to send a letter to the dean within 10 days of getting the result outlining your reasons. The dean would then decide if you had a case or not. Under such a system you could appeal on two grounds, mitigating circumstances and academic irregularities. Academic irregularities basically meant something had gone wrong in the way the assessment had taken place. If you had failed on more than one occasion and your appeal was unsuccessful then you would face a review committee (also known as a review board.)
You can’t appeal on the grounds of examiner bias because of the myth of the “anonymous marking” system. For some reason at postgraduate level you can appeal on the grounds of examiner bias but at undergraduate level you can’t.
The problem here is that you write to the dean. As this website will show the dean will always back his own colleagues to the hilt. You don’t get a hearing and you don’t get to see your papers. In effect you are shooting in the dark.
In some circumstances you may face a review panel.
Student review
If you had failed an exam on more than one occasion then a student would face a review committee. You would have four options. To withdraw from the university, to make a submission by letter, to attend in person or to do nothing. Of course, you could make a submission by letter and attend in person.
Basically, the course tutor would put his/her evidence forward in a statement, which you would receive about a week before the hearing-typically at a weekend to make sure you worried over a weekend. You would be given some time in which to respond. At the hearing, the course tutor would put the case forward- in my experience, they lie through their teeth. The committee would ask the course tutor questions. You would then put your case forward. The committee could then ask you questions. They’d then ask you to leave the room and they’d make their decision.
The FSRC had four options
To allow you to proceed unconditionally
To proceed with certain conditions
To be allowed another attempt
To exclude the student
Usually, if the course tutor recommended another attempt you’d be granted one. If the course tutor did not then you would be excluded.
If you were excluded then the student had the right to appeal to the senate.
In my experience they are nothing but pre-determined kangaroo courts unless you manage to have the means to fight back- not many have the means to do this. If you face such a review you need a lawyer. When I was a student I was told that they occur most commonly in the medical faculty and it was the non-white students who were the most frequently called students. Educational lawyers are hard to come by.
The fundamental difference between the appeals process and the review process.
The student review procedure is more open and fair than the appeals process however there is a fundamental difference. The appeals process is initiated by the student. The student review procedure is initiated by the university. A student review does have the power to recommend a change in the examination mark to the board of examiners. You also get a hearing with student review. You don’t with the appeals process.
The problem here is what happens if a dean or a lecturer falsifies your results.
The drawbacks of “anonymous marking” unless papers are returned.
If you read the above site you will see that at the time Sheffield university had an anonymous marking system in place but there are drawbacks.
Forgetting your anonymous number
At some universities, students have to write their university registration number on their examination scripts. The problem here is some people forget what it is. At some institutions, it is on your student union card. I must digress and tell you of one occasion. A housemate of mine (who will remain nameless) went to an exam starting at 0930. At 0925 I noticed that she had left her union card that had her registration number on behind. By this stage she would have been in the exam room. I had to rush down to the exam centre myself go to the exam room and tell the invigilator who gave my housemate her card so she could put her number on.
In the horrific stress of an exam, it is very easy to forget to take your registration number. This logically brings me to the next point.
Not using the proper number
I have a taped interview with Professor Weetman (during my research year) who told me that students don’t always put their anonymous numbers of their papers. Sometimes you get bank account numbers, Switch cards, visa numbers, mobile phone numbers, even passport numbers have been used. He actually said “You won’t believe what we get.”
At first, I didn’t believe him so I checked. It proved to be correct. The best I ever saw was a student who put the serial number of his television set!
People don’t normally forget their names and I think people should get a mark for putting their names on their papers!
Flouting “anonymous marking”
It is very easy to flout.
For example, Dr Peters (then undergraduate course tutor in psychiatry and now undergraduate dean of Sheffield university school of medicine) would openly admit to having the list of names and numbers before the papers were marked. He would insist that you wrote your names on your papers and openly call it a “silly system.” He would do this openly, with impunity and with the university’s full knowledge. When I met Mr. Page (then undergraduate dean) on 23 October 1997 he admitted that Peters was flouting anonymous marking.
Ironically one week later Page wrote a letter to Richard Allan MP saying that the medical school adhered to the “anonymous marking” system- he knowingly misled the MP- see (4)
Ironically at a student review hearing the flouting was brought out. The chair Professor Woods the dean said “Although this is an academic review I have to remind everyone here that anonymous marking is not discretionary. Anonymous marking is a mandatory part of the university regulations and must be observed in all written examinations at this university. The faculty of medicine is unusual in that we have clinical exams and vivas that are conducted face to face and therefore cannot be anonymous. However the rule of anonymous marking applies. There are no exceptions to this rule. I do not want to have to repeat myself about this matter again.”
When I did psychiatry in my group of 61 students 11 were of ethnic minority. Of the 9 students who had to do a repeat clinical exam (where there is no anonymity) 8 were of foreign background. When Mr. Webster from the Commission for Racial Equality wrote to me he was very concerned about obstetrics and Gynaecology and paediatrics. However when I told him of Peters he was very concerned. Peters was promoted to undergraduate dean. At the resit Dr Peters told us he had been asked by the university not to mark the papers as there were allegations of racial bias. Despite that he still marked the papers and flouted “anonymous marking.”
Indeed when I did my final Objective Structured Clinical Exam we had to give stickers to the examiners.
SUSHANT VARMA
922073201
GROUP O
START AT STATION 6
This is a clear violation. Just as I walked out of the exam Dr Peters walked in. Why was he there? He had no legitimate reason and he could easily have had access to my papers.
Are others doing the same now? Will others do the same if this system is brought in? From the experience of being in lectures with Dr Peters it is very easy to intimidate students into this.
A false sense of security.
As stated in the above website although Mr. Page said “any academic member of staff with a will to identify the name of an individual form their registration number could do so but when faced with having to mark nearly 200 or so scripts to a tight deadline would waste time doing so.” He did not mention the useless confidentiality flap nor the fact that not every examiner would mark 200 scripts. The question is what about students on courses where there are not so many students? On top of that what about resits where there are very few students? In medicine there are a disproportionate number of ethnic minority students at resits. Given that scripts would be put in numerical order it would be very easy to find them.
Singling out individual students
Worse still I have seen cases where individual students have been singled out. What they used to do in Sheffield was during the exam you would have to fill in a yellow slip which would have your desk number, registration number and your name. If someone was so motivated they could in theory come to your exam room, find what desk you were at, look for the registration number and then trace your papers. This actually did happen to a friend of mine.
From a personal perspective in this case under my advice this person rang the Vice Chancellor after being victimised. Inadvertently she mentioned my name. When she did the VC’s assistant went very quiet.
When I did psychiatry I was singled out and I was the only candidate as such the examiners knew who I was. Where is the anonymity?
Confidentiality flap
Some universities use the confidentiality flap system. While this system is a step (but only a step) in the right direction when I was a student it was next to useless as it was impossible to seal down. Even if such a system is brought in there is no guarantee that the examiner won‘t open up the flap himself. Unfortunately, there are unscrupulous examiners who will do this (although the number is not known). However, even with such a system, it is still open to abuse for reasons described on this website. There is also the issue of an unscrupulous examiner recognising a particular student’s handwriting.
Indeed I remember in one course the course tutor admitted to me that they would open up the flap before marking. Then again the confidentiality flaps we had were useless. However, if your university uses such a system then tape it down.
I do accept that at the end of the day marks have to be allocated to candidates. What I would say is that the marking of examination papers should be done under strict supervision in exactly the same way as candidates sit exams. Then and only then can the flaps be opened by a non-academic and when the examiner has left the room.
The way round anonymous marking that they don’t want you to know about.
At Sheffield University they admit to a flaw on the website (5) The full text is here as it is possible that they may change the website.
i. School of Medicine
· Information linking students’ registration numbers to students’ names is retained by the School’s administrators only until the whole marking process is complete.
· The examination scripts to be marked by each examiner are sorted by numerical order of the student registration numbers by the School’s administrators. The scripts are then given to the examiner along with a mark sheet for completion listing the students’ registration numbers in the same numerical order.
· Mark sheets are returned the School’s administrators who will put them into the master spreadsheet which will identify students by both their names and registration numbers once all the marks are available for discussion by the examining board.
Here there is an obvious flaw. The numerical order of candidate number increases by increments of one. However for obvious reasons the list of students is kept alphabetically. The following will illustrate: (you may recognise these names)
Montgomery Burns 123456
Kent Brockman 123457
Ned Flanders 123458
Barney Gumble 123459
Reverend Lovejoy 123460
Nelson Muntz 123461
Lisa Simpson 123462
Seymour Skinner 123463
Clancy Wiggum 123464
Therefore if you wanted to identify a particular student it would not be difficult no matter what Mr. Page said in his letter to Richard Allan MP. It is very easy for an academic to find a list of candidates by name. If an examiner wanted to fail Ned Flanders all he would need to do (assuming that the scripts had been placed in numerical order) get the 3rd paper deliberately downgrade the marks and unless the papers were returned nobody would ever know. For 200 or so candidates it is a simple job to find what would be the numerical appearance of a script in the pile of scripts to be marked.
That said it is just as easy for an administrator to put a post-it note on any student who they want to deliberately downgrade. After marking all that is needed is to remove the post-it note and nobody would ever know.
This reminds me of an incident when I did paediatrics. I was at the Sheffield Children’s Hospital and some student coursework had been submitted for speech science students. An academic had to mark some coursework that only had the anonymous number on it. However, I noted that each piece of coursework had a post-it note with the student’s name on it. I was horrified at that. If only I had a camera!
Because of the actions of lecturers at Sheffield, I had to repeat a year. As such have a look at this
Sushant Varma 921234567 (showed that I started in 1992)
A Aardvark 951234568 (showed that this person started in 1995)
The issue here is that such a person’s script could, in theory, be the first on the pile of scripts to be marked if they are to be kept in numerical order. As such it would have been very easy to single out anyone in that position.
ii. Department of Law
· Students are asked to identify their work using their registration number upon submission.
· Examiners are provided with a disk which contains a list of the students’ registration numbers alongside which marks should be input.
· The disks containing the marks are returned to the department’s administrative staff to be downloaded into the master spreadsheet of marks, which will also identify students by name as well as registration number.
Again the same would apply.
This is not limited to Sheffield. Here is an extract from the University of Manchester Website (6) an archived extract from the University of Manchester website to see the original website. (The text is here in case they decide to change it).
It is very easy to see how the same could occur.
Mistakes
Even with “anonymous marking” there is scope for errors as the following will show.
Jack Sanders 123456789
Jacky Sanderson 123456790
Jane Sellers 123456791
John Smith 123456792
June Snape 123456793
(These are fake characters)
Note the initials and the similarity in number. It is easy to see how they could get marks mixed up. In the case of ethnic minority students, it is even easier to make mistakes. It is ironic that the system that is supposed to protect against bias can actually lead to mistakes and possible allegations of bias.
The other thing that examiners can do is just to ignore certain parts of the script. If they are not marked then they can’t allocate marks and that way the student can be downgraded. If anything happens they have an excuse.
The A-level crisis where examination papers were wrongly marked highlighted this problem when students exam results got mixed up or wrongly marked.
Here are some such articles (7)-(12.)
If there has been a mistake the first person who will notice will be the student-see the ultimate solution below.
When such articles are published I always get a terrible feeling of “I told you so!”
Monitoring of failure rates by disadvantaged groups.
Although the CRE was very concerned about the lack of monitoring of exam failure rates the question of “Who does the monitoring?” arises. If you see the section on the CRE investigation or the solicitor‘s letter about Sheffield university then you will see what I mean.
In another article, I’ll talk about the ultimate solution.
Bibliography
(1) Mark my words National Union of Students https://www.dropbox.com/s/g8ztwqc1voogxvw/markmywords.pdf?dl=0
(2) Racist Slur exposed https://www.dropbox.com/s/wvppdvy00y0ezzw/racistslurexposed.pdf?dl=0
(3) What is the difference between formative and summative assessment?https://www.cmu.edu/teaching/assessment/basics/formative-summative.html
(4) Mr Page to Richard Allan MP 30th October 1997 https://www.dropbox.com/s/xg1rzcgas47n4s1/mrpage30october1997.pdf?dl=0
(5) University of Sheffield handbook http://www.shef.ac.uk/tlsu/handbook/28.html (date checked July 2005)
(6) https://www.dropbox.com/s/t54ng8voefqn6kt/manunfile.pdf?dl=0
(7) Exams body admits results error http://news.bbc.co.uk/1/hi/northern_ireland/1576582.stm 3 October 2001
(8) Wrong test results for 14-year-olds http://news.bbc.co.uk/1/hi/education/2290334.stm Tuesday, 1 October, 2002,
(9) Students’ angry A-level e-mails http://news.bbc.co.uk/1/hi/education/2292469.stm Wednesday, 2 October, 2002
(10) Students in A-level exam blunders http://news.bbc.co.uk/1/hi/wales/2045951.stm 14 June 2002
(11) A-level results missing http://news.bbc.co.uk/go/em/fr/-/1/hi/wales/3151461.stm 14 August 2003
(12) School’s A-levels wrongly marked 5 May 2005 http://news.bbc.co.uk/go/em/fr/-/1/hi/england/beds/bucks/herts/4517007.stm
