“21. …Knowledge and Skills are assessed on the basis of grades of Merit (M), Satisfactory (S), Borderline (B) and Unsatisfactory (U). Professionalism is assessed on the basis of either Acceptable or Unacceptable. The marks for Professionalism are treated in a different way from those for Knowledge and Skills. To pass a Stage a student must be in good standing in respect of Professionalism but the grade is otherwise not taken into account in assessing the overall grade.”
“56. …These numbers are summed for each domain. To balance the proportionate weight of Communication in Station 4 it is counted twice (thus ensuring that all four stations carry equal weight in the final calculation). The average is then calculated to give a domain total. In the case of the Skills domain this is n/12 (where ‘n’ is the sum of the marks given for all the Skills assessment criteria across all four stations) and in the Knowledge n/8 (where ‘n’ is the sum of the marks given for all the Knowledge assessment criteria across all four stations).”
“58. As can be seen from the figure above the Claimant received a Skills total for the MOSLER of 29/12 that was rounded up to 2.42 and he received a Knowledge total for the MOSLER of 21/8 that was rounded up to 2.63.”
“58. … M > 3.54; S > 2.54 & < 3.54; B > 1.54 & <2.54; U < 1.54.”
“58. … An average is used here to compare equally weighted stations…. each station in the assessment episode carries equal weight despite having differing numbers of criteria for which they are marked.”
“Determination of Outcomes for ‘Clinical and Communication Skills’ and ‘Knowledge and Critical Thought’ You are assessed by: • Summative Clinical Encounter assessment in all components of Stage 5 except Preparation for Practice. • Unseen examination in the End of Stage Examination. [Table setting out the domains which are assessed in the six in-course assessments and in the End of Stage Examination omitted] • For each assessment episode you are graded according to the M, S, B and U grading scheme such that at the end of the year you will have a final overall grade (M, S, B or U) for both Skills and Knowledge. • In order to determine a final outcome for Skills and Knowledge, a combination of grade averaging and Look-Up Tables (see below) are used. • To average grades, each grade will be converted to a numerical score (M=4, S=3, B=2, U=1) and average calculated and the number converted back to a grade using the following thresholds (M>3.54, S>2.54, B>1,54, U<1.54).”
“87. Students will receive timely and accurate guidance about assessments, including assessment format, length and range of content, marking schedule and contribution to overall grade.”
“The assessment of medical students is necessarily complex and difficult to describe.”
“(i) the University provides on p14 of the MBBS Stage 5 Handbook for 2011, a description of how the MOSLER grade in the skills domain is calculated. I accepted the description as clear and transparent, but the University calculated the grade in a way different from what is described in the Handbook. Had the University adhered to its own published description for calculating the MOSLER grade in skills, I would not have failed my exams and would have qualified as a doctor”
“There appears to be a misunderstanding on Mr Crawford’s part as to what is described on page 14 of the Stage Handbook (Appendix 1). Page 14 details the way in which the grade from the MOSLER is combined with other grades within the Knowledge domain. It does not describe how the MOSLER grade itself is determined. The contribution of the MOSLER to the Skills and Knowledge domains is different. Pages 13 and 14 of the handbook details this difference and clearly states how the MOSLER grade is combined with other assessments in both the Skills and Knowledge domains to result in a final Skills and Knowledge grade. The regulations stated in the Handbook were followed. Mr Crawford’s grades were calculated in line with the published regulations within the handbook. I believe that Mr Crawford has misunderstood the information presented.”
“It would be my presumption that when the word ‘average’ is used then a simple arithmetic average is meant. When an alternative such as geometric or harmonic average is to be used then it would be normal to specify that fact As an exercise I repeated the calculation with geometric and harmonic means rather than arithmetic means and produced higher numbers, 24 and 40 respectively, of discords so I conclude that a different interpretation of ‘average’ is not the answer to the discrepancies.”
“Mr Carus has clearly gone into great detail in his report and if his assumptions regarding the calculations of marks for the MOSLER examination were correct, then his outcome would have been correct. However, Mr Carus has based his calculations on erroneous assumptions. Mr Carus has assumed that each of the individual criteria feeding into the skills and knowledge domains are equally weighted. This is not the case. The communication criterion within Station 4 is double-weighted. When the MOSLER examination was developed, it was designed with Station 4 being different to Stations 1 to 3. Station 4 was to focus on a candidate’s advanced communication skills. For this reason, it was decided to double-weight the communication criterion within this station. This also allowed for Station 4 to be equally weighted in terms of its contribution to the examination as a whole. Stations 1 to 3 each contributed 5 individual criteria or marks to the overall mark. Station 4 did not have an examination criterion and therefore would only be contributing 4 criteria or marks to the overall grade. The validity of double-weighting the communication criteria therefore was confirmed. The MOSLER has been running now for a number of years. The way in which the examination has been marked and results calculated has not changed. I can confirm that the marking and results calculations for this year have not changed. As stated in my previous response to Mr Crawford’s appeal, the Stage Handbook contains information on how final examination or assessment grades are combined. The Stage Handbook (sic) does not provide information on how individual criteria or marks within a single assessment or examination combine to provide the final examination or assessment result. It does not detail weightings applied to component elements of assessment. Having reviewed Mr Carus’s report, I can confirm that his calculations are based on erroneous assumptions. The calculations presented by Mr Carus have not taken into account the weighting of different criterion or components of the MOSLER examination. Having reviewed Mr Crawford’s grades and calculations again, I am satisfied that the combination of Mr Crawford’s grades and his final outcome accord to the information on final grade calculations that is presented in the Stage [5] Handbook. I therefore confirm the original decision of the Board of Examiners, that of fail.”
“The final part of the appeal refers to a claimed error in the calculation of Mr Crawford’s ‘clinical and communication skills’ result. I must admit that reading the Stage 5 Handbook left me rather confused and I sought the advice of another panel member, who was also unclear as to how the results were computed. As a result I asked the Chair of the Board of Examiners to clarify whether the calculations performed on Mr Crawford’s behalf by an actuary were correct. Dr Lunn has pointed out that the calculations performed by the actuary weighted the individual components equally, when the reality is that certain components are weighted more than others. This is not clear from the Handbook and I suggest that Dr Lunn might see whether a different form of words might be chosen to make this more transparent. However, Dr Lunn has assured me that, taking this different weighting into account, the marks as recorded for Mr Crawford are correct and therefore I cannot support this part of the appeal.”
“This extra information has been supplied as an addendum to my instructions and I take it on trust as it is not to be found in the MBBS Handbook. Nor is it to be found in either of the results spreadsheets sent to me. Indeed the practice of weighting seems to be contra-indicated by the statement in the Handbook: “To average grades, each grade will be converted to a numerical score (M=4, S=3, B=2, U=1) and average calculated and the number converted back to a grade using the following thresholds (M>3.54, S>2.54, B>1.54, U<1.54 ).” “To average grades, each grade will be converted to a numerical score (M=4, S=3, B=2, U=1) and average calculated and the number converted back to a grade using the following thresholds (M>3.54, S>2.54, B>1.54, U<1.54 ).”
“In the light of this fundamental change in the Handbook, I am applying to the Adjudicator for a review of his/her decision to reject my appeal.”
“No evidence to suggest that there was any procedural irregularity in the handling of your Academic Appeal. Dr Lunn assured the Appeal Adjudicator that the correct marks were recorded for you and changes to the Handbook for the forthcoming academic year do not affect this.”
“It was recognised that the advice contained in the MBBS Stage 5 Handbook could cause confusion and was unclear. However, I am satisfied that method by which your marks were calculated were checked and confirmed as correct. Whilst I am critical of the University for the lack of clarity in the way that the marking procedure was set out in the MBBS Stage 5 Handbook, I am not persuaded that this lack of clarity meant that a procedural irregularity in the calculation of your marks had occurred. I consider that the University’s decision not to uphold this aspect of your appeal was reasonable in all the circumstances.”
“The Scheme does not cover a complaint to the extent that: ….3.3The matter complained about was the subject of Court or Tribunal proceedings and those proceedings have been concluded, or the matter is the subject of Court or Tribunal proceedings and those proceedings have not been stayed.”
“28. We accept that the straight-forward meaning of the bullet points to which the learned Deputy Judge refers is that marks would be calculated by averaging and that no weighting would be applied. However, the OIA has not heard the arguments which were made to the learned Deputy Judge, and it must be noted that his remit was only to decidewhether Mr Crawford’s case was arguable. We are not persuaded by Mr Crawford’s contention that the learned Deputy Judge’s comments means that his claim cannot fail.”
“29. The University argues, in its letter to the OIA dated8 October 2012 , that the paragraph relates to each assessment episode. It draws attention to the bullet point paragraph which precedes the two on which Mr Crawford relies:- “For each assessment episode you are graded according to the M, S, B, and U grading scheme such that at the end of the year you will have a final overall grade (M, S, B, U) for both Skills and Knowledge.”
“Details of the individual marking criteria within the in-course assessments are available on the in-course summative assessment forms. Details of individual criteria and weighting of criteria for the MOSLER and OSCE are not published. It should be noted that for these final examinations different weightings may be applied to different criterion [sic] depending on the components and focus of the given station within the examination.”
“32. Since we have concluded that the wording of the Handbook was ambiguous, we must go on to consider what effect that ambiguity had on Mr Crawford. We have concluded that it had no effect. This is for the following reasons: 32.1 The assessment of medical students is necessarily complex and difficult to describe. We are satisfied that the information provided to students by the University, both in the Handbook and in final year presentations, was consistent with the requirements set out in the extract from “Tomorrow’s Doctors” (2003), “Assessing student performance and competence.”
“In accordance with the principles of openness and best practice the University will disclose to appellant’s any relevant case documentation at an early and appropriate stage and in particular, in advance of any determination of the outcome.”
“42. The statutory scheme, in my judgement, provides an inexpensive, fairly rapid and comprehensive avenue by which challenges to a decision to expel a student for alleged cheating at exams can be fairly resolved.”
“45. Judicial review is a remedy of last not first resort. I wish to make it clear, both to this Claimant and to others in a similar position to her, that complaints of this nature should not ordinarily be pursued by judicial review, but should be pursued …by complaint to the Office of the Independent Adjudicator. That is the appropriate procedure for resolving complaints about alleged unfair expulsion from a university.”
“However Dr Lunn has assured me that, taking this different weighting into account, the marks as recorded for Mr Crawford are correct and therefore I cannot support this part of the appeal.”
“regarding the calculation of marks for the MOSLER examination were correct, then his outcome would have been correct.”
“The calculations presented by Mr Carus have not taken into account the weighting of different criteria or components of the MOSLER examination. Having reviewed Mr Crawford’s grades and calculations again, I am satisfied that his results have been calculated correctly. I am also satisfied that the combination of Mr Crawford’s grades and his final outcome accord to the information on final grades calculations that is presented in the Stage Handbook. I therefore confirm the original decision of the Board of Examiners, that of Fail.”
“The final part of the appeal refers to a claimed error in the calculation of Mr Crawford’s clinical and communication skills’ result. I must admit that reading the Stage 5 Handbook left me rather confused and I sought the advice of another panel member, who was also unclear as to how the results were computed. As a result I asked the Chair of the Board of Examiners to clarify whether the calculations performed on Mr Crawford’s behalf by an actuary were correct. Dr Lunn has pointed out that the calculations performed by the actuary weighted the individual components equally, when the reality is that certain components are weighted more than others. This is not clear from the Handbook and I suggest that Dr Lunn might see whether a different form of words might be chosen to make this more transparent.”
“Challenges to the academic judgement of the examiners on an assessment outcome or the level of award recommended cannot form the basis of an appeal.”
“30. I have considered the question of the OIA very carefully. The fact that the OIA complaints procedure is no longer available is by virtue of the Claimant choosing not to pursue it initially and then maintaining that course following receipt of the acknowledgement of service of the other parties. He would have been within the Rules of the OIA scheme had he abandoned his judicial review proceedings at that stage and submitted his claim to the OIA with a request for it to be entertained out of time. The circumstances of this case are not, in my judgment exceptional in such a way as to justify the exercise in my discretion to grant permission for judicial review when that original remedy would have been available to the Claimant had he made different choices.”