Alastair explained how Louis Thurston had developed this theory of assessment in the 1920s, based on simply comparing one piece of work directly with another. Alastair argued that abstract assessment criteria did not help in the process of marking, as examiners inevitably convert the abstract into concrete exemplars, increasing variability and unreliability. So why not just compare work directly? If enough comparisons between two different pieces of work are made by enough judges, a very reliable rank order emerges (the one that always wins moves to the top, the one that always looses goes to the bottom and the others spread appropriately between). I understand that QCA use this system already to monitor inter-board comparability, basically to ensure an ‘A’ in maths from OCR is the same as an ‘A’ in maths from Edexcel.
The problem lies in the scale of the award. With twenty paper scripts and half a dozen judges it can be done round a table, but when there are thousands of scripts and dozens of judges it becomes a logistical impossibility. However, with the advent of web-based portfolios, like the e-scape set of portfolios, are available anywhere and anytime each assessor has an internet connection. Multiple copies can be viewed at anytime, making the paired process possible in a high-stake assessment for the first time.