Valid assessment is about measuring that which we should be trying to measure.
Phil Race Making Learning Happen.
The Guardian website quiz ‘Life in the UK: could you pass the citizenship test?’ has been provoking a lot of discussion amongst my friends. None of my friends, UK citizens or otherwise, have been able to pass the citizenship test yet.
I suspect that the Guardian has selected some ‘greatest hits’ amongst the questions and that most obscure questions have been deliberately chosen. But, if the citizenship test is really about assessing British values, British history and British culture it is a total failure. We can’t be sure that new British citizens are able to participate fully in British society, appreciate British history and understand British customs but we can be sure that all our new citizens are successful learners of trivia.
Does it measure what we are trying to measure? The Home Office need to read Phil Race.
Mantz Yorke’s article in the latest edition of Studies in Higher Education “Summative Assessment: dealing with the measurement fallacy” has caught my eye. In my view assessment is (and arguably should be) anxiety producing for the assessor as well as the student. Yorke’s concern, as he notes in the first paragraph is for the students who are neither very strong or very weak, the bulk of students “where differences between individuals tend to be small, but can have a large impact on opportunities” (p.251).
Reflected in over 70 references the article covers a lot of ground including the UK degree classification system and the appropriateness of a classification grade which somehow summarises all summative assessment over the course of three or four years of study. Changing the classification for a letter grade or an overall percentage is not in itself any better—Yorke quotes the VC of Bedfordshire University’s evidence to the House Commons universities, Sciences and Skills committee “… as a chemist I would be telling my students not to average the unaverageable, then I would walk into an examination board and do exactly that”.
Yorke identifies many of the familiar and less familiar flaws in assessment and concludes that we need to acknowledge these before any improvements can be made. He writes of taking an overly judgemental approach to summative assessment to challenge the measurement (and seemingly scientific) fallacy of summative assessment. This would (my summary here from Yorke’s points on pp.262-265.
- Make it unambiguously clear (to students and others) that summative assessment are judgements, not refined measurements.
- Would help deal the variability in the use of the percentage scale (if percentage scale is really an accurate way of describing the marks given to students) between disciplines and for difference types of assessment within disciplines.
- Require students to accept that assessment grades are judgements of their work and not precise measurements. (Perhaps this would be the most difficult sell here).
I can’t do this article justice in summary form—it is well worth a read for anyone who deals with assessment in higher education.
Reference: Mantz Yorke, “Summative assessment: dealing with the ‘measurement fallacy’,” Studies in Higher Education 36, no. 3 (2011): 251-273. http://dx.doi.org/10.1080/03075070903545082