Excellent lecture here from Tansy Jessop, Professor at Southampton Solent University. Makes important points about 'pedagogic research' being put into its own category and the 'own goal' of Boyer's Scholarship of Teaching of Learning.
NB: This post is about the problems of pie charts and is not a criticism of the hard-working people at Hailsham Town Council.
A few years ago I came across Stephen Few’s Save the Pies for Dessert article on the perils (and general pointlessness of most pie charts).
My local town council has just provided a gift illustration for the useless pie chart in a recent newsletter.
I’ll just outline some of the numerous problems with the chart:
1. There is no indication of how much money will be spent in each category, so we don’t know the overall size of the pie (perhaps they don’t know yet).
2. There are 29 categories. This means that 29 different colours are needed in the key. For example, four shades of light blue look to same to me. I don’t know which slice refers to Hailsham Revitalization, which to Election Costs, which to Outdoor maintenance and which to Hellingly PC subsidy. I am not colour-blind, but I cannot tell the difference.
3. Some the categories are so small they can barely be seen.
4. The main thing I can decipher is most money is spent on something green, something blue and something yellow.
So what is the alternative?
In this case it would be preferable just to list the amount of money (or percentage of the budget) being allocated to each category. If a visual representation is really needed a bar chart would be preferable.
Are there any real reasons for using pie charts?
1. Some readers might understand the budget better visually, but with 29 categories they would need an incredible eyesight and and superb ability to distinguish colours.
2. They might be useful where there are a small number (<5) of categories. However, the data would probably be still be better presented as a table.
3. They can be very pretty and use up some space.
Can it done worse?
1. In black and white/ greyscale.
2. Using spreadsheet's 3D pie chart feature (It looks nice but the perspective distorts the proportions.)
I’ve been running workshops on research methods for some years now, but have done very little on visual research methods. My colleague Pauline Ridley recently ran a session on visual methods for our PGCert participants which I rather enjoyed. Visual methods can be useful in representing hard to verbalise experiences, visually abstract concepts and representing processes. Pauline was very keen to point out that the purpose of visual research methods is not for the researcher ‘to interpret’ visuals produced by others, but to use visual methods alongside methods focused on written or spoken language.
My two attempts to draw the concept of ‘learning’ (as challenged by Pauline) appears below. (A discussion of my drawing abilities would require a more substantial blog post).
The first image represents a very traditional image of teaching and learning. A teacher (on the left) talks and ‘transfers knowledge’ to the seated student who makes notes. It is interesting that this was the first things that came to mind as I has spent many years emphasising the need to escape didactic knowledge-transfer approaches to learning and teaching.
The second image was inspired by a remark made by Mark Goodwin who was my personal tutor during the second year of my geography degree at Aberystwyth. He said something to the effect, “It’s called reading for a degree for a reason... you are supposed to read”. (The image is supposed to be of a person reading a book). That utterance took less than 10 seconds to pass his lips, but it is something I’ve never forgotten and pass onto others.
Pauline Ridley and Angela Rogers(2010) Drawing to learning series (Open Access)
My latest article 'The UK Teaching Excellence Framework (TEF) as an illustration of Baudrillard’s hyperreality' has been published in Discourse: Studies in the Cultural Politics of Education. Not all university staff feel able to write about the TEF in the way I have done here and one reader has fed back to me on the importance of having a radical response to the TEF.
Canning, John (2017) The UK Teaching Excellence Framework (TEF) as an illustration of Baudrillard’s hyperreality. - Discourse: Studies in the Cultural Politics of Education. http://dx.doi.org/10.1080/01596306.2017.1315054
Abstract: This article examines the ‘Teaching Excellence Framework’ (TEF) for UK universities through the lens of Jean Baudrillard’s concept of hyperreality. I argue that the TEF is a hyperreal simulacrum, a sign which has no traceable genealogy to the practice of learning and teaching.
D. R. E. Cotton, W. Miller, and P. Kneale (2017) The Cinderella of academia: Is higher education pedagogic research undervalued in UK research assessment? Studies In Higher Education http://dx.doi.org/10.1080/03075079.2016.1276549
I thought I’d share a few thoughts on Cotton et al’s recent paper on the status of pedagogic research. As an HE pedagogic researcher myself, it is tempting to nod my head profusely taking comfort in the knowledge that there are a few people out there who understand the situation.
The central trust of the paper concerns the 2014 Research Excellence Framework and the particular politics surrounding the inclusion and exclusion of HE research in the Education ‘Unit of Assessment’. Cotton et al also touch on the fact that many HE pedagogic researchers have teaching-only contracts or are non-academic staff and are therefore intelligible for the REF. I was a non-academic member of staff in my previous job, so my research, for good or ill, was not visible to the university’s processes. Even though there were advantages to being off the ‘REF radar’, being a ‘non-academic’ meant that my research was somehow invisible to the university.
My main point of interest in this article was the authors’ discussion of the relationship (or lack of relationship) between pedagogic research and Scholarship of Teaching and Learning (SoTL). I have long been concerned about Boyer’s (1990) separation of SoTL from the ‘Scholarship of Discovery’ (that is original research that advances knowledge):
To most academics, scholarship means reading papers and being informed, not undertaking primary research. So when pedagogic research and SoTL are conflated, it implicitly devalues the former. To make further progress in developing the profile of pedagogic research, and integrate it into research assessment, high quality pedagogic research should be viewed as something quite distinct from SoTL. Whilst it may contribute to teaching enhancement in HE (as may discipline-based research through the research–teaching nexus), until it is viewed inherently as a research endeavour, rather than as ‘scholarship’, submitting HE pedagogic research into the REF will continue to be open to challenge. (Cotton et al 2017)
I played a small role (Masika et al 2016) in the recent HEA-funded SoTL project cited by Cotton et al (Fanghanel et al 2016) carrying out interviews with people in educational development units about SoTL at the institutional level. My personal conclusion in carrying out the interviews was that rather than being contested, SoTL is a concept which few people have any views on. This distinction is important – it is not that people have different understandings of SoTL that is the issue, but that SoTL seems to have an almost mystical, deistic status. We believe SoTL exists, but do not agree what it is and behave as if its existence has no material consequences.
When SoTL and pedagogic research are conflated we end up in a situation where quality research into higher education teaching and learning is given parity of esteem with the practice of reading a book or article on teaching every now and again which is reported institutionally as ‘participating in the Scholarship of Teaching and Learning’.
Boyer, E. L. (1990), Scholarship reconsidered: Priorities of the professoriate. Carnegie Foundation for the Advancement of Teaching
Cotton, D. R. E; W. Miller, and P. Kneale (2017) The Cinderella of academia: Is higher education pedagogic research undervalued in UK research assessment? Studies In Higher Education http://dx.doi.org/10.1080/03075079.2016.1276549
Fanghanel, J., J. Pritchard, J. Potter, and G. Wisker. 2016. Defining and Supporting the Scholarship of Teaching and Learning (SoTL): A Sector Wide Study. HEA Report. https://www.heacademy.ac.uk/resource/defining-and-supporting-scholarship-teaching-and-learning-sotl-sector-wide-study.
Masika, Rachel, Wisker, Gina and Canning, John (2016) Defining and supporting the scholarship of teaching and learning (SoTL): A sector-wide study, SoTL Case Studies[. York: HEA https://www.heacademy.ac.uk/resource/defining-and-supporting-scholarship-teaching-and-learning-sotl-sector-wide-study
My article on metrics and the proposed Teaching Excellence Framework now appears on the Guardian Higher Education Network website.
By virtue of its title Innovations in Education and Teaching International sets its aspirations very high. I’ve just been browsing the latest printed edition where Stewart and Stewart’s article ‘Teaching Bayesian Statistics to Undergraduate Students through Debates’ grabbed my attention. Like most people I have been educated in classical (or frequentist) statistics and have virtually no knowledge of Bayesian statistics—hence the title caught my attention. The article has therefore reminded me to rectify my ignorance.
The title gave me no clue to what I was about to read about though. The lecturer (second author Wayne Stewart) performs debates between ‘frequentists’ and ‘Bayesians’ using two ventriloquist dummies—Freaky the frequentist and his opponent, a doll depicting Thomas Bayes (c.1701-1761) in his persona as a Presbyterian minister (photographs appear in the article).
Personally I find ventriloquist dolls pretty sinister. However as an approach worthy of the label ‘innovation’, it’s going to be hard to surpass. Most of the students found the dummies funny though and claimed to have learned from them.
Sepideh Stewart and Wayne Stewart (2014). “Teaching Bayesian Statistics to Undergraduate Students through Debates.” Innovations in Education and Teaching International 51, no. 6: 653–63.
Nearly 15 years ago I submitted my first ever article to a peer reviewed journal. That paper was a version of “Motivation for volunteering on heritage railways”. The fact that the paper is now self-published on my website indicates that the paper was never published. It was never actually rejected, but after six years under review I’d had enough.
I won’t name the journal or the editors involved. Although this is not an area I publish in now, (or am likely to) I am somewhat risk adverse. The three editors involved in the process are no longer editing the journal. My memory on some points might be a bit hazy, especially regarding the order of some of the events, but here goes:
- I submitted the paper in January 2000. It was based on my MSc dissertation. This was the days before electronic or online submission (at least was for this journal). I sent the paper (2 printed copies plus a 3.5 inch floppy disk) to the editor at a UK university.
- I heard nothing. I tried to contact the editor but could not find a phone number or email. I got in touch with the press. Still nothing. [Submit somewhere else which desks rejects it. My PhD supervisor assured me this was OK since I hadn’t heard anything after about a year.]
- Two years later (!) a new editor gets in touch by email with a single review of the paper. The reviewer makes helpful comments. I spend a couple of months responding to the comments. I then send the new version to the editor by email.
Interlude: Strange twist here which is nothing to do with the process. My MSc supervisor forwards me an email from a postgraduate student who wishes to read my dissertation. I get in touch with him and he comes to Bristol to read my dissertation in the library. We arrange to meet up and have a cup to tea together. It turns out that his supervisor was the reviewer and guessed that I was the author. Reviewer suggested to student that he get hold of my dissertation as it might be useful for his own work. I now know the identity of the reviewer and know that the reviewer guessed my identity.
- A few months later I get a second review from the same single referee. Comments were mostly helpful, but referee says that a lot of new work has been published since he/she last read the paper. (This indicates that the review was probably done soon after I submitted the paper, but I was not sent the review until about two years after it was conducted.)
- I revise the paper again and send it to the editor. We are probably up to about mid-2003 or 2004 by this point. The editor and I exchange lots of emails. Editor makes lots of helpful comments, helps me with matters of writing style etc. Indicates he/she is keen to publish it. No further reviews are conducted.
- Correspondence with the editor comes to a close. He/she recommends that it should be published in one of the 2006 editions. However, there is a catch… he/she is stepping down from the editorship and won’t be editor in 2006, so it’s not his/her decision.
- 2006 arrives and I hear nothing from the new editor. I email the new editor (not a name I know) who gets back to me fairly promptly. He finds my paper in the trail left by his predecessor. He informs me that it is now journal policy for all papers to be reviewed by at least two people. Mine has only been reviewed by one person so it can’t be published. Editor also informs me that that they want to publish more things which are not about railways.
- Get fed up and withdraw the article.
- Sit on it for a while then put it on my website.
Fortunately I now have many positive experiences of the publishing process to share. It has not been all plain sailing, but even the worst experiences have not come close to the ludicrousness of this article’s journey through peer review. Here a few lessons I would pass on.
- NEVER EVER let any journal not get back to you for one year, let alone two years. You are perfectly within your rights to submit elsewhere.
- Try to be clear on whether your article has been accepted or not.
- If you are not sure if the editor agrees with the reviewers, ask.
- I think I let the new 2006 editor off the hook. It seems a bit strange that a new editor would feel able to apply new rules about reviewer numbers retrospectively and ignore the recommendation for his predecessors.
A technical note to myself here, but may help others as well. When I imported .mp3 files from my Olympus Digital Voice recorder WS-831 to my PC it was no longer possible to play the files. The solution is to go to Windows Explorer and change the extension of the file from filename.MP3 (MP in capitals) to filename.mp3 (mp in small letters). Then they should play on the PC. They can also then be imported into other applications (e.g. audacity) without any problems.