Category Archives: questionnaire design

Headteachers' National Standards of Excellence Survey (for Primary School governors)

After sharing my  primary school staff satisfaction questionnaire in a previous post another school governor John Millington (@jmillington on Twitter) has kindly shared the questionnaire he devised to evaluate staff views of the Headteacher's performance. It is shared here as  a Word document.

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

Designing a questionnaire for school staff (UK Governors/ School leaders)

This post may interest:

  • School governors/ leaders planning a staff questionnaire
  • Any seeking a (very) brief introduction to the options of designing any pencil and paper questionnaire.
  • Anyone looking for a LaTeX template for questionnaire design (this questionnaire is based on my slight adaptation of the paperandpencil.sty by Miriam Dieter & Anja Zwingenberger)

First thing: the actual questionnaire I designed

In my capacity as a school governor I designed a staff questionnaire last summer. I've made this available for anyone to use or adapt as they see fit. (Some of these questions are based on ones OFSTED ask teachers).

A basic consideration of options

There are many ways to do a staff questionnaire. Naturally online is fairly straightforward using third party software such as surveymonkey (free basic service, then £), or if you are more technically minded using various polling software and plugins in your own website (e.g. for wordpress). However in terms of online security paper and pencil questionnaires remain useful, and in a small(ish) school where everyone is in one place the benefits of setting up, administering and analysing an online questionnaire are probably not as great as for a larger organisation.

You may or may not be particularly worried about the presentation of your questionnaire. However, using your regular word-processing package (e.g. MSWord (£), LibreOffice writer (free and open source) etc.) will usually turn out to be more hassle than it's worth. Every time you try to move a box to a different place it ends up moving a whole load of other text (you've probably have experienced this behaviour when trying to put a photograph into a Word document).

If you wish to stick with a WYSIWYG (What You See Is What You Get) a publishing tool such as MSPublisher (£) or Scribus (free and open source) are likely to yield better results than word processing software. These will enable you to have more control over the page layout.

A bit more technical: using LaTeX

The questionnaire I designed is built in LaTeX and I've made the source code available in github. The key advantage of LaTeX (more about LaTeX here) is that you can have complete control over the layout with a combination of standard commands and some stylesheets. The .tex code can then be exported to .pdf leading to a document which can be printed off.

Link summary

.pdf version of the questionnaire we used at our school.  There are online services which can covert .pdf to Word (.doc/ .docx) though your mileage may vary.

Github access to the source code.

Further help

I'm happy to provide a small amount of free help (subject to my availability) to UK schools wishing to use the questionnaire (e.g. add school name/ logo, remove questions or make relatively minor amendments).

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

Questionnaires in LaTeX using the paperandpencil package

I've been using the paperandpencil (.pdf document) package for creating questionnaires in LaTeX. 1 I thought the package worked very well, but just a couple of notes to myself which others may find useful. I've found it works well, but a couple of things were unclear. The .pdf version of the manual displays but the homepage linked from other sites no longer appears to work. However I have just found a download page on the QDDS website.

  1. The file paperandpencil.sty needs to place in the same directory as the .tex file. I had trouble finding it, but found the code over on Github. I pasted this into a text editor, saved it as paperandpencil.sty, and put it in the same directory as my .tex file.
  2. The document class needs to be {scrreprt}. This is in the first page documentation, but I managed to miss it.
  3. To set page numbers \pagestyle{plain} is required and the \pagenumbering and \setcounter options need to be set.

A minimal code example here:

\documentclass[a4paper, 11pt]{scrreprt}
\usepackage{paperandpencil}
\usepackage[top=2.5cm,bottom=2.5cm,left=2cm,right=2cm]{geometry}
\pagestyle{plain}
\begin{document}
  \pagenumbering{arabic}
  \setcounter{page}{1}
  \section*{Title goes here}
\end{document}
  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

Notes:

  1. Produced by the Questionnaire Development Documentation System, based at the University of Duisburg-Essen

Taxpayers' cash should not be used to fund faith schools, say voters (But what did the survey really ask?)

headline

Taxpayers' cash should not be used to fund faith schools, say voters. Labour wants talks on teaching of religion as poll shows 58% of the public urge abolition or axing of state funds

The above headline in The Guardian intrigued me so I thought I would take a look at the actual questions asked. The survey was carried out by a company called Opinium on behalf of The Observer. On Twitter I tried to find out what questions were actually asked; interesting Richard Adams, Education Editor at the Guardian did not know, but said he would do some digging. I eventually found a pdf of the survey results on Opinium’s website.

In the interests of full disclosure I am a Christian with two children, one of whom is pre-school and the other attends a non-faith primary school. I did not attend a faith school myself. I’m not that interested in faith schools per se, but headlines such as the above invariably lead to debates about the relationships between religion and society that I feel duty bound to take an interest in.

The reason I’ve written this post is that I am very interested in questionnaire design. Questionnaire design is harder than most people think, but one would have thought a company which does surveys in its day to day work would be quite good at designing questionnaires. This one is so lousy I can only think it was politically motivated. It fact it is so bad I’m not entirely clear what the political motivation might have been.

Let’s start from the very beginning and take a look at Question 1. Respondents are asked to choose the opinion which is most like their own—note carefully -- not their opinion but the one that “best describes” their view of faith schools.
There are only three possible opinions plus a ‘Don’t know/ No opinion’ option.

The three available opinions are:

1. I have no objection to faith schools existing and being funded by the state
2. I have no objection to faith schools existing but they should not be funded by the state (i.e. private schools may be faith schools but not state schools)
3. Faith schools should be banned entirely

The best describes bit might be some sort of attempt to meaningfully take into account some of the nuances involved in such arguments but how views such as the following, which I suspect are widespread, fit into these three options.

1. I have no objection to faith schools existing and being funded by the state as long as they are only Christian/ CofE/ Catholic/ Muslim etc. (delete as appropriate)
2. I have no objection to faith schools existing and being funded by the state as long as they are NOT Christian/ CofE/ Catholic/ Muslim etc. (delete as appropriate)
3. I agree with faith schools but don’t agree with private faith schools because I don’t agree that there should be private schools.
4. I agree with faith schools as long as they don’t take their faith aspect too seriously.
5. I think all schools should be faith schools.
6. I agree with taxpayer funded faith schools as long as it’s not a really weird religion like…

As 1 and 2 above indicate there is really no such thing as a faith school, merely schools which are in one way or another connected to a specific faith, religion or belief system. Faith schools vary of course within faiths —Oasis academies, CoE voluntary aided village primary schools and ‘fundamentalist’ Christian private schools could all be described as Christian faith schools but they are certainly not the same in their ethos.

Question 5: In your view, how serious a risk is there of some predominantly Muslim schools encouraging their pupils to adopt extremist views?

1.Very serious
2.Quite serious
3.Not very serious
4.Not at all serious
5.Don’t know/ no opinion

The only specific religion mentioned in the survey is Islam. I’m not sure how one could answer this question in any meaningful way. This is in the context of the recent ‘Trojan Horse’ controversy of course, the full facts of which have not really emerged. The ‘Trojan Horse’ schools are not faith schools, or Muslim schools, but schools in which most children (and their parents) identify as Muslims. Overall 74% of those questioned thought there was a serious risk (i.e. very serious or quite serious), but what do we mean by extremist views? Extremists, however defined do not need extremist schools to incubate their views and activities. A lot of teachers would probably feel they are being credited as having far more influence on young people than they actually have.

The footer from the pdf of the report.
The footer from the pdf of the report.

A couple of further points: Although the findings are available there is a statement of confidentiality in the footer of each page. This may have been an oversight, or it might be a case that this data was never meant to be made public. I’ll assume the former until I have evidence to the contrary. It is also unclear how the survey was carried out, though it is possible to register with Opinium as a 'survey filler in' and I suspect people who fill in surveys are less likely to be in the ‘No opinion’ camp.

Whatever the truth the 58% figure does not strike me as particularly likely or unlikely—I’m not even sure if this is higher or lower than what I suspect would be the percentage of people disagreeing with schools having a religious ethos or culture. The Observer is evidently trying to influence the Labour Party on the matter of faith schools. I just wish they had designed a better survey.

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

British Academy publishes position statement on quantitative skills

From the British Academy 'Society Counts' webpage.

The British Academy has launched a Position Statement on the issue of a quantitative skills deficit in the humanities and social sciences. Well-rounded graduates equipped with core quantitative skills are vital if the UK is to retain its status as a world leader in research and higher education, rebuild its economy and create a modern participating citizenry. Quantitative methods facilitate ‘blue skies’ research, and without them, effective, evidence-based policy-making would be unthinkable. Yet, the UK currently displays weak quantitative ability within its humanities and social sciences.

The online book for Statistics for Humanities I am working on is funded under the Languages and Quantitative Skills programme.

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

In (sort of) defence of ratemyprofessors.com

A few weeks back the Times Higher published an article on student survey fatigue. Students fill in some many surveys including the National Student Survey, module surveys and institution wide surveys (coincidently, some the questions on the latter surveys are similar or identical to those on the NSS). I suggested on Twitter that the ineffectiveness of current surveys means that we want to do more surveys to fix the shortcomings of the existings surveys in order to give a ‘truer’ picture. Much of the reason that we are considering a national survey of language students stems from the shortcomings of the NSS outlined in our recent report. Should this new national survey go ahead I’m sure that it too will have its own shortcomings.

Legg and Wilson’s recent paper in Assessment and Evaluation in Higher Education on the reliability of www.ratemyprofessors.com vis-à-vis in course evaluations is interesting in its own right (their title "RateMyProfessors.com offers biased evaluations" deserves an award for clarity). But at least one thing could be said in defence of www.ratemyprofessors.com. It is unambiguous about who is the target of the evaluation—the teacher. With most other surveys it is unclear whether the students are being asked about the course, the teacher, the content, the university or the programme of study. The students don't know when they are answering the questions and we don't know when we are analyzing their answers.

See also: 4 ways to Avoid Survey Fatigue in Higher Education

 

 

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

Was the 2011 census piloted?

One-year old Elijah refused to give a coherent answer to the national identity question. He was also vague about his religious beliefs. I put him down as speaking English even though he does not say very much. Noting this on my facebook page, I discovered that other parents had similar problems. To the question "Where were you living twelve months ago" one friend claimed to have written, on behalf of his six month old son, "In my mother's womb".

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon