Tag Archives: National Student Survey

Could Galvanic Skin Response (GSR) bracelets replace other methods of teacher evaluation?

I’ve been thinking a lot about student engagement since my trip to Nottingham a couple of weeks ago. The theme of that conference was ‘student engagement’ and my task was to speak about the National Student Survey. During the course of the discussion a student opined that student engagement is often seen as synonymous with doing surveys of students. And as I often hear students are getting all surveyed out.

The news that the Bill and Melinda Gates Foundation has given over $1 million to Clemson University to research “‘Galvanic’ bracelets that measure student engagement” is responded to with incredulity by Valerie Strauss in her blog for the Washington Post. Clemson University’s website describes the project thus:

Purpose: to conduct a pilot study to measure student engagement physiologically with Galvanic Skin Response (GSR) bracelets, which will determine the feasibility and utility of using such devices more broadly to help students and teachers.

According to Wikipedia (a source I only use for find out things I don’t know anything about), Galvanisitc Skin Response (GSR) is also known as skin conductance:

Skin conductance, also known as galvanic skin response (GSR), electrodermal response (EDR), psychogalvanic reflex (PGR), skin conductance response (SCR) or skin conductance level (SCL), is a method of measuring the electrical conductance of the skin, which varies with its moisture level. This is of interest because the sweat glands are controlled by the sympathetic nervous system,so skin conductance is used as an indication of psychological or physiological arousal. There has been a long history of electrodermal activity research, most of it dealing with spontaneous fluctuations or reactions to stimuli.

Whilst I share Strauss’s scepticism (despite my total ignorance of this field of study), this project brings in another dimension to debates about measuring student engagement. Is there a ‘Brave New World’ in which teacher evaluation instruments will be replaced with student sweat analysis?  Will the perception of scientific objectivity appeal to policy makers?

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

Some questions that didn't make it onto the National Student Survey

Richardson , J., Slater J. and Wilson, J. (2007) The National Student Survey: development, findings and implications. Studies in Higher Education 32.5, pp. 557-580 :

This paper from 2007 reports on the piloting of the UK National Survey (NSS). The settled version of the questionnaire contains 22 questions, but the pilot contained 45 items. Interesting the pilot contained negative questions (where agreeing indicated a negative experience). Questions on workload were dropped due to a lack of internal consistency.

To do well on this course you mainly need a good memory

It was clear what standard was required in assessed work.

It was clear what I was required to attend, prepare, and to do throughout the course.

I found the overall workload too heavy

I feel confident in the subject knowledge I acquired

I had as much contact with staff as I needed

Overall questions

Overall I feel the course was a good investment

I would recommend the course to a friend

I wonder whether these would have been preferable to the "Overall, I am satisfied with my the quality of my course questions which is tends to take preference over all others.

The second pilot contained the question “It has been difficult to answer many of the questions because of the variability of my experience (Interesting only 12% strongly agreed and 18.2% agreed).

Just thought it would interesting to share the questions which did not make it.

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

NSS: What does "The course has helped me to present myself with confidence" mean?

 

The report from the LLAS Subject Centre National Student Survey project last academic year is now online. In the project we focused in on eight of the 22 questions. Whilst many of the questions were found to be problematic, this one was especially difficult to unpack.

Question 19: The course has helped me to present myself with confidence.

From the report

When answering this question, many students initially thought about giving oral presentations.  It was also linked to employability and interviewing skills, but the question of whether this was about personal confidence or academic confidence was unclear. And where students reported an increase in confidence, was this down to the skills their course had given them, their year abroad, their work placements, or was it just part of being four years older?

One member of staff observed that the NSS is carried out at a time where students are at their most anxious, perhaps looking for work, perhaps worried about the future. In languages it was suggested that this question might be thought about in the context of L2 competence or confidence in dealing with people from other cultures. ―It’s a bit of a weird question said one student. ―It really wants you to say “yes”, because if you say “no”, you‘re saying something bad about yourself.

Further thoughts

Some further thoughts here. Some a little pedantic maybe, but that’s what happens when you start to unpack the question with students and lecturers.

What this question might mean Possible assumptions Other issues
Employability

Doing oral presentations

Feeling confident in person

Interviewing skills

Self-belief

Able to express opinions without fear.

Able to challenge the opinions of others.

Not anxious

Students can stand up for themselves

Students are confident they will get a good job.

Students were unable to present themselves with confidence at the beginning the course.

Confidence comes from going the course.

Presenting oneself with confidence is a good thing (some students might benefit from being less confident)

A course which does not help students present themselves with confidence is not a good course.

The student who answers this question in negative might have been better off doing a different course or studying at a different place.

Confidence might come from sources other than the course e.g. student societies, increased age, work experience, time spent abroad

Does a negative answer to this question suggest that the course was in any way inadequate?

Some evidence of students thinking about L2 language confidence, but this question was for students of all disciplines.

Students who answer this in the negative are saying something bad about themselves.

Student anxiety or lack of confidence indicates poor teaching or course design.

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon

National Student survey: opening dialogue between students and lecturers

I am currently organising a workshop for languages, linguistics and area studies lecturers on making the most of National Student Survey (NSS). The workshop will include reports by three colleagues who have been funded by LLAS to carry out research into how students and staff understand the NSS questions. What does the statement, “overall, I am satisfied with the quality of my course” really mean? The NSS has not been without its critics, but its use in league tables and websites providing information to potential students makes it impossible to ignore.

I don’t think there is much to be gained from chasing the ratings. A couple of percentage points here and there can make a big difference to a department’s position in a league table, and the potential for wasted time, money and effort  in making superficial changes in the vague hope of squeezing out an extra 3% satisfaction is a real possibility. In my view the real opportunity the NSS provides is to open dialogue between students and their teachers.

The workshop will be held in London on 22 June 2011. You can register online at the LLAS website.

  • Twitter
  • del.icio.us
  • Digg
  • Facebook
  • Technorati
  • Reddit
  • Yahoo Buzz
  • StumbleUpon