This is a note I drafted early in 2015 but, apparently, failed to publish. I rediscovered it this morning while searching for something else that I needed for an abstract I was submitting, so I thought I’d post it to see what people thought. (For reasons I cannot explain the article has retroactively published itself on the date that I first drafted it even though I published it on 10th May 2016.)
There was a brief conversation on the Oak Table List when the UKOUG Tech 14 scores came out about the fact that the UKOUG used a range of 1 – 6 for rating speakers when the rest of the world used 1 – 5. Personally I think 1 – 5 is a bad idea and, as a speaker and an organiser, I think 1 – 6 is better and what I’d really like is 1 – 10. (In fact it’s possible that the UKOUG uses 1 – 6 as a consequence of a remark I made a few years ago when I was on the board of the UKOUG.)
Here’s how I’d argue my case. If you have 1 – 5 then, when supplying a rating, your choices are (nominally):
- I stayed to the end but don’t have any particular opinion
The range of possibilities is too low: where’s the rating for “a couple of nice points, but not particularly good”, or “I wish I’d gone to something else, but it’s not so bad I want to walk out”. With a range of 1 – 6 you’re denied the possibility of an uninformative default of 3; and have a range of three ranks of “good” and three of “bad”.
Note, also, that I said “nominally” when listing the choices – unfortunately it’s a well-known effect that the extreme ends of a range of ratings tend to be automatically excluded, and the value viewed as meaning “average” is some way above the mid-point of the range (another hand-waving reason there for not having a mid-point value). If you set a range from 1 – 10, the actual values will tend to fall in the range 4 – 9, which gives you 6 values where you can get a proper feeling for how well the presentation was received, and if you get a 10 you know it went down well (which isn’t necessarily the same as it being a good presentation, of course) and if you get less than 4 you know you’ve got something to worry about.
In my dim and distant past I was a school teacher (teaching maths and computer science to the 11 – 18 age group) and the rating system for reports where I taught was two-part: A-E for effort, 1-5 for results. Nominally C-3 meant that your effort and results were typical of your year group. Realistically any pupil who got a C-3 was hugely offended and the average rating given was more like B-2.
(Inevitably there was at least one pupil I taught who spent his time trying to get his report card filled with E-1 for every subject.)
Your comments are welcome – and if there are enough then they may be of some assistance to the people who organise and speak at the conferences you attend. In fact, with that thought in mind I’ll even put up a poll on the three ranges I’ve mentioned in this article.