Here are my comments on the article about design guidelines for online questionnaires.
- Article was published 6 years ago, and that's a bit old in a field that's technology dependent. Hey, even Twitter and Facebook weren't available (at least to the general public) then. A lot's happened since.
- I'd be interested to know how many online surveys are based on a probability sample. Most of the online surveys I'm aware of are panels consisting of self selected subjects
- For an author who is writing about "design", Lumsden's article wasn't a great example of design principles itself (eg, the introductory paragraph for section 3)
- I didn't agree with the following comment (near bottom of page 1) :
For online-questionnaires, sampling error ("the result of attempting to survey only some, and not all, of the units in the survey population … can arise when all but a small portion of the anticipated respondent set is alienanted (and so fails to respond) as a result of, for example, disregard for varying connection speeds, bandwidths limitations, browser configurations, monitors, hardware and user requirements during the questionnaire design process
From what I've read, there are two types of error in Total Survey Error:-
- sampling error
- non-sampling error
Sampling error is the error that arises because we're sampling only a subset of the population. Non sampling error includes non response and measurement errors.
I thought that what Lumsden was referring to was the online "equivalent" of non response; possibly willing respondents who have been put off because "it's too hard".
- Surprised about the recommendation that magnitude estimate questions shouldn't be asked on a online survey because you need the interviewer to explain such questions.
o Wasn't exactly sure what sort of question this is – when I googled the exact term it came up with only 9 results, so it's not a commonly used word.
o surely online would be an ideal mode to have supplementary material to explain more complex or unusual concepts.
- I searched some of the blogs I read on market research to see if there were any more recent comments on on-line questionnaire design. I think the one key comment I came across was that online questionnaires should behave like every other web application that has wide use. When Facebook, Twitter, Amazon and so on start using some functionality, then that's a good time for that feature to be used in online survey design. Another issue, and one that wouldn't have existed back in 2005 was that now you need to consider what device the respondent is using.
http://surveypractice.org/2010/12/08/device-respondent-has-used/
- I also came the emerging idea that the techniques of online gaming could be incorporated in online questionnaires so as to make for a more engaging experience for respondents.
Comments made in the discussion on the paper (p.59):
Bynner suggested considering different types of error irrespective of the question type:
Bias in the mean brought about by inaccurate measurement; and
Bias in measures of relationships brought about by correlated errors. This type of error can be a result of common method bias, for example. Consider two articles on common method bias:
No comments:
Post a Comment