What can we measure in a questionnaire
- Types of question
o Behaviour
o beliefs
o Knowledge
o attitudes
o attributes
- Behavioural Questions
o do or have done
o measure
§ whether or not subject exhibited certain type of behaviour
§ frequency
§ time elapsed since behaviour
o example
§ is subject working (or did work) with pres-school age child
- Belief
o what is true or false – rather than on accuracy of beliefs
o beliefs not necessarily indicator of behaviour
o example
§ what subject believes to be effect of day care centre on emotional development of pre school age children.
- knowledge
o access subject's knowledge of particular facts à establish accuracy of their beliefs
o example
§ what subject knows about govt programs to assist parents with pre-school children to work part time.
o market research example
§ product awareness
§ product attribute awareness
§ product price
- difference between belief and knowledge
o belief
§ what is the average number of cups of coffee you think you should drink in a day
o knowledge
§ what is the average number of cups of coffee that an average person would consume in a day
- attitude
o what subject thinks is desirable
o example
§ whether or not mothers with pre school children ought to participate in workforce
- attribute
o age / education / occupation
Principles of Question Design
- reliability
- validity
- discrimination
- response rate
- same meaning for all respondents
- relevance
- Reliability
o same respondent should answer question same way on different occasions
- validity
o question should measure what it is intended to measure
§ example
· how healthy are you
o measure of health
o or
o optimism / pessimism
- discrimination
o sufficient variation allowed in response
- response rate
o usually caused by
§ question content
· sensitive
· irrelevant
· repetitive
· poorly worded
· difficult to understand
· insufficient response categories
§ method of administration
§ questionnaire length
- same meaning for all respondents
- relevance
o is question really necessary
Question Wording
- is language simple
- can questions be shortened
- is question double barreled
- is question leading
o does this seem like a good idea to you
- is question negative
- is respondent likely to have necessary knowledge to answer question
- will words have same meaning for all respondents
- is there a prestige bias
- is question ambiguous
- is frame of reference / context of question clear
o example
§ how often do you see your mother
§ within the last year, how often would you have seen your mother on average
- does question contain gratuitous qualifiers
o …… even if it leads to a decline in living standards
- is question dead giveaway
Question Response Format
- closed or forced choice
- open ended
- closed response format
o advantages
§ responses can guide subjects, eliminating irrelevant answers
§ easier to analyse
§ quicker to process
o disadvantages
§ response options must be exhaustive à problem with unanticipated answers
§ response options may be too narrow
- open ended response format
o uses
§ response options too large to pre code
§ attitudinal info where response options unknown / feedback is required
§ get at general feelings
§ find out reasons for opinions
o disadvantages
§ more demanding for respondents to answer
§ more time consuming to process
§ responses may not be relevant
§ difficulty to code
o advantages
§ many possible answers are allowed
§ do not have to anticipate all responses
§ data can be analysed quant and qual
§ clear insight into respondent logic / thinking
Principles of developing question responses
- principles
o exhaustiveness / inclusiveness
§ eg, categories for marital status
§ attitude questions
· include dk or no opinion
§ other category
o exclusiveness
o balancing categories
§ same number of alternatives on either side of neutral position
Numerical Rating Scales
- likert scales
- horizontal rating scales
o opposite attitude positions
- semantic rating scales
o opposite adjectives
- vertical rating ladder
Other Response Formats
- checklists – tick all that apply
- binary choice formats
o dichotomous
§ do you smoke
o paired comparisons
- multiple choice formats
Non Commital Responses
- no opinion / don't know
- inclusion of middle alternative
Response Sets
- tendancy to respond to question in some characteristic manner regardless of content.
o social desirability
o acquiescence
o non - acquiescence
- reducing social desirability
o mention that everyone does it.
o use an authority
o build in an excuse
o ask a less specific question
Order of Questions
- start with enjoyable questions
o easily answered
o factual
o obviously relevant to objectives of survey.
- go from
o easy to difficult
o easy to difficult
- open ended at end of survey
- sections
- use filter or contingency questions
- mix up pos / neg questions to avoid acquiescence
- consider randomizing
- use variety of formats
-
Pilot Testing
- pretesting
o ask participants how they interpret question meaning
o rephrase?
o is range of response alternatives sufficient
o is question necessary / redundant
o flow
o estimated time right ?
o clear instructions
o skips simple and clear to follow
o sufficient space
No comments:
Post a Comment