Selecting Areas
- Can't go back to subjects and ask additional questions
- Think ahead about research problem
- research problem à concepts to be measured.
- What concepts mean
- indicators devised for concepts / how variables are links
- How we will analyse
Question Content
- Five types
o Behaviour – what people do
§ Difficult to extrapolate from behaviours to beliefs and attitudes
o Beliefs – what's true / false
o Knowledge – establish accuracy of beliefs
o Attitudes – what they think is desirable
o Attributes – characteristics – eg, age, gender
- Direction, extremity and intensity of attitudes
o Direction – question
o How extreme – eg, likert scale
o Intensity
Principles of question design
- Reliability
o Question should be answered same way on different occasions for same person
o Avoid ambiguity etc
- Validity
o Measures what we think it does / measures what it is intended to do
- Discrimination
o Sufficient response alternatives
o Need finer grain
o Sensitive to measuring real & meaningful differences in sample
- Response rate
o Content
o Construction
o Method of administration
o Length
- Same meaning for all respondents
- Relevance
Wording questions
- Is language simple
- Can question be shortened
- Is question double barrelled – more than one question
- Is question leading
- Is question negative
- Is respondent likely to have necessary knowledge – first filter
- Will words have same meaning for everyone
- Is there prestige bias – eg asking about pope's view on birth control / attitudes about pope vs attitudes about bc.
- Is question ambiguous
- Is question too precise
- Is frame of reference for question sufficiently clear
o Eg, time frame
- Does question artificially create opinions
o Where people don't have existing opinion
- Is personal or impersonal wording preferable
o How they feel about something
o How people feel about something
- Is question wording unnecessarily detailed or objectionable
o Eg, precise income vs income band
- Does question have dangling alternatives
o Would you say that it is frequently, sometime, realy or never that you ....
- Does question contain gratuitous qualifiers
o Do you favour or oppose increasing number of uni places even if it leads to a decline in standards
- Is question a dead giveaway
o Uses all inclusive or exclusive words
o These words allow no exception
Selecting Question Type
- Open and closed formats
- Closed / forced choice
o Can create false opinions if not enough choice allowed
o Speed up completion
o Easier to code
o Do not discriminate against less articulate
o Useful prompts
- Response formats and level of measurement
o Level of measurement of variable
§ Based on response categories of variable
§ Relation of one response category to another
o Interval level preferred à allows more powerful analysis techniques
o choice of statistical methods
o responses to similar question – can be nominal / ordinal / interval à depending on how question worded
Developing Question Responses
- Exhaustiveness (or inclusiveness)
- Exclusiveness – one and only one answer à eg ask for most important / split into multiple questions
- Balancing categories
o Imbalance of responses in one direction
Developing Response Alternatives for Closed Choice Questions
- Numerical rating scales
o Likert scales
§ statement reflecting particular attitude / opinion : agreement / disagreement
o Horizontal rating scales
§ Opposite attitude positions
§ Where along continum their position is
o Semantic differential
§ Opposite adjectives – rather than attitude positions
o Vertical rating ladder
§ Rank – eg, rank universities
- Scores
o Out of 10
o Feeling thermometer
- Common points
o one and only one answer
o responses high to low
o way in which each item answered is not constrained by way in which other items in set are answered
- Ranking
o Rate importance / strength of agreement relative to the way other items in set have been rated.
o relative (rather than absolute) importance
- Checklists
o only select items that apply
- Binary choice formats
o Dichotomous questions
o Paired comparisons
- Multiple choice formats
o Multiple nominal categories – eg marital status
o Ordinal categories – how often do you go to church
o Ordered attitude statements – degree of belief about god
o Numerical answers - income
Non Committal Answers
- No opinion and don't know responses
- Laziness
- Inclusion of middle alternative
o Neither for nor against
Number of Response Categories
- Dichotomies
- Five point scales
- Longer scales
Response Sets
- Aquiensence
o tendency to agree with statement regardless of content
- Social desirability
o Look good
o Techniques to reduce
§ Everybody does it
§ Use an authority
§ Build in excuse
§ Ask less specific question
- Aquiescent response sets
Questionnaire layout
o Answering procedures
o Filter or Contingency questions
o Instructions
§ General
§ Section
§ Question
§ Go to
o Use of space
o Order of questions
§ Easy to more difficult
§ Concrete to abstract
§ demographic at end
§ meaningful sections
o Setting up for coding
o Questionnaire length
- Telephone questionnaires – additional considerations
o Number of response options – memory
o Layout
- Pilot testing – evaluating questions and questionnaires
o Question development declared or participating pre test
o Questionnaire development
o Polishing pilot test
- Pilot Testing Items
o Variation – mostly similiar answers – not as useful for analysis
o Meaning
o Redundancy
o Scalability
o Non-response
o Acquiensent response set
§ See if people agree with 2 opposite questions
- Pilot Testing Questionnaires
o Flow
o Question skips
o Timing
o Respondent interest and attention
- How to pilot test
o Who to pre test
o How many
o Who to conduct interviews
o Code responses
§ Open ended
§ Other
o Interviewer debriefing
- Questionnaire Design Checklist
No comments:
Post a Comment