First Examination of Observational Errors vs Errors of Non Observation
- Up to now – errors of non observation – failure of survey to measure some portion of population, and on costs of reducing those errors
- Now – errors in data obtained – measurement errors
- Sources
o Interviewer
o Respondent
o Survey questionnaire
o Mode of communication
- Issues
o Different designs are used to estimate variable measurement errors and biases
o Researchers who conceptualize an error as a bias often seek to identify cause of error and eliminate it – those who conceptualize error as source of increased variation in survey statistics often attempt to measure it and adjust their conclusions to its presence
o Most past work studies only one of the design features as source of measurement error
Laboratory Experiments Resembling the Survey Interview
Measures External to Survey
- One way to estimate measurement error – compare result of survey measurement to another indicator of same characteristic.
o Obtain external data on individual persons in survey – eg, police or hospital records
o Compare external population parameter values with survey based estimates of same quantity.
Randomized Assignment of Measurement Procedures to Sample Persons
- Eg, randomized assignment of interviewer to sub samples of respondents.
- Split sample experiments – two different versions of same question
- Split sample experiments concerning measurement error
- Biases with interviewer behaviour
- Interviewing procedures
- Interpenetration for estimation of variable measurement error
o Interviewer effect
§ White interviewers appear to influence more moderate racial attitudes among black respondents who would give more radical answers to same question asked by black interviewer
o Ordering of questions or response categories
- Practical problems with randomization as a procedure for estimating measurement error
Repeated Measurements of same persons
- Same questions
- Multiple measures of same concept (in same survey)
- Direct questioning of respondent about their understanding of question
- Re-interviews to estimate measurement error
- Multiple indicators of same characteristic administered in same questionnaire
- Measuring characteristics of the interview itself
- Direct measures of cognitive steps in response formation
Summary of Individual Techniques of Measurement Error Estimation
- Replication of identical measures over trials
- Replication of indicators of same concept within trials
- Randomization of measurement procedures to different persons
- Collection of correlates of measurement error
Combinations of Research Features
No comments:
Post a Comment