Public Opinion Quarterly Vol 74, No 5, pp 849 – 879
Robert M Groves , Lars Lyberg
- TSE – concept that purports to describe statistical properties of survey estimates, incorporating a variety of error sources
- Among set of alternative designs, design that gives smallest total survey error (for given fixed cost) should be chosen
- Theory laid out by Neyman apply only when non sampling errors are small
- Sampling variance is measurable in most probability sample surveys, but other components of the notion (of TSE) cannot be measured directly without significant alteration of the typical survey designs
- Survey design components
o Identify population
o Describe sample
o Access responding units among the sample
o Operationalise constructs that are targets of measurement
o Obtain responses to the measurements
o Summarise data
o For estimating some stated population parameter
- 1950s – most textbooks treating surveys were sampling texts
- Sampling errors are inherently errors of non-observation
- Relate process quality with total survey error
- Groves et al 2004
o Two separate inferential steps required in surveys
§ First inference is from response to a question for a single respondent and the underlying construct of interest to the measurement
§ Second inference is from estimate based on set of respondents to target population
- Development of typologies continues
- TSE
o Nested taxonomy of concepts of error
§ Variance & bias
§ Errors of observation and non-observation
- Construct validity
o If one were interested in a child's intelligence, we would not define the true value as the score a teacher would assign on a particular day, but rather a more permanent attribute separate from measurement itself.
- MSE = sampling variance + response variance + covariance of the response and sampling deviations + the squared bias
- Re interview approach
- Interpenetration and re interviews are necessary for measurement of response variability components
- Multiple indicators
- TSE not the only way to think about information quality
- Relevance and credibility
- Survey designer faces problem of how much budget to spend on measuring quality versus on other things
- Growing evidence that measurement error models are highly variable by type of measurement.
- Value of decomposition of errors
o Has worked
o By isolating measurement error sources, have learnt how to construct better questionnaires
o By separating types of non response, we are learning to attack non contact and refusal separately, given their different covariances with many survey variables
- Interplay of different error sources
o Respondents with lower response propensities tend to have higher likelihood of measurement biases.
- How a multi mode, multi frame multi phase world may help us
o Offer built in contrasts that can be informative about error properties of survey statistics
No comments:
Post a Comment