Notes: Traditional Reliability Strategies
- Reliability and various means to estimate it.
Generic Description of the Concept of Reliability
- Meaning of reliability
o Precision in mental appraisal
o Precision is determined by consistency of randomly parallel measures across several appraisals
o How well samples of approximate assessment stimuli represent the universe of possible stimuli
o Repeated measuring
o Content domain – broad vs narrow à broad – greater likelihood that sample does not hold enough breadth and depth to represent field fully.
o Reliability is a necessary but insufficient condition for validity
o Reliability is a measure of the absence of random error
- Definition of reliability in CTT
o Lord & Novick : the reliability of a test is defined as the squared correlation between the observed score and true score.
o In practice, only have observed scores
Considerations for Reliability Assessment
- Relaibility as integral to measurement theory
- Reliability is more than a procedure
- Reliability of scores, not of instruments
- Confusing reliability with validity
o A consistent inaccuracy is still inaccurate
- Reliability and error as used in popular speech
- Comments on reliability in the Standards text
- A giant in the field of reliability appraisal
o J Cronbach
Theory of Reliability
- Reliability as error variance
- Developing the index
- Index vs coefficient
- Estimating true score from observed score with index and coefficient
Problem of parallel measures
- Identifying parallel measures
- Test – retest
- Other proximal estimates
- Depressed calculations with shortened test length
Computing Reliability Coefficients
- Spearman-Brown Prophecy (or Correction) Formula
Standard Error of Measurement
No comments:
Post a Comment