Validity – Modern Measurement
Validity at its Essence
- Information yielded by test is appropriate, meaningful, and useful for decision making
- Degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of the tests.
Importance of validity to measurement
Validity of scores – not of instruments
- Validity concerns decisions based on yielded scores of an appraisal activity
- Degree to which interpreting and using test scores for a particular decision I supported by evidence and theory.
- Three aspects of validity
o Interpretations of test scores in particular assessment situation, and not to features of a given instrument
o Evokes formal, evaluative process
o Exploration into psychology
Establishing validity as a shared endeavor
- Making supported decisions
Validity as described by the Standards text
Background of validity
Validity Advancing : Messick & Cronbach
Validity as Evaluative Argument
Cronbach's Facets of validity
Source of validity evidence
Sources collectively present information about confidence level for making a score based inference in a specific situation
Validity evidence based on a test's content
- Identify and articulate test content domain
- Not always easy with psychological areas
Validity evidence based on response processes
- Reasoning as opposed to using standard algorithm or memory
Validity evidence based on internal structure
- A carefully devised and well researched theory can provide a better platform for the development of estimable constructs than can weak or imprecise groundings.
- Psychometric methods available to empirically investigate the internal structure
o Factor analysis (FA) and related data reduction methods
o Cluster analysis, principal component analysis
o Confirmation of psychologically based theories : confirmatory FA
o Multitrait – multimethod matrix
o Alibility / proficiency parameter estimation techniques (IRT)
Common Factor Model
- Devised by Charles Spearman
- Test comprises items that both share commonalities and have unique influences
- Commonality among items not uniformly distributed.
Validity Coefficient and Coefficient for Determination
- Correlational relationship between measure and a relevant external criterion
- Correlation
o Pearson' r
o Spearman rho
o Phi coefficient
o Tetrachoric correlation
o Contingency coefficient
- Validity coefficient
Standard Error of Estimate
Validity evidence based on relations to other variables
The Criterion Problem
- Real world – problem arises when a correlational relationship is sole / primary source for validity evidence
- Correlational reliability as criterion related evidence for validity of a test is depressed by degree of measurement error in criterion.
- Trying to anchor a boat to a boat
The restriction of range and normalcy in variables
- Sensitivity of the correlational coefficient to issues related to the range and normalcy of the variables employed in making correlation ( linearity , hetro-homoscedasitcity)
Validity evidence based on external considerations
- Face validity as a source of evidence
GW Comments
- Can the theoretical issues addressed here about educational tests etc apply also to say credit scoring, where the data used in the test is not necessarily obtained from the test subject.
- Look at biography of Charles Spearman
No comments:
Post a Comment