1 Types of Validity Content Validity Criterion Validity Construct Validity Predictive Validity Concurrent Validity Convergent Validity Discriminant Validity Adapted from Sekaran, 2004

2 Construct Validity Extent to which hypotheses about construct are supported by data 1. Define construct, generate hypotheses about construct’s relation to other constructs 2. Develop comprehensive measure of construct & assess its reliability 3. Examine relationship of measure of construct to other, similar and dissimilar constructs E.g. Height & Weight; Networking & career outcomes study; Learning Style Orientation measure Types of Validity

3 Multi-trait multi-method matrix Convergent validity coefficient Absolute size of correlation between different measures of the same construct should be large & significantly different from zero Discriminant validity coefficient Correlations between the same construct measured by different methods should be significantly different from correlations of Different constructs measured by different methods (across methods & across constructs) Different constructs measured by same method (method bias) Establishing Construct Validity

4 O-HSR-HO-WSR-W O-H 1.00 SR-H.981.00 O-W.55.561.00 SR-W.68.69.921.00 Convergent & Discriminant Validity Corr b/w Objective (O) & Self-Reports (SR) of Height & Weight

5 Convergent validity (across methods) Objective and subjective measures of height are correlated.98 Objective and subjective measures of weight are correlated.92 Convergent & Discriminant Validity of Objective & Self-Reports of Height & Weight

6 Discriminant validity (across constructs) Objective measures of height & weight are corr.55 Subjective measures of height & weight are corr.69 STRONG CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by same methods i.e., Are.92 &.98 significantly different from.55 &.69? Convert rs to z scores and compare Discriminant Validity of Objective & Self-Reports of Height & Weight

7 Discriminant validity (across constructs) Objective height & subjective weight are corr.68 Subjective height & objective weight are corr.56 WEAK CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by different methods i.e., Are.92 &.98 significantly different from.56 &.68 Discriminant Validity of Objective & Self-Reports of Height & Weight

8 For convergent validity, different measures of the same construct should be highly correlated Note: In networking study, diff measures= diff subscales –Corr b/w diff measures of career success (promotion, salary, perceived career success) range from.20 to.36 (moderate support).20 &.36 should not be sig diff from each other –Corr b/w diff measures of networking (increasing internal visibility, socializing etc.) range from.03 to.43 (weak support).03 &.43 should not be sig diff from each other Establishing Construct Validity in Networking Study

9 For discriminant validity, different measures of the same construct should be more highly correlated than different measures of different constructs –Correlations b/w career success & networking (.01 to.35) should be sig different from Corr b/w diff measures of career success (.20 to.36) Corr b/w diff measures of networking (.03 to.43) Establishing Construct Validity in Networking Study

10 Developed items by generating critical incidents (Study 1) –N=67 –Yes/no responses to statements –Recall of learning events Two types of learning: theoretical, practical (2) Two types of outcomes=success, failure (2) 2 x 2 events per participant 112 items constructed in total Learning Style Study

11 Study 1 Part 2: Created 112 items from critical incidents –Administered to 154 participants –5-point likert scale (agree/disagree) Extracted 5 factor solution w/factor analyses 54 items loaded highly on the 5 factors Content validity: item sorting by 8 grad students –Also administered personality scale Learning Style Study

12 Item Development Study (study 1) Convergent Validity High reliabilities of subscales of Learning Style (.81-.91) Corr b/w different measures (subscales) of learning style =.01 to.32 but 1 only corr is significant –Weak support for convergent validity of new learning style measure Discriminant validity Corr b/w different measures of different constructs (Learning Style & personality).42 to.01 should be lower than and sig diff from corr b/w different measures of same construct (subscales of learning style).01 to.32 Learning Style Study

13 Validation Study (study 2) –N=350 -193 –New learning style, Personality, old Learning style, preferences for instructional & assessment methods –Construct validity Confirmatory factor analysis confirms 5 dimensions Reliability of new learning style subscales=.74 to.87 compared to… –Reliability of old learning style subscales=.83 to.86 –Reliability of personality subscales=.86 to.95 Learning Style Orientation Measure

14 Validation Study (study 2) –Convergent validity Corr b/w similar measures of key construct =Corr b/w diff subscales of new learning style 01 to.23 should be comparable to… –Corr b/w similar measures of other constructs in the study Diff subscales of old learning style.23 to.40 Diff subscales of personality.01 to.27 Learning Style Orientation Measure

15 Validation Study (study 2) –Discriminant validity Corr b/w measures of similar constructs= Corr b/w new learning style subscales & old learning style =.01 to.31 Corr b/w measures of different constructs –Corr b/w new learning style & personality subscales is.01 to.55 –Corr b/w old learning style & personality subscales=.02 to.38 Learning Style Orientation Measure

16 Validation Study –Incremental validity (aka construct validity) Additional variance explained by old vs. new learning style measures in preferences for assessment & instruction Learning Style Orientation Measure DVLSOM (new) LSI (old) Subjective assessment.15.01 Interactional instruction.21.04 Informational instruction.06.00

17 Types of Validity Content Validity Criterion Validity Construct Validity Predictive Validity Concurrent Validity Convergent Validity Discriminant Validity Adapted from Sekaran, 2004