Questionnaire Construction: Items

Item Content !Judgments - refer to items where there is a correct response.

!Sentiments - refer to items that elicit responses about personal reactions, preferences, interests, attitudes, values, and likes and dislikes.

Closed Ended Items - Advantages !Comparability of answers. !Easier to code. !Item meaning clearer. !Answers tend to be complete and relevant. !Greater inclination to answer sensitive questions. !Easier to answer.

Item Response Categories !Closed Ended (Fixed Alternative) These items allow the subject to select from one or more categories provided by the questionnaire.

!Open Ended - These items do not specify response categories.

Closed Ended Items - Disadvantages !Guessing or random answering. !Inappropriate/irrelevant response categories. !Too many categories. !Undetected differences in interpretation. !Less variation in answers. !Higher likelihood of clerical error.

1

Open Ended Items - Advantages

Open Ended Items - Disadvantages

!Can use when all possible response categories are not known. !Allow for more detail, clarification, and qualification. !Can be used when there are too many potential answer categories to list. !Preferable for complex issues. !Allow more opportunity for exploration & self expression.

!May lead to collection of worthless & irrelevant information. !Data are not standard from person to person. !Low intercoder reliability. !Require superior respondent writing skills. !Questions may be too general for respondent to understand.

Item Construction Guidelines

Item Construction Guidelines - 2

!Avoid double barreled questions. !Make items clear.

!Break up response set opportunities. !Avoid negative items.

!Avoid biased items and terms. !Avoid long questions. !Make sure that respondents are competent to answer questions.

!Provide an explicit middle. !Carefully word threatening questions

Slope and Intercept

Correlation and Regression

2

Slope and Intercept

Slope/Intercept Equation Y = a + bx Y = 7 + 2X

X

7 + 2X

0 1 2 3 4 5 6 7 8 9 10

7 + 2(0) = 7 + 0 7 + 2(1) = 7 + 2 7 + 2(2) = 7 + 4 7 + 2(3) = 7 + 6 7 + 2(4) = 7 + 8 7 + 2(5) = 7 + 10 7 + 2(6) = 7 + 12 7 + 2(7) = 7 +14 7 + 2(8) = 7 +16 7 + 2(9) = 7 +18 7 + 2(10) = 7 + 20

Y =7 =9 = 11 = 13 = 15 = 17 = 19 = 21 = 23 = 25 = 27

Slope/Intercept Equation

The Regression (Prediction) Line

The Regression (Prediction) Line

The Regression (Prediction) Line !Ypred = a + bX "Ypred is the predicted value of Y "a is the Y-intercept, and " b is the slope

!E = Y - Ypred !Y = Ypred + E !Y = a + bX + E

3

Error and Prediction  E = Y - Ypred

Error Sum of Squares SSE

!When X = 2, Y = 4 and Y = 5.  The predicted value of Y is 

 SSE = Σ(E)² = Σ(Y-Ypred)²

Ypred = 3.5 + 0.5(X) = 3.5 + 0.5(2) = 3.5 + 1.0 = 4.5

 The prediction errors for the two values of Y are  



Y = 4: E = Y - Ypred = 4 - 4.5 = -0.5, & Y = 5: E = Y - Ypred = 5 - 4.5 = +0.5

Slope and Intercept Computational Formulae

Computing the Regression Coefficients

 Slope 

Y 0 2 4 6 8

SSXY b = ——— SSX  

_ _ _  where  SSXY = Σ(X-X)(Y-Y), & SSX = Σ(X-X)²

 Y Intercept __ __ a = Y - bX  

Computing the Regression Coefficients Y 0 2 4 6 8

_ Y-Y 0-4 = -4 2-4 = -2 4-4 = 0 6-4 = +2 8-4 = +4

X 1 3 2 5 4

_ X-X 1-3 = -2 3-3 = 0 2-3 = -1 5-3 = +2 4-3 = +1

X 1 3 2 5 4

Computing the Regression Coefficients _ _ _ _ _ _ Y Y - Y (Y - Y)² X X - X (X - X)² (Y - Y)(X - X) 0 -4 16 1 -2 4 (-4)(-2) = +8 2 -2 4 3 0 0 (-2)(0) = 0 4 0 0 2 -1 1 (0)(-1) = 0 6 +2 4 5 +2 4 (+2)(+2) = +4 8 +4 16 4 +1 1 (+4)(+1) = +4 SSY = 40 SSX = 10 SSXY = +16



_ _ _  SSX = Σ(X-X)² = Σ(X-X)(X-X) _ _   SSXY = Σ(X-X)(Y-Y)

4

Computing the Regression Coefficients  Slope 

Computing the Regression Coefficients

SSXY +16 b = —— = —— = +1.6 SSX 10

      

 Y Intercept __ __ a = Y - bX = 4 - (+1.6)(3) = 4 - 4.8 = -0.8

Computing the Regression Coefficients: Exercise Y X 10 0 8 2 6 4 2 6 4 8

Sums of Squares Type

Formula

Definition

Regression

_ SSR = Σ(Ypred - Y)²

Error

SSE = Σ(Y - Ypred)²

Total

_ SST = Σ(Y - Y)²

A measure of the total variability of predicted score values around the mean A measure of the total variability of obtained score values around their predicted values A measure of the total variability of obtained score values around the mean

Computing the Regression Coefficients: Setup _ _ _ _ _ _ Y Y - Y (Y - Y)² X X - X (X - X)² (Y - Y)(X - X) 10 0 8 2 6 4 2 6 4 8 SSY = SSX = SSXY =

Sums of Squares

_ _ Y - Y = (Y - Ypred) + (Ypred - Y)

SST = SSE + SSR .

5

Standard Error of Estimate !Variance Error 

SSE  sE2 = ——  n-2

!Standard Error of Estimate  sE = √(sE2)

Standard Error of Estimate: Exercise !Total Sum of Squares = 3884.550 !Regression Sum of Squares = 1413.833 !Error Sum of Squares = 2470.717 !Prediction Equation:  Ypred = 9.232 + 0.817X

Standard Error of Estimate

Standard Error of Estimate

Proportion of Variance Explained (PVE)

Coefficient of Determination











SSR PVE = ——— SST (SSXY)2  PVE = ———— SSX SSY 

 

r2

 



SSR = PVE = ——— SST

SSE 1 - r2 = 1 - PVE = ——— SST

6

PVE: Exercise

PVE

!Total Sum of Squares = 3884.550 !Regression Sum of Squares = 1413.833 !Error Sum of Squares = 2470.717 !Prediction Equation:  Ypred = 9.232 + 0.817X !Standard Error of Estimate = 11.72

PVE: Exercise

Correlation and Regression

!Total Sum of Squares = 2363.750 !Regression Sum of Squares = 1745.624 !Error Sum of Squares = 618.126 !Standard Error of Estimate = 5.86 !PVE = ?

The Correlation Coefficient 

SSXY r = ————— √SSX SSY











SSXY b = ——— SSX

Computing the Correlation Coefficients _ _ _ _ _ _ Y Y - Y (Y - Y)² X X - X (X - X)² (Y - Y)(X - X) 0 -4 16 1 -2 4 (-4)(-2) = +8 2 -2 4 3 0 0 (-2)(0) = 0 4 0 0 2 -1 1 (0)(-1) = 0 6 +2 4 5 +2 4 (+2)(+2) = +4 8 +4 16 4 +1 1 (+4)(+1) = +4 SSY = 40 SSX = 10 SSXY = +16

SSXY +16 +16 +16 r = ———— = ———— = ——— = —— = +0.8 √SSX SSY √(10)(40) √400 20

7

Correlation

Standard Error of Estimate (The Easy Way)



 

Spearman Rho (rs )

!Used when X and/or Y are ordinal variables !Procedure "Assign ranks to X # RX "Assign ranks to Y # RY "For each pair, compute d = RY - RX 6Σd "rs = 1 - ———— n(n2-1)

Applied Measurement Theory

sE =

sY√((1-r2)

n-1 ———) n-2

Phi (φ) Coefficient !Used when X and Y are nominal variables !Procedure (BC) - (AD) φ= ————————— √(A+B)(C+D)(A+C)(B+D)

Obtained, True, and Error Scores X=T+E

Reliabilty 0 ≤ rkk ≤ 1

X is the observed score T is the true score, and E is the error score.

8

Evaluating Reliability Score Variance

σX2 = σT2 + σE2

Procedures for Evaluating Reliability !Retest (Stability) !Parallel Forms (Equivalence)

The Reliability Coefficient

σT2 rkk = r2XT = —— σX2

Retest Reliability  rkk = r1st,2nd !One group of people !One testing procedure (instrument)

!Internal Consistency (Item Homogeneity)

!Two measurement times.

Internal Consistency Reliability Parallel Forms Reliability  rkk = rform a, form b !One group of people !Two testing procedures (instruments) !One measurement time.

!Split Half Estimation !Coefficient Alpha (Cronbach’s Alpha) !One group of people !One testing procedures (instruments) !One measurement time.

9