EDUCATIONAL

AND

PSYCHOLOGICAL MEASUREMENT

1977, 37

THE VALIDITY OF TRADITIONAL COGNITIVE MEASURES AND OF SCALES OF THE STUDY ATTITUDES AND METHODS SURVEY IN THE PREDICTION OF THE ACADEMIC SUCCESS OF EDUCATIONAL OPPORTUNITY PROGRAM STUDENTS WAYNE S. ZIMMERMAN, HENRY PARKS, AND KENNETH GRAY California State University, Los Angeles WILLIAM B. MICHAEL

University of Southern California

For a sample of approximately 100 Educational Opportunity Program (EOP) university students who had been admitted to a special summer preparedness program at a major metropolitan campus in the California State University College system, validity coefficients of traditional cognitive measures and of the scales of the Study Attitudes and Methods Survey (SAMS) were calculated with respect to each of three criterion measures of academic success. The most valid predictor was the Comprehensive Tests of Basic Skills, (Reading Vocabulary) measure, although high school grade point average and college entrance examination scores yielded statistically significant validity coefficients. In two of three stepwise multiple regression analyses, the SAMS—Academic Drive-Conformity and SAMS— Alienation scales provided the second and third greatest proportions of statistically reliable variance to the prediction of the academic criterion measure in question.

IN 1973 on one of the large metropolitan campuses in the California State University and College system a special preparedness program was offered during the summer for students who were scheduled that fall to enroll in the Educational Opportunity Program (EOP). These students were not qualified according to the regular entrance requirement, an acceptably high standing on a weighted combination of high school grade point average (HSGPA) and scores on a college entrance

465

466

EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT TABLEI

Intercorrelations, Means, and Standard Deviations of Seventeen Predictor and Three Criterion

(N

a

b

All decimal points omitted from correlation coefficients. Variables 17, 18, and 19 are criterion measures; all others

are

=

89-110)

predictors.

examination, either the College Board Scholastic Aptitude Test (SAT) (Educational Testing Service, 1973) or the American College Testing (ACT) Program Examination (Science Research Associates, 1973).

Throughout the increasing reading

program

a

great deal of emphasis

was

placed

on

and mathematical competencies as well as study skills, and evidence for readiness to pursue college work was sought from additional testing. This testing included administration of the Comprehensive Tests of Basic Skills (CTBS), Form R, Level 4, (CTB/McGraw Hill, 1973) and the Study Attitudes and Methods Survey (SAMS) (W. Michael, J. Michael, and Zimmerman, 1972). At the conclusion of the program, alternate forms of the CTBS examination (Form Q, Level 4) were administered to assess any changes in the level of competencies in reading and arithmetic that might have

WAYNE S. ZIMMERMAN, ET AL. Measures for

a

Sample of Students

occurred

during

the

in

an

Economic

eight-week

467

Opportunity Program (EOP)’-’

summer

experience, although these

not relevant to the purpose of this paper. The major purpose of this paper was to determine predictive validities for the non-traditional SAMS variables and to compare these

change

scores were

validities with those of traditional academic predictors, which in this instance included HSGPA and both the ACT and SAT. Three criteria of academic success were employed: Number of College Units Attempted (NA), Number of Grade Points Earned (GP), and College Grade Point Average (GPA). It was hoped that the findings might help to expand the information base for making decisions regarding the academic promise of the EOP students as well as for identifying candidates who might benefit most from counseling, study skills improvement classes, and tutoring services.

EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT

468

Method Brief

descriptions of and intercorrelations among 17 predictors (Variables 1-16, and Variable 20) and the three criterion measures (Variables 18-20) are given in Table 1. It should be noted that entering students were required to take only one entrance examination, either the ACT or SAT, and that because of the small sample size an estimated equivalent score was assigned for the examination not taken based on the score on the examination taken. This treatment accounts for the spuriously high correlation (.98) between the ACT and the SAT scores. It should also be noted that high raw scores on the three SAMS scales of Study Anxiety, Manipulation, and Alienation represent a presence of these attributes. Thus the negative validity coefficients for these scales suggest that a lack of anxiety, manipulation, or alienation is positively related to academic success. Complete data were available for 110, 102, 101, 102, 101, 89, and 98 students, respectively, on Variables 1-6; 7-9; 10; 11-12; 13-14; 15-16, and 20; and 17-19. Despite these small sample sizes, step-wise multiple regression analyses were undertaken for each of the three criterion measures (Variables 17, 18, and 19) in relation to a set of seven predictors (Variables 2, 4, 6, 7, 9, 15, and 20). It was concluded, however, that the small sizes precluded meaningful applications of related crisscross-validation efforts. The data for the step-wise multiple regression analyses are summarized in Table 2.

Summary of the Findings The following major results are summarized: 1. The validity coefficients for the subtests of

the CTBS with the a varied to with median value of .30. All but from.17 three criteria .46, three of the 24 coefficients were statistically significant beyond the .05 level. 2. The SAMS Academic Drive-Conformity scale and the Alienation Toward Authority scale yielded statistically significant coefficients of .22 and -.26, respectively, with the criterion measures of UA and GPA. 3. The validity coefficients for the ACT-SAT variables ranged from .26 to .39. All were statistically significant beyond the .05 level. 4. The High School Grade Point Average (HSGPA) predictor yielded statistically significant validity coefficients of .25, .34, and .33, respectively, with the NA, GP, and GPA criterion measures. 5. With one exception the CTBS Reading Vocabulary predicted all three criteria of academic success more accurately than did either HSGPA or the scores on college entrance examinations. 6. In the step-wise multiple regression analyses, the CTBS-Reading Vocabulary contributed the largest proportion of variance to the

WAYNE S. ZIMMERMAN, ET AL.

469

prediction of each of the three criterion measures. As can be seen from the entries in Table 2, High School GPA (Variable 20) entered as the second or third most important variable in the prediction of each one of the three criterion measures. In the prediction of the NA and GPA criterion variables, respectively, SAMS-Academic Drive-Conformity (Variable 2) and SAMS-Alienation (Variable 6) provided the second and third greatest proportions of variance. TABLE 2

Stepwise Multiple Regression Analyses in Predicting Each of Three Criterion (Variables 17, 18, and l9Ja _______

a

Decimal

points

on

all entries except F ratios omitted.

Measures

470

EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT

Conclusions To the extent that the summer preparedness program was effective, it might be argued that the validity coefficients of the predictor test variables which were administered prior to exposure to the program would be attenuated insofar as the students who were weakest in the attributes measured would theoretically have the most to gain from the experience. Nevertheless, based on the findings, the following tentative conclusions are offered: 1. The best single predictor of success of EOP students in the first year of their college experience following a summer preparedness program was the measure of reading vocabulary. This finding was in line with expectation based on hundreds of previously published studies involving prediction of college performance with standardized tests of verbal comprehension. The validity coefficients, although modest, compared favorably with those afforded by the traditional entrance examinations, in an earlier general sampling of the traditional entrance examinations at the college where this study was conducted (Zimmerman and Michael, 1967). That the CTBS Reading Vocabulary score proved to be a more valid predictor than was the ACT/SAT score might be largely due to the fact that since only total scores were used for these scholastic aptitude measures the more valid verbal variance was diluted with the less valid quantitative variance. 2. That two of the SAMS scales were the second and third most effective predictor variables in the first and third step-wise multiple regression analyses would suggest that these two scales may afford a modest degree of promise as supplementary indicators of the potential academic success of minority students. .

REFERENCES

CTB/McGraw Hill. Comprehensive Tests of Basic Skills. Monterey, Calif.: CTB/McGraw Hill, 1973. Educational

Testing

Service.

College

Board Scholastic

Aptitude Test.

Princeton, N. J.: Educational Testing Service, 1973. Michael, W. B., Michael, J. J., and Zimmerman, W. S. Study Attitudes and Methods Survey. San Diego: Educational and Industrial Testing Service, 1972. Science Research Associates. American College Testing Program Examination. Chicago: Science Research Associates, 1973. Zimmerman, W. S. and Michael, W. B. A comparison of the criterionrelated validities of three college entrance examinations with different content emphases. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1967, 27, 407-412.