I can do maths

I can do maths

Revised Reports

Revised Reports

 2    I c a n d o M a t h s S u p p l e m e n t

PAT Maths Third Edition and I Can Do Maths revised reports Andrew Stephanou, ACER

Introduction PATMaths 3rd Edition The Progressive Achievement Tests in Mathematics Third Edition (PATMaths 3rd Edition) consists of a screening test (Test A) and seven tests of increasing difficulty. The tests cover Years 2 to 10 in Victoria, New South Wales and the Australian Capital Territory, or Years 3 to 11 in Queensland, South Australia, Western Australia and the Northern Territory. Year levels for Queensland, South Australia, Western Australia and the Northern Territory are indicated in brackets ( ). PATMaths 3rd Edition can be administered in group settings via a paper–pencil, multiple-choice test. I Can Do Maths I Can Do Maths (ICDM) is a test of beginning mathematical ability in the first three years of school. It is orally administered to students individually or in small groups and consists of items that are presented in the form of pictures. It contains open-ended and multiple-choice questions. Research This supplement was prepared in response to schools who wanted to assess mathematics achievement in lower year levels where norms were not available on the PATMaths 3rd Edition scale. Norms for Years 1(2) and 2(3) have been obtained with data from the ICDM norming study through the calibration of the two ICDM tests on the PATMaths 3rd Edition scale. Revised reports and tables have been prepared for Test A, Test 1 and the two ICDM tests. The PATMaths 3rd Edition scale was extended to lower levels to provide a numeracy continuum in an ACER research project. This was done to show the growth that occurred between March and September 2005 for students in Years 1(2) to 9(10). A total of 14 tests were prepared using items from ACER

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

I can do Maths Supplement

3

and NZCER PATMaths that had already been calibrated onto the same scale, a selection of ICDM items and new items. These tests were administered on two occasions making sure that no student attempted the same test twice. Equating of the items that were not already calibrated onto the PATMaths 3rd Edition scale was achieved through common items in some tests and through administration of two tests to the same students. Common case equating data were obtained from a sample of 93 Year 1 students, 95 Year 2 students and 120 Year 3 students. The selected ICDM items calibrated on the PATMaths 3rd Edition scale provided the link required to calibrate all ICDM items on this scale and consequently to obtain a score equivalence table for each of the two ICDM tests. A score equivalence table allows each test score to be translated into a scale score in patm units. The achievement of the ICDM Year 1 and Year 2 norming samples was then measured on the PATMaths 3rd Edition scale providing norms below Year 3 for PATMaths 3rd Edition tests. The relative positions of the selected ICDM items on the PATMaths 3rd Edition scale and on the ICDM scale were very close, validating the equating of the two scales. With this work it is now possible to compare results on the PATMaths 3rd Edition tests with results on the ICDM tests. From this process, we are now able to provide: • Norms for Years 1(2) and 2(3) for PATMaths 3rd Edition Test A and Test 1. • ICDM test scores scaled onto the PATMaths 3rd Edition scale, which allow progressive testing in numeracy from Year 1(2) to Year 10(11). List of revised reports and tables 1 Student reports for ICDM A and ICDM B 2 Norm tables for ICDM A and ICDM B 3 Combined reports for ICDM A and ICDM B 4 Transformation of test scores (%) to scale scores (Revised Figure 14 in PATMaths 3rd Edition manual)

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

4

I can do Maths Supplement

1 Student reports Student reports for ICDM Test A and ICDM Test B have been provided (see pages 5 and 6), as per the PATMaths 3rd Edition student reports described on pages 18, 19 and 87 – 94 of the manual. The scale in the PATMaths 3rd Edition manual extends from 5 to 105 patm units. The calibration of the relatively easier ICDM tests on the same scale requires an extension of this scale below the origin that was selected in 2004 when the PATMaths 3rd Edition norming data were analysed. It is necessary to show the scale from - 15 to 85 patm units in order to display the location of the ICDM items and the transformation of the ICDM test scores to scale scores. The reference year levels for norming the two ICDM tests are 1(2) and 2(3). Year levels for Queensland, South Australia, Western Australia and the Northern Territory are indicated in brackets ( ).

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

I can do Maths Supplement

5

ICDM A Student Report Scale score (patm)

Test score

Student name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student year Level: . . . . . . . . . . . . . . . . . . . . . . . . . .

85

Date of testing: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Test raw score: . . . . . . . . .

Year 3(4)

Scale score(patm): . . . . . . ± . . . . . .

75

Year level of reference group: . . . . . . . . . . . . . . . . . .

9 95

90

80

70 6 9 95 8

30 65

7

Year 2(3)

60

55

7

50 50 5

28

95 8 90

60

40

3

50 5 10

35

30

30 4 2 20

5 3

5

40 30 4

2 1

29

25 1

20

28 22

25 27

24

22 21

26 25 21

20

23

19 15

18

24 20

16

17

20 3

10

16 15

10 5

27

23

10

50 5

30

26

20

60

80 70 6

4

70 6

40 7

45

40

80 30

9

29

60

90

Year 1(2)

Percentile rank: . . . . . . . . . .Stanine: . . . . . . . . . . . . .

70 8

2 1

5

0

14 13 12

18 19 13

17

11

10

9 8

-10

7 6

-15

12

14

11

9

10 -5

15

6

5 7

3

4

5 4

8 2

Number

1

Space

Measurement

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

6

I can do Maths Supplement

ICDM B Student Report Scale score (patm)

Test score

Student name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student year Level: . . . . . . . . . . . . . . . . . . . . . . . . . .

85

Date of testing: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Year 3(4)

33

Scale score(patm): . . . . . . ± . . . . . .

75

Year level of reference group: . . . . . . . . . . . . . . . . . .

9 95

70 8

90

80

70 6 9 95 8

60

31

55

30

50

29

60 50 5

7

30 9 95 8 90

60

40

3

50 5 10

35

30

25 1

10

50 5 5

40 30 4 20 3

2 1

20

5

1

27 26

28

24 21

22

20 19

15

20 24 25 23

18

16 22

17

18

15

5

13

17

12

6

11 10 9

0

15

9

14

10

26

19

16

10 2

27 31 30 29

21

2 5 3

33

23

30 4 20

32

25

20

60

80 70 6

4

70 6

40 7

45

40

80

Percentile rank: . . . . . . . . . .Stanine: . . . . . . . . . . . . .

28

90

Year 1(2)

32

65 7

Year 2(3)

Test raw score: . . . . . . . . .

11

13

14

12

8

4

10 5

8 7

-5

-10

6

7

5

3

4 -15

1

3

2

Number

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

Space

Measurement

I can do Maths Supplement

7

2 Norm tables Norm tables for ICDM Test A and ICDM Test B have been provided (see pages 8 and 9), as per the PATMaths 3rd Edition norm tables described on pages 16, 17 and 63 – 70. The reference year levels for norming the two ICDM tests are Years 1 and 2. Although the norming data for Years 1 and 2 are the data collected for the ICDM tests, the norms presented here may not be identical to those reported in the ICDM guide (p. 15), due to the different norming methodologies adopted in the two studies and to the different way of displaying the results. Comparison of results obtained with ICDM and PATMaths 3rd Edition tests should be based on scale scores in patm units. Test scores can be transformed into scale scores using the norm table provided for each test. A given difference in scale scores anywhere on the scale corresponds to the same amount of change in the construct because scale scores are measures on an interval scale. Further, as errors have been reported for scale scores, it is possible to test whether the difference between a student’s achievement on the scale as measured on two occasions, with the same or different tests, is statistically significant. Some stanines appear to be missing on the ICDM A table. For example, if you look at a student who gets a test score of 27 on the ICDM A, she will come out at Year 2(3) stanine 6, and one who scores 28 will come out at stanine 8. So it appears as if stanine 7 is missing from the norm tables. However, a score of 27 means the student will score better than 77% of the reference sample, which corresponds to a stanine of 6. There is also a possibility that the student falls into stanine 7 due to measurement error, but we have chosen to include a conservative estimate of their abilities. Also, test scores, scale scores and percentiles always contain some error and should be viewed as intervals rather than specific points (see the Error column in the tables). As shown in the tables, there tend to be larger errors associated with more extreme scores.

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

8

I can do Maths Supplement

ICDM A supplement norm table Test score (number Scale score Error correct) (patm) (patm)

Percentile rank and stanine Year 1(2) %ile rank

Year 2(3)

stanine

%ile rank

30

67

>10.7

99

99

29

55.7

10.7

99

98

28

47.2

8.1

99

27

41.6

7.0

98

77

26

37.2

6.3

96

62

25

33.5

5.9

92

24

30.2

5.6

86

23

27.1

5.4

79

22

24.3

5.2

70

21

21.7

5.1

60

20

19.2

5.0

50

19

16.8

4.9

41

18

14.5

4.8

32

17

12.2

4.7

24

16

10.0

4.7

18

15

7.8

4.7

13

14

5.7

4.6

9

13

3.5

4.7

6

12

1.4

4.7

11

-0.9

4.7

10

-3.1

9 8

9

8 7 6 5

90

47 35 24 16 10 6

Year 3(4)

stanine 9 8 6 5 4 3 2

%ile rank

stanine

91

8

70

6

47

5

31

4

21 14 9 6 2 1

4

1 1

1

1

1

1

1

1

1

1

1

1

3

1

1

2

1

1

4.8

1

1

-5.4

4.9

1

1

-7.9

5.0

1

1

1

7

-10.5

5.2

1

1

1

6

-13.2

5.4

1

1

1

5

-16.3

5.7

1

1

1

4

-19.7

6.1

1

1

1

3

-23.9

6.8

1

1

1

2

-29.1

7.9

1

1

1

1

-37.3

10.6

1

1

1

0

-48

>10.6

1

1

1

3 2

1

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

1

2

4

2

4

3

1 1

1

I can do Maths Supplement

9

ICDM B supplement norm table Percentile rank and stanine

Test score (number correct)

Scale score (patm)

Error (patm)

%ile rank

33

79

>10.7

99

99

32

68.5

10.6

99

99

31

60.2

8.0

99

99

30

54.8

6.8

99

98

29

50.6

6.2

99

28

47.0

5.8

99

90

27

43.8

5.6

99

83

26

40.8

5.3

98

75

25

38.0

5.2

97

65

24

35.4

5.1

94

55

23

32.9

5.0

91

22

30.5

4.9

87

21

28.1

4.8

81

Year 1(2)

Year 2(3)

stanine

9

8 7

%ile rank

95

45 36 27

stanine

9

8 7 6 5 4

%ile rank

stanine

98

9

93

8

80

7

68

6

56 46 37 29

7

25.8

4.8

75

4.7

67

18

21.4

4.7

59

17

19.2

4.6

50

16

17.1

4.6

42

15

15.0

4.6

34

14

12.8

4.6

26

13

10.7

4.6

19

12

8.5

4.7

14

11

6.3

4.7

10

10

4.1

4.8

6

1

1

9

1.7

4.9

4

1

1

8

-0.7

5.0

2

1

7

-3.3

5.2

1

1

1

6

-6.1

5.4

1

1

1

5

-9.2

5.7

1

1

1

4

-12.7

6.2

1

1

1

3

-16.9

6.8

1

1

1

2

-22.3

8.0

1

1

1

1

-30.6

10.7

1

1

1

0

-42

>10.7

1

1

1

10 5

4 3 2

1

6

2

2

5 3 2 1

4

1

2

1

1

1

1

1

1

1

1

1

1

3

9

23.6

3

4

13

20

14

5

22 17

19

6

19

Year 3(4)

1

1

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

10

I can do Maths Supplement

3 Combined reports A more detailed description of the combined reports is available in the PATMaths 3rd edition manual (p. 22). These provide qualitative descriptions of what the student is likely to achieve when they perform at a certain level on the PATMaths tests. The calibration of the ICDM items below the PATMaths 3rd Edition items has necessitated an extension of the description of the PATMaths 3rd Edition scale to lower levels. Scale descriptors based on ICDM items have been included.

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

I can do Maths Supplement

11

ICDM A Student Combined Report Scale score (patm)

Test score

85

Student name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student year Level: . . . . . . . . . . . . . . . . . . . . . . . . . . Date of testing: . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

80

Test raw score: . . . . . . . . . Scale score(patm): . . . . . . ± . . . . . .

75

Year level of reference group: . . . . . . . . . . . . . . . . . . Percentile rank: . . . . . . . . . .Stanine: . . . . . . . . . . . . .

70 30 65

60

At the level of ability shown a student is typically able to

55

solve addition and subtraction problems with larger numbers;

50

29

28 45

and also measure in centimeters;

40

30 27

29

26 35

and also understand three-digit numbers; calculate change and read a calendar; and also choose the correct number operation; understand order of events, and read clocks and simple graphs; distinguish from left to right; . and also continue a counting pattern;

.

30

20

27

24 23

25

28 22

25

22 21

26 25 21

20

23

19 15

18

24 20

16

17

and also solve problems involving numbers less than 10; recognise value of coins; recognise many simple geometric shapes and solids;

10

15 5

0

and also understand comparative terms such as ‘shortest’, ‘smallest’, ‘largest’; .

14 13 12

18 19 13

17

11

10

-5

9 8 7 6

-15

15 12

14

11

9

10

-10

and also count at least ten objects; understand ‘more’ and ‘less’. .

16

6

5 7

3

4

5 4

8 2

Number

1

Space

Measurement

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

12

I can do Maths Supplement

ICDM B Student Combined Report Scale score (patm)

Test score

85

Student name: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student year Level: . . . . . . . . . . . . . . . . . . . . . . . . . . Date of testing: . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

80

33

Scale score(patm): . . . . . . ± . . . . . .

75

70

Test raw score: . . . . . . . . . Year level of reference group: . . . . . . . . . . . . . . . . . .

32

Percentile rank: . . . . . . . . . .Stanine: . . . . . . . . . . . . .

65

At the level of ability shown a student is typically able to solve addition and subtraction problems with larger numbers;

60

31

55

30

50

28 45

and also measure in centimeters;

40

33 27

27

31 30 29

26

28

25 35

and also understand three-digit numbers; calculate change and read a calendar; and also choose the correct number operation; understand order of events, and read clocks and simple graphs; distinguish from left to right; . and also continue a counting pattern;

29

32

.

24 21

23

30

22 21

25

20

20 19

20 19 24 25 23

18

16 22

17

18

16 15

15

and also solve problems involving numbers less than 10; recognise value of coins; recognise many simple geometric shapes and solids;

5

12

6

10

11

13

14

12

8

4

10 5

8 7

-5

-10

and also count at least ten objects; understand ‘more’ and ‘less’. .

17

9

0

and also understand comparative terms such as ‘shortest’, ‘smallest’, ‘largest’; .

13 11

15

9

14

10

26

6

7

5

3

4 -15

1

3

2

Number

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

Space

Measurement

I can do Maths Supplement

13

4 Transformation of test scores (%) to scale scores A comparison of the R.difficulty of the eight PATMaths 3rd Edition tests (Test A and Tests 1 – 7) is on page 14. The test characteristic curves (TCC) for ICDM A and ICDM B have been added to those of the eight PATMaths 3rd Edition tests. An explanation on how to interpret this figure is given on page 58 of the PATMaths 3rd Edition manual. It can be observed that the TCCs of the ICDM tests are not parallel to those of the PATMaths 3rd Edition tests. In fact, the slope of a TCC depends on the distribution of the R.difficulties of the items of a test. The student reports show clearly that items in each of the ICDM tests are more widely distributed along the scale than items belonging to any given PATMaths 3rd Edition test.

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

14

I can do Maths Supplement

Transformation of test scores (%) to scale scores Scale score (patm) 105

100 95

90 85

80 75

Test 7 70 Test 6 65

Test 5 60 Test 4 55

Test 3 50 Test 2 45

40

Test 1

35

30 Test A

ICDM B

25

20

ICDM A

15

10 5

Test score (%) 0

10

20

30

40

50

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH

60

70

80

90

100

I can do Maths Supplement

15

References Doig, B. & De Lemos, M., (2000) I Can Do Maths Teacher’s Guide, ACER Press, Camberwell Effective Teaching and Learning Practices Initiative for Students with Learning Difficulties. (2006) Project reports in preparation, ACER. Lindsey, J., Stephanou, A., Urbach, D. & Sadler, A. (2005) Progressive Achievement Tests in Mathematics Third Edition Teacher Manual, ACER Press, Camberwell

COPYRIGHT © 2006 AUSTRALIAN COUNCIL FOR EDUCATIONAL RESEARCH