Comparing NAEP and the Next Generation Science Standards (NGSS) Teresa Neidorf, Yan Wang, Alka Arora, and Kim Gattis American Institutes for Research June 24, 2015 – CCSSO/NCSA – San Diego, CA

The NGSS and NAEP Frameworks K-12 Framework (NRC 2012) NAEP Science Framework

NGSS (Lead States 2013)

NAEP TEL Framework

NAEP Mathematics Framework

Overview of the Session I.

Introduction Goals of the study Overview of NGSS and NAEP frameworks

II. Methods I.

Results Science comparisons TEL comparisons Mathematics comparisons

II. Summary and Implications 3

Goals of the Framework Comparison Study  Determine the extent of alignment between the NGSS and the NAEP STEM frameworks • NAEP Science • NAEP Technology and Engineering Literacy (TEL) • NAEP Mathematics (supplements the science and TEL comparisons)

 Provide evidence of where NAEP includes measures of the student understandings described in the NGSS  Identify areas where NGSS and NAEP differ in content coverage • Inform ongoing discussions of NAEP’s role in emerging national and state systems of STEM assessments

Dimensions of the NGSS and NAEP Frameworks NGSS

NAEP Science

NAEP TEL

Disciplinary Core Ideas (DCIs) (Content Domains)

Content Areas

Assessment Areas

Scientific & Engineering Practices

Science Practices

TEL Practices

Crosscutting Concepts

NGSS Peformance Expectations integrate all 3 dimensions

NGSS: - Describe what ALL students should know and be able to do across grades K-12 NAEP Frameworks: - Guide development of assessments at grades 4, 8 and 12 - Provide a measure of student proficiency (across achievement levels) 5

Example NGSS Performance Expectation Table

Source: NGSS lead states (2013)

6

Comparing the Content Dimension NGSS

NAEP Science

NAEP TEL

Content Domains

Content Areas

Assessment Areas

Physical Sciences

Physical Science

Life Sciences

Life Science

Earth & Space Sciences

Earth & Space Sciences

Engineering, Technology, and Applications of Science (ETS) - Engineering Design - Links Among Engineering, Technology, Science, & Society

Design & Systems Technology & Society Information & Communication Technology (ICT)

Organization of Content NGSS

NAEP Science

NAEP TEL

Content Domain

Content Area

Assessment Area

Core Idea

Topic

Subarea

Component Idea

Subtopic

Performance Expectation (PE)

Content Statement

Lower Elementary (grades K-2) Upper Elementary (grades 3-5) Grades 4 & 8 Middle School (grade 8) High School (grade 12)

Assessment Target

Lower Elementary (grades K‐2) Upper Elementary (grades 3‐5) GradesMiddle School (grade 8) 4, 8 and 12 High School (grade 12) 8

Organization of Science Content NAEP

NGSS Content Domain

Earth & Space Sciences

Content Area

Earth & Space Sciences

Core Idea

Earth’s place in the universe

Topic

Earth in space and time

Component Idea

Earth and the solar system

Subtopic

Objects in the universe

Performance Expectation

MS‐ESS1‐2:  Develop and  Content use a model to describe  Statement the role of gravity in the  motions within galaxies  and the solar system.

E08.02: Gravity is the force  that keeps most objects in  the solar system in regular  and predictable motion.  Those motions explain such  phenomena as the day, the  year, phases of the moon,  and eclipses. 9

Comparing the Practices Dimension NGSS

NAEP Science & TEL

Scientific & Engineering Practices

Practices

1. Asking questions (science) and defining problems (engineering) 2. Developing and using models 3. Planning and carrying out investigations 4. Analyzing and interpreting data 5. Using mathematics and computational thinking 6. Constructing explanations (science) and designing solutions (engineering) 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information

NAEP Science 1. 2. 3. 4.

Identifying science principles Using science principles Using scientific inquiry Using technological design

NAEP TEL 1. Understanding technological principles 2. Developing solutions and achieving goals 3. Communicating and collaborating

10

Research Questions 1. Science: How similar (or different) are the NGSS performance expectations in physical sciences, life sciences, and Earth and space sciences to the content and practices in the NAEP science framework at the corresponding grade levels? 2. TEL: How similar (or different) are the NGSS performance expectations in engineering, technology, and applications of science to the content and practices in the NAEP technology and engineering literacy framework at the corresponding grade levels? 3. Mathematics: To what extent are the mathematics-related NGSS performance expectations and practices aligned with the content and skills specified in the NAEP mathematics framework, and at which grade (s)? 11

Methods – Science & TEL 1. Content mapping (AIR research team) • Grouped relevant portions of the NGSS performance expectations (PEs) with NAEP science and TEL framework objectives that covered related content at the corresponding grade level • Identified non-grouped NGSS PEs and NAEP objectives

2. Expert panel review and ratings (7 members) • Content similarity • Grade level alignment (science only) • Practices alignment

3. Aggregation & Analysis (quantitative and qualitative)

Content Mapping Example (Physical Sciences) NGSS

NAEP

HS-PS2-4 Use mathematical representations of Newton’s Law of Gravitation and Coulomb’s Law to describe and predict the gravitational and electrostatic forces between objects.

P12.22 Gravitation is a universal attractive force that each mass exerts on any other mass. The strength of the gravitational force between two masses is proportional to the masses and inversely proportional to the square of the distance between them. --P12.23 Electric force is a universal force that exists between any two charged objects. Opposite charges attract while like charges repel. The strength of the electric force is proportional to the magnitudes of the charges and inversely proportional to the square of the distance between them. Between any two charged particles, the electric force is vastly greater than the gravitational force. 13

Example Non-Grouped Science Objectives NGSS

NAEP

4-LS1-2 Use a model to describe that animals receive different types of information through their senses, process the information in their brain, and respond to the information in different ways

None

None

E08.07 The atmosphere is a mixture of nitrogen, oxygen, and trace gases that include water vapor. The atmosphere has a different physical and chemical composition at different elevations.

14

Expert Panel Ratings • Content Similarity - Each NGSS/NAEP grouping of objectives at the corresponding grade was reviewed and compared for similarity of content using a 4-point scale. 4 = Exactly or almost the same 3 = Quite similar, but with some differences 2 = Quite dissimilar, but with some overlap 1 = Substantially or wholly different

• Grade Level Alignment (science only) - Panelists identified alternative objectives at other grade levels that covered similar content. • Practices Alignment – Each NGSS performance expectation was rated for alignment with a NAEP practice (primary/secondary)

Example NGSS/NAEP Science Ratings DCI

NGSS

NAEP

Performance Expectation

Content Statement

Energy MS-PS3-1 Construct and interpret graphical displays of data to describe the relationships of kinetic energy to the mass of an object and to the speed of an object.

Content Rating (1, 2, 3, 4)

NAEP Science Practices

2

ISP

P8.8 Objects and substances in motion have kinetic energy

USP USI UTD

ISP USP USI UTD

Identifying Science Principles Using Science Principles Using Scientific Inquiry Using Technological Design

Results • Content Overlap – NGSS and NAEP objectives judged as covering related content at the corresponding grade level (final groupings determined by expert panel)

• Content Alignment – NGSS and NAEP objectives rated as “similar” (two-thirds or more of panelists assigned a rating of 3 or 4)

• Practices Alignment – NGSS performance expectations whose associated practices were aligned to a NAEP practice (most frequently identified by panelists as the “primary” practice)

Science Comparison Results

18

Content Overlap of NGSS Performance Expectations and NAEP Science Content Statements NGSS

NAEP

100% 84% 80%

84%

88%

74% 64%

60% 40%

29%

20% (14)

(33)

(55)

(43)

(67)

(49)

0% Grade 4

Middle School

High School 19

Alignment by Content Area NGSS Performance Expectations Aligned to NAEP Science Overall Physical Sciences Life Sciences Earth & Space Sciences 100% 80% 60% 40%

48%

47% 40%

36%

54%

53%

50%

42%

46%

42%

42%

29%

20% 0% Grade 4

Middle School

High School 20

Examples of Similar Content in Science Grade 4

Middle School

High School

Forms of energy

Evidence of chemical reactions

Patterns in the periodic table

Particulate nature of matter

Nuclear fission & fusion

Structures & functions in organisms Effects of weathering & erosion Human use of natural resources

Organs & organ systems

Gravitational & electric forces

Interactions in ecosystems Role of DNA Natural selection Model of the solar system Lithospheric plate movement

Energy flow through trophic levels Natural selection & evolution Big Bang theory Plate tectonics 21

Content & Grade Level Alignment NGSS Performance Expectations Aligned to NAEP Science 1 0.8 0.6 0.4 0.2

21%

36%

9% 47%

46% 6%

0 Grade 4 Lower Grade

Middle School Corresponding Grade

High School Higher Grade

22

Content & Grade Level Alignment NAEP Science Content Statements Aligned to the NGSS 100% 80% 60%

12% 23%

16%

40% 20%

71% 47%

56% 3%

0% Grade 4 Lower Grade

Grade 8 Corresponding Grade

Grade 12 Higher Grade

23

Content Emphasized at Different Grades NAEP

NGSS

Basic properties of light & sound Daily patterns in movement of Sun and moon Magnetic properties Life cycles of organisms Properties & changes in states of matter Force of Earth’s gravity

Grade 4

Grade 1 Grade 1 Grade 3 Grade 3 Grade 5 Grade 5

Relationship between energy and speed Periodic table Cell differentiation Earth’s layers and magnetic field

Grade 8

Grade 4 High School High School High School

Kinetic energy relationship to mass and speed Formula for electric force Genetic variation and effects of mutation Exothermic and endothermic reactions

Grade 12

Middle School Middle School Middle School Middle School 24

Examples of Unique Content in Science NAEP

NGSS

Grade 4: Descriptions of relative motion

Grade 4: Knowledge of the senses

Grade 8: Properties of acids & bases

Middle School: Role of brain processing

Grade 12: Different types of particle motion (translation, rotation & vibration)

High School: Chemical reaction rates and equilibrium Across Grades: Waves and their application in technologies for information transfer

25

Example Science Comparisons NGSS PE

Rating

Summary Statement

MS-LS4-6: Use mathematical representations to support explanations of how natural selection may lead to increases and decreases of specific traits in populations over time.

Similar

Both NGSS and NAEP include natural selection at grade 8. Both frameworks address the effects of natural selection on traits within a population.

MS-LS3-1: Develop and use a NGSS Only model to describe why structural changes to genes (mutations) located on chromosomes may affect proteins and may result in harmful, beneficial, or neutral effects to the structure and function of the organism.

L08.11 Using Science Principles Only NGSS includes the possible effects of genetic mutation in middle school. The concept of genetic mutation is included at grade 12 in NAEP (L12.09). Using Science Principles

26

NGSS Performance Expectations Aligned with NAEP Science Practices Identifying Science Principles Using Scientific Inquiry High School 3% Middle School

7%

Grade 4 Overall 4%

Using Science Principles Using Technological Design

64%

18%

55%

27%

57%

21%

60%

0% 20% 40% NAEP Framework Targets: 20 – 30% Identifying Science Principles 30 – 40% Using Science Principles 30% Using Scientific Inquiry 10% Using Technological Design

11% 21%

22% 60%

13%

80%

13% 100%

27

TEL Comparisons

28

TEL – Technology and Engineering Literacy Assessment  TEL aims to measure whether students are able to apply their technology and engineering knowledge and skills to real-life situations.  TEL is computer-based and uses interactive scenariobased tasks to gauge what students know and can do  National administration (8th grade) in 2014  Introduction to TEL: https://nces.ed.gov/nationsreportcard/tel/

29

TEL – Content Areas and Practices Content Areas: Design & Systems Technology & Society Information & Communication Technology (ICT) Practices: Understanding technological principles Developing solutions and achieving goals Communicating and collaborating 30

TEL – Assessment Targets Students know that: D.8.2: Technology advances through the processes of innovation and invention. Sometimes a technology developed for one purpose is adapted to serve other purposes. Students are able to: D.8.4: Simulate tests of various materials to determine which would be best to use for a given application.

31

NGSS – Performance Expectations Engineering, Technology, and Applications of Science (ETS) ETS1: Engineering Design ETS2: Links Among Engineering, Technology, Science, and Society Performance expectations in:  Engineering Design  Sciences with connections to ETS

32

TEL Content Mapping Example (Engineering Design) NGSS

NAEP TEL Framework

MS-ETS1-1 Develop a model to generate data for iterative testing and modification of a proposed object, tool, or process such that an optimal design can be achieved.

D.8.5 Redesign an existing tool to make it easier to accomplish a task. --D.8.9 Construct and test a model and gather data to see if it meets the requirements of a problem. --D.8.18 Modify a moderately complicated system so that it is less likely to fail. Predict the extent to which these modifications will affect the productivity of the system.

ETS1.B: Developing possible solutions ETS1.C: Optimizing the design solution

Design & Systems: Nature of Technology / Engineering Design / Maintenance and Troubleshooting

33

Example NGSS/NAEP TEL Ratings – Similar (ETS) NGSS

NAEP

Content Rating

Performance Expectation

Assessment Target(s)

(similar, not similar, NGSS only)

MS-ETS1-2: Evaluate competing design solutions using a systematic process to determine how well they meet the criteria and constraints of the problem.

D.08.07 D.08.08 D.08.09

Similar Both include designing, constructing, testing, and defining models in the design process.

NAEP TEL Practice: Developing solutions and achieving goals ETS1.B: Developing possible solutions

Example NGSS/NAEP TEL Ratings – Not Similar (ETS) NGSS

NAEP

Content Rating

Performance Expectation

Assessment Target(s)

(similar, not similar, NGSS only)

MS-ETS1-3: Analyze data from tests to determine similarities and differences among several design solutions to identify the best characteristics of each that can be combined into a new solution to better meet the criteria for success. ETS1.B: Developing possible solutions ETS1.C: Optimizing the design solution

D.08.04 D.08.06 D.08.09

Not similar Although both include generating multiple solutions and evaluating solutions, NAEP does not include identifying the best characteristics and combining them into a new solution.

NAEP TEL Practice: Developing solutions and achieving goals

Example NGSS/NAEP TEL Ratings – Similar (Science 1) NGSS

NAEP

Performance Expectation

Objective(s)

MS-PS3-3: Apply scientific principles to design, construct, and test a device that either minimizes or maximizes thermal energy transfer.

TEL: D.08.06 D.08.08 D.08.09

ETS1.A: Defining and delimiting an engineering problem ETS1.B: Developing possible solutions PS3.A: Definitions of energy PS3.B: Conservation of energy and energy transfer

Content Rating (similar, not similar, NGSS only) Similar Both include designing, constructing, testing, and defining models in the design process. Items/tasks in NAEP TEL would provide the necessary physical science concepts. NAEP TEL Practice: Developing solutions and achieving goals

Science: P08.10

Similar – NAEP science includes thermal energy transfer at grade 8. NAEP Science Practice: Using technological design

Example NGSS/NAEP TEL Ratings – Similar (Science 2) NGSS

NAEP

Performance Expectation

Objective(s)

MS-LS2-5: Evaluate competing design solutions for maintaining biodiversity and ecosystem services.

TEL: D.08.06 D.08.08 T.08.07

ETS1.B: Developing possible solutions ETS2.B: Influence of science, engineering, and technology on society and the natural world LS2.C: Ecosystem dynamics, functioning, and resilience

Content Rating (similar, not similar, NGSS only) Similar Both include generating and evaluating multiple solutions and considering the impact of humans and technology on the environment. Items/tasks in NAEP TEL would provide the necessary life science concepts. NAEP TEL Practice: Developing solutions and achieving goals

Science: L08.08

Not similar – NAEP science does not have a focus on maintaining biodiversity at grade 8.

Example NGSS/NAEP TEL Ratings – Not Similar (Science) NGSS

NAEP

Performance Expectation

Objective(s)

MS-PS1-3: Gather and make TEL: T.08.03 sense of information to describe that synthetic materials come from natural resources and impact society. ETS2.A: Interdependence of science, engineering, and technology ETS2.B: Influence of engineering, technology and science on society and the natural world PS1.A: Structure and properties of materials PS1.B: Chemical reactions

Content Rating (similar, not similar, NGSS only) Not similar NGSS focuses on the impact of synthetic materials produced from natural resources. TEL more broadly includes positive and negative impacts on society from the introduction of new or improved technology. NAEP TEL Practice: Understanding technological principles

Science: None

The production and impact of synthetic materials is not explicitly included in NAEP science.

Example NGSS/NAEP TEL Ratings – NGSS only NGSS

NAEP

Performance Expectation

Objective(s)

TEL: MS-LS1-1: Conduct an None investigation to provide evidence that living things are made of cells; either one cell or many different numbers and types of cells

Content Rating (similar, not similar, NGSS only) NGSS only Requires the use of specific technology to provide scientific evidence and is not included in the NAEP TEL framework.

ETS2.A: Interdependence of science, engineering, and technology LS1.A: Structure and function

Science: L08.01

Similar – NAEP science includes the cellular makeup of organisms at grade 8. NAEP science practice: Using scientific inquiry

NGSS Performance Expectations in ETS Aligned with NAEP TEL Assessment Targets Number and Percent Aligned with TEL Grades 3-5 Total Number Engineering Design

Middle School

Aligned

Total Number

3

3

Science (with connections to ETS)

4

Total

7

High School

Aligned

Total Number

Aligned

4

3

4

3

3

11

5

16

6

6

15

8

20

9

(86%)

(53%)

(45%)

NGSS Performance Expectations in ETS Aligned with NAEP TEL Assessment Targets 100%

100% 86%

75%

80% 60%

75%

75%

53% 45%

45% 38%

40% 20%

(3) (4)

(15) (4) (11)

(20) (4) (16)

Grade 4 (3-5)

Middle School

High School

(7)

0% Total

Engineering Design

Science (ETS)

NAEP TEL Assessment Targets Aligned with NGSS Performance Expectations Number and Percent Aligned with NGSS Grade 4

Total Number Design and Systems

Grade 8

Aligned

Total Number

19

6

Technology and Society

15

Information and Communication Technology Total

Grade 12

Aligned

Total Number

Aligned

19

6

19

5

0

15

2

15

5

13

0

13

0

13

3

47

6

47

8

47

13

(13%)

(17%)

(28%)

NAEP TEL Assessment Targets Aligned with NGSS Performance Expectations 100% 80% 60% 40% 20%

32%

17%

13%

13%

0% 0%

0%

Grade 4 Total

33% 28%26% 28%

32%

Design & Systems

0% Grade 8

Grade 12

Technology & Society

ICT

NGSS Performance Expectations Aligned with NAEP TEL Practices Overall

12%

62%

High School 5% Middle School

55% 20%

Grade 4 (3-5)

15% 60%

14% 0%

7%

86% 20%

40%

Understanding Technological Principles Developing Solutions & Achieving Goals Communicating & Collaborating

60% 30% 40%

80%

100%

NAEP TEL Targets

30% 44

Results - Similarities  There was strong alignment between NGSS and NAEP TEL in the area of engineering design • Defining design criteria and constraints • Generating and comparing multiple design solutions • Developing and testing models to optimize design solutions

 There was also some alignment on “effects of technology on society and the natural world”

45

Results - Differences • NAEP TEL is not aligned with NGSS performance expectations in ETS that require the application of specific science concepts • NAEP TEL covers a much broader range of content than what is included in the ETS performance expectations in NGSS. • Design & Systems: Nature of technology; systems thinking; maintenance & troubleshooting • Technology & Society Interaction of technology & humans; effects of technology on the world of information & knowledge; ethics, equity, and responsibility • Information & Communications Technology (ICT) 46

Mathematics Comparisons

47

Research Question To what extent are the mathematics-related NGSS performance expectations and practices aligned with the content and skills specified in the NAEP mathematics framework, and at which grade(s)?

48

Dimensions of the NGSS and NAEP Mathematics Framework NGSS

NAEP Mathematics

Disciplinary Core Ideas (DCIs) Content Areas (Content Domains) Scientific & Engineering Practices

Mathematical Complexity

Crosscutting Concepts

49

Comparison of NGSS and NAEP Frameworks - Practices Dimension NGSS

NAEP Mathematics

Scientific & Engineering Practices

Content Strands

Number Properties and Operations 1. Asking questions and defining problems Measurement 2. Developing and using models 3. Planning and carrying out Geometry investigations 4. Analyzing and interpreting data Data Analysis, Statistics, and 5. Using mathematics and Probability computational thinking 6. Constructing explanations and Algebra designing solutions 7. Engaging in argument from evidence NGSS Crosscutting Concept: 8. Obtaining, evaluating, and Scale, proportion and quantity communicating information

Mathematics Methodology

51

Method Used the same three stages as Science and TEL however, the approach and procedures were different • Standard Mapping • Expert panel review and ratings • Aggregation & Analysis (quantitative and qualitative)

52

Step I AIR’s NAEP mathematics experts presented initial mappings between the NGSS mathematics-related performance expectations and the NAEP mathematics framework objectives and levels of mathematical complexity to the Science and TEL Expert Panel.

53

Example NGSS

NAEP Mathematics

Cognitive Dimension

Performance Expectation (PE)

Content Area – Data, Statistics and Probability

Levels of Complexity

ESS2-1 Represent data in tables and graphical displays to describe typical weather conditions expected during a particular season.

Subtopics:

Low

Data Representation including specific types of graphical displays by grade level

Moderate High

Characteristics of Data Sets Mathematical Reasoning with Data 54

Step II The Science and TEL Expert Panel proposed using the NGSS “Common Core State Standards Connections” for Mathematics as a starting point. AIR ‘s NAEP Validity Studies Panel had conducted a framework comparison between the CCSS Mathematics and the NAEP Mathematics Framework so AIR’s mathematics experts investigated using the results as a potential link between NGSS and NAEP. “A Study of the Alignment Between the NAEP Mathematics Framework and the Common Core State Standards for Mathematics (CCSS-M).” available at air.org. 55

Mathematics Comparison Procedure (NVS report)

• NGSS Performance Expectations (NGSS)

• Common Core State Standards in Mathematics

• NAEP Mathematics Framework

• AIR prepared an initial mapping document (NGSS – CCSSM – NAEP) for NGSS performance expectations that have a mathematics related practice • Expert Panel (3 NAEP experts; 1 NGSS expert)

Content Mapping Example NGSS

CCSSM

NAEP

8-PS3-1

8-EE.A.2

8-NOP-2a

Construct and interpret graphical displays of data to describe the relationships of kinetic energy to the mass of an object and to the speed of an object.

Standard Description: Use square roots and cube root symbols to represent solutions to equations of the form x2 = p and x3 = p, when p is a positive rational number.

Establish or apply benchmarks for rational numbers and common irrational numbers in contexts.

[PE notes include examples of riding bicycle at different speeds or rolling different sized rocks downhill]

Evaluate square roots of perfect squares and cube roots of small perfect cubes. Know that square root of 2 is irrational.

8 –Alg-3c Perform basic operations, using appropriate tools, on linear algebraic expressions including grouping and order of multiple operations involving basic operations, exponents, roots, simplifying and expanding.

Mathematical Practice: Reason abstractly and quantitatively

57

Expert Panel Procedure • Reviewed each NGSS performance expectation to determine the mathematics content and skill that might be involved in items that could be developed to assess it • Determined by consensus if the mathematics identified was included in NAEP- this involved removing or adding NAEP objectives to the initial groupings • Considered adjacent grade levels in NAEP • Identified some additional performance expectations that did not have CCSSM connections specified 58

Mathematics Results

59

Results Most NGSS performance expectations involving mathematicsrelated practices were aligned to the NAEP framework. At grades 4 and 8, some portion of the mathematics involved was aligned with NAEP objectives at the next higher grade. • Grade 4: Percentages & rates; geometrical models; large time scales; physical attributes (area, volume, weight/mass) • Grade 8: Non-linear relationships; rate of change; working with multiple variables; mathematical reasoning with data.

60

NGSS Performance Expectations Aligned with NAEP Mathematics Objectives Across Grades NAEP Grade Level NGSS Grade Level Grade 4

Grade 4 Only

Grades 4&8

8%

92%

Middle School

Grade 8 Only

Grades 8 & 12

60%

27%

High School 100% 80% 60% 40% 20% 0%

14% 92%

Grade 12 Only

83% 83%

60% 27% 8% Grade 4

Middle School

14% High School

Example of ES PE-Aligned with the NAEP Fourth Grade Objectives  Practice: Analyze and interpret data to make sense of phenomena, using logical reasoning, mathematics, and/or computation.  Performance Expectation: Analyze and interpret data from maps to describe patterns of Earth’s features (4-ESS2-2)  Aligned with: Grade 4 Number and Algebra objectives • Related to graphing/ interpreting points on a grid and solving real world problems involving numbers.

62

Example of ES PE-Aligned with the NAEP Fourth and Eighth Grade Objectives  Practice: Develop and/or use models to describe and/or predict phenomena.  Performance Expectation: Develop a model to describe that light reflecting from objects and entering the eye allows objects to be seen (4-PS4-2)  Aligned with: Grade 4 and 8 Geometry objectives • Grade 4: Properties of paths between points, drawing angles, and identifying images resulting from reflections, translations, and rotations. • Grade 8: Describing the effect of reflection

63

Example of MS PE-Aligned with the NAEP Eighth and Twelfth Grade Objectives  Practice: Analyze and interpret data to provide evidence for phenomena  Performance Expectation: Analyze and interpret data to provide evidence for the effects of resource availability on organisms and populations of organisms in an ecosystem.(MS-LS2-1)  Grade 8: Number, Measurement, Algebra, and Data  Grade 12: Algebra and Data objectives • quadratic or exponential relationships, bivariate data

64

Example of HS PE-Aligned with the NAEP Twelfth Grade Objectives  Practice: Apply concepts of statistics and probability (including determining function fits to data, slope, intercept, and correlation coefficient for linear fits) to scientific and engineering questions and problems, using digital tools when feasible.  Performance Expectation: Apply concepts of statistics and probability to explain the variation and distribution of expressed traits in a population.(HS-LS3-3)  Aligned with Grade 12: Number, Geometry, Algebra, and Data objectives • geometric models, probability, reasoning with data, functions

65

Example Mathematics Summary Table NGSS Practice NAEP Objectives Apply scientific reasoning, theory, and/or models to link evidence to assess the extent to which reasoning and data support the explanation or conclusion

Grade 12: Number, Measurement, Algebra, Data

Summary Statement NGSS includes applying scientific reasoning and evidence from ancient Earth materials, meteorites, and other planetary surfaces to construct an account of Earth’s formation and early history. (HS-ESS1-6) This is aligned with NAEP grade 12 objectives related to operating with many different kinds of numbers; measuring physical quantities involved in developing various models and assigning meaningful units to the measured values; using scale models; representing data in tables, charts, graphs, and spreadsheets appropriately and effectively; using statistical parameters to understand data sets; fitting a trend line on the data; reasoning with data; recognizing pattern in the data including progressions (arithmetic and geometric); modeling relationship observed in the data as functions (linear, quadratic, exponential, trigonometric); and solving equations or inequalities. 66

Summary and Implications

67

How much of the NGSS are covered by NAEP science and TEL?

68

Content Overlap with NAEP Science and TEL Frameworks (Percentage of NGSS PEs in each grade band) All Content Domains 100%

92%

90%

(45)

(59)

(71)

Grades 3-5

Middle School

High School

80% 60%

56%

40% 20% 0% 69

Content Alignment with NAEP Science and TEL Frameworks (Percentage of NGSS PEs in each grade band) 100%

All Content Domains

80% 60% 40%

49% 11%

54%

56%

10%

13% 3%

2% 20%

36%

41%

44%

Grades 3-5

Middle School

High School

0% Aligned to science Only Aligned to TEL Only

Aligned to both science & TEL 70

Summary and Implications  About half of the NGSS performance expectations in the sciences and engineering design were aligned with the NAEP science or TEL framework at the corresponding grade (upper elementary, middle school, and high school).  Assessments based on the NGSS and NAEP assessments would be aligned to some degree, but each would also have unique content and different emphases in terms of science and TEL practices.  Alignment of NGSS-based assessments with NAEP science assessments would likely be moderate at the middle and high school levels and lower at grade 4. 71

Summary and Implications (cont.)  The NGSS emphasize some content at different grades than the NAEP science framework. While NAEP may include similar content, assessment items may differ in terms of reading load, depth/breadth, and cognitive load at the different grade levels.  The content and practices embodied in NGSS performance expectations involving engineering design are not fully covered by either the NAEP science or NAEP TEL framework.  Both NAEP science and TEL assessments include interactive computer tasks (as well as hands-on tasks in science) that provide for a deeper measure of students’ understanding of science and engineering principles and their ability to fully engage in the practices. 72

Summary and Implications (cont.)  Tasks developed to assess the NGSS may require some mathematics that is beyond the corresponding grade in the NAEP mathematics framework. This suggests that NGSS-based assessments may differ from NAEP science and TEL assessments in terms of the mathematics and quantitative skills required.

73

Final Note This is a framework comparison. Assessment item comparisons that build on the results of this study can provide more complete information on the alignment between the NGSS and NAEP science, TEL, and mathematics.

74

Teresa Neidorf 202-403-6194 [email protected] 1000 Thomas Jefferson Street NW Washington, DC 20007 General Information: 202-403-5000 TTY: 887-334-3499 www.air.org

75