Matisse Foundation Grant

Summer Bridge 2011 Program Evaluation

MS 223 Principal: Ramon Gonzalez New York City Public Schools November 5, 2011

Sarah Benis Scheier-Dolberg Teachers College, Columbia University

Summer Bridge 2011 Program Evaluation

2

Executive Summary This report is written for the Matisse Foundation, a primary funder of Summer Bridge at MS 223, The Laboratory School of Finance and Technology, beginning a 5-year evaluation effort. What is Summer Bridge? Serving over 100 students, the Summer Bridge integrated arts program launched in July 2011 to serve all incoming 6th grade students at MS 223, a public school in the South Bronx. In collaboration with the Matisse Foundation and professional artists and local community organizations, Summer Bridge was created to provide all students in the city the opportunity to experience high quality arts in order to positively impact the artistic, academic, and psychological well being of children. Evaluation Summer Bridge met (or surpassed) all program goals related to participation, art experiences, student attitudes, and school reform. Student achievement goals will be measured throughout the year. Process Goals 1. Participation: The program met its goal to serve a minimum of 100 students, serving 105 6th students (of 160 entering 6th grade and 477 students attending MS 223 over all). Ninety-seven students had perfect attendance.

2. Art Experiences: Summer Bridge met its goals of ensuring that all participating students (1) learn music theory, (2) select an instrument to learn, (3) sight-read a basic composition, (4) create at least two visual art pieces patterned after styles of notable artists but through the lens of their own experiences, (5) and visit cultural institutions. Only 38% of MS 223 incoming students had taken formal arts classes outside of school before attending Summer Bridge. 105 students selected an instrument, received two weeks of extended instruction, and performed at the program’s end.

All students attended a weeklong course in basic music theory and created original pieces of art; 95 students created at least five pieces of art (sculptures, printmaking, pop-art reflective self portraits, positive/negative space drawing). Prior to the program, less than 40% of MS 223 incoming students had visited one cultural institution. This summer all Summer Bridge students visited at least two cultural events.

Summer Bridge 2011 Program Evaluation

3

3. Student Attitudes The program slightly improved students’ views of themselves as artists, their inclination to involve their families in conversations regarding the arts, their positive attitude of the arts and its connection to improved performance in schools. 70% of the students reported they were above average artists (12% increase from start of program), and 61% of students reported talking with their families about the arts every day or most days (14% increase from start of program). In almost every survey response, girls tended to have a more positive attitude toward the arts than boys. 73% of girls reported that they loved art, and 84% of girls said the arts were important. 81% of students from non-English speaking homes reported that they wanted to learn more about the arts, and 61% thought the arts would help in other classes. Additionally, Summer Bridge positively impacted students’ attitudes toward reading and the number of books students read over the summer. After the program, 90% of the students reported that their reading had improved, and 90% of the students reported that they planned to read at least 4-6 more books before the start of the school year. Outcome Goals

4. School Reform: The program met its goal to invite principals from high-poverty schools to the Summer Bridge laboratory site for leadership development focused on closing the reading achievement gap through the arts. Seven principals and support staff visited MS 223 three times this summer to study innovative arts summer programming.

5. Student Achievement: Summer Bridge is continuing to collect data to assess whether or not it has met its second outcome goal related to student achievement. Only preliminary findings based on summer data are presented in this report. 105 students read 6 or more books during 3 weeks of Summer Bridge. Specifically in relation to the program’s goals to impact student achievement as well as demonstrate a quantifiable increase in students’ reading fluency and comprehension of one independent reading level or greater, more data needs to be collected during the 2011-2012 school year to accurately assess the program’s progress on this goal.

Summer Bridge 2011 Program Evaluation

4

INTRODUCTION This report presents preliminary findings to the Matisse Foundation and also institutes the first step in a five-year evaluation of the Summer Bridge Program, student achievement, and student attitudes regarding the arts and academics. MS 223 is a public district middle school in the New York City Department of Education. The school serves 480 students in grades 6-8. MS 223 opened in 2003 as a replacement for a persistently dangerous and underperforming junior high school in the Mott Haven neighborhood of the South Bronx. Since then, the school has grown in size and popularity for innovative programming, a safe learning environment. The principal of MS 223, Ramon Gonzalez, serves as the organizational head of the Summer Bridge Program and has been recognized for his educational leadership and as a visionary leader by such diverse organizations and publications as Smithsonian Magazine, The Wall Street Journal and The New York Times Magazine. The Summer Bridge Program at MS 223 launched in 2011 as an intensive, three-week integrated arts program for incoming 6th grade students. The program is a collaboration of several organizations, MS 223, Arete Education, Inc., Multicultural Music Group, Inc., and Yale Alumni Association and was funded by the Matisse Foundation. The program is founded on the belief that students need not just remediation but deep rich academic, artistic and cultural experiences to help them grow intellectually and socially. The purpose of the Summer Bridge Program is to close the achievement gap for students by preventing and reversing summer learning loss through a program of intensive learning in and through the arts. The program goals include: Process Goals 1. Participation: To serve a minimum of 100 students during the mandatory three-week program. 2. Art Experiences: (a) To ensure that all participating students learn music theory, select an instrument to learn, and sight-read a basic composition; (b) To provide all participating students an opportunity to create at least two visual art pieces patterned after styles of notable artists but through the lens of their own experiences and visit cultural institutions. 3. Student Attitudes: To measure and positively impact student attitudes regarding the arts and reading. Outcome Goals 4. School Reform: To invite principals from high-poverty schools to the Summer Bridge laboratory site for leadership development focused on closing the reading achievement gap through the arts.

Summer Bridge 2011 Program Evaluation

5

5. Student Achievement: (a) To document through surveys, interviews, and test scores the effects of art programming on student achievement; (b) To demonstrate a quantifiable increase in students’ reading fluency and comprehension of one independent reading level or greater. This report will answer the following research questions: 1. Did Summer Bridge succeed in serving 100 students over three weeks of programming? 2. Did students participate in music theory learning experiences, select an instrument, and learn to site-read a basic composition? 3. Did students participate in arts daily and create original pieces of art? 4. What were the student outcomes related to reading and the arts of Summer Bridge? 5. Did other principals from high-poverty schools visit to learn about arts programming? 6. What effects, if any, did arts programming having on student achievement? PROGRAM DESCRIPTION When arts instruction is woven into the curriculum, it has been shown to close gaps in student reading scores by nearly 15 points over a three-year period.1 The Summer Bridge Program asserts that if students experience high-quality art instruction, their attendance and academics will synergistically improve. However, arts funding for public schools has been repeatedly cut; this program was designed to remedy such cuts based on the school’s commitment to emphasizing the arts as a direct pathway to student achievement. Summer Bridge Program aims to use the arts to improve students’ engagement in their academic subjects, attendance, and overall feelings about school. The program’s central focus has been create a high-quality music and visual arts program that gives MS 223 students an in-depth sequential experience working with the best materials as they explore various media under the direction of professional artists. The focus of the program has been on sustaining reading and academic growth in and through the arts, enabling students to achieve on the level of their more advantaged peers. The program hopes to offer an artistic, academic, social, and physically active program that exposes students to opportunities they would not otherwise experience during the summer months. The program’s stated goal is to provide a comprehensive program that will meet intellectual, emotional, and creative needs of our students. Incoming 6th grade students at MS 223 participated in a range of activities across many subjects. In the morning students primarily focused on academic subjects and the arts were incorporated to strengthen academic instruction. In the afternoon, students studied music and art and participated in a sports program. Students participated daily 1

                                                                                                                                                       

Research findings from Montgomery County, Maryland as presented in Real Visions. (2007). Montgomery County Public Schools Arts Integration Model Schools Program, 2004-2007: Final evaluation report. Berkeley Springs, WV: Real Visions.

Summer Bridge 2011 Program Evaluation

6

in visual art or instrumental music instruction and worked under the guidance of professional art teachers as well as professional artists. All instructors share professional background as artists and the specialists are in addition, certified by New York State/City as educators. Instructors served as mentors to help students learn to work with new methods and materials and find connections to students’ personal environs and life experiences. The musical component allowed students to learn basic music theory and learn in small instrumental sections based on their choice of percussion, strings, woodwinds, or brass instruments. Those who preferred to concentrate on vocal studies worked with a choral instructor. Students rehearsed with a professional in their area and were performed as a complete ensemble for their families and peers at the conclusion of the program. Summer Bridge also provided multiple opportunities for students to participate in cultural arts events such as the Broadway shows, The Adams Family and Spiderman Turn up the Dark, and one museum visit. Through collaboration with the Center for Arts Education (CAE), MS 223 participated in the School Arts Support Initiative (SASI), which demonstrated the crucial link that school principals needed an opportunity to support each other with observations and strategies to ensure that arts programming remained vibrant in their schools. The program was designed for this summer but also to ensure scalability citywide, so Summer Bridge Program also included essential opportunities to share insights gained from this innovative program with principals from other high-poverty schools within a high-needs district of New York City. METHODS Data collection for a full five-year program evaluation is ongoing. Data sources for this preliminary evaluation of the 2011 Summer Bridge Program include program documents (attendance records, grant proposals, principal self-evaluation, teacher lesson plans, student reading logs, newspaper article), pre- and post-student surveys, principal and assistant principal interviews, and a student focus group. MS 223 leaders developed two pre-surveys, one in reading and one in the arts to administer to all students at the start of the program. Of the 105 students attending Summer Bridge, 101 students completed the pre-survey in reading and 97 students completed the pre-survey in the arts. At the end of the program, an outside evaluator designed the post-survey that combined questions from the two pre-surveys.2

2

                                                                                                                                                       

Administering one survey at the close the program helped to limit the summer programming time devoted to students taking surveys; 82 students took this post-survey. Because of the end-of-program performances and preparations, scheduling time for all students to take the survey online proved difficult, and fewer students were able to take the post-survey.

Summer Bridge 2011 Program Evaluation

7

After the completion of the program, 10 students were randomly selected to participate in a one-hour focus group with the outside evaluator. This focus group, the pre- and post-surveys, interviews with the principal, and analysis of various program documents were used to generate the results, discussion, and recommendations in this program evaluation.

RESULTS – PROCESS GOALS

Student Participation

Summer Bridge was able to achieve its key process goals for students. (1) Participation 105 students participated in the Summer Bridge three-week program, and 97 students had perfect attendance. Students in the focus group unanimously agreed that attending Summer Bridge was a positive experience and nine out of ten students wanted to attending Summer Bridge after completing 6th grade. (2) Art Experiences Prior to Summer Bridge Program, less than 40% of MS 223 incoming students had visited one cultural institution. This summer they visited at least two cultural events. Students in the focus group discussed their fascination with the arts they participated in as audience members of the Broadway shows The Adams Family and Spiderman Turn up the Dark. Before the program, 38% of MS 223 incoming students had taken formal arts classes outside of school.

Summer Bridge 2011 Program Evaluation (2) Art Experiences (continued) This summer 105 students selected an instrument, received two weeks of extended performance instruction, and performed in front of families at the end of the program. 101 students attended a weeklong course in basic music theory. 105 students created original pieces of art. 95 students created at least five pieces of art (sculptures, printmaking, pop-art reflective self portraits, positive/negative space drawing).

Student focus group data showed that these music and art projects were highlights of the program for students, especially the first day students received their instruments and the final day when they preformed for their parents and teachers. (3) Student Attitudes As part of its efforts to improve student achievement, Summer Bridge was also able to achieve its key goals connected to student attitudes toward the arts and reading.

Art Attitudes (all students) Seventy percent of the students reported they were good or excellent artists at the end of the program (12% increase from start of program). Sixty-one percent of students reported talking with their families about the arts every day or most days (14% increase from start of program). Students in the focus group talked about how much their parents supported the programming, both for its focus on the arts and its introduction to middle school academic work at MS 223.

8

Summer Bridge 2011 Program Evaluation

9

Art Attitudes (girls vs. boys) In almost every survey response, girls tended to have a more positive attitude toward the arts than boys. 73% of girls reported that they loved art, and 84% of girls said the arts were important. Art Attitudes (English language learners) 81% of students from non-English speaking homes reported that they wanted to learn more about the arts, and 61% thought the arts would help in other classes.

Reading Attitudes (all students) After the program, 90% of the students reported that their reading had improved, and 90% of the students reported that they planned to read at least 4-6 more books before the start of the school year.

Summer Bridge 2011 Program Evaluation

10

RESULTS – OUTCOME GOALS (4) School Reform Summer Bridge was able to attract school leaders from other high-need schools within the district. Seven principals and support staff visited MS 223 three times during the summer to learn about innovative arts summer programming. (5) Student Achievement All 105 students read six or more books over the summer during 3 weeks of Summer Bridge. Students in the focus groups by far spent more time talking about the books they read this summer than another other topic related to Summer Bridge. Students in the focus group discussed their favorite books at length. Regarding program goals relating to student achievement and reading levels, more data will need to be collected and analyzed during the 2011-2012 school year. While focus group data suggest that students saw their reading levels go up over the summer and voice a feeling of preparedness for middle school academics, more data is needed to substantiate focus group findings. These research areas will drive the five-year evaluation efforts that will examine survey data, independent reading levels, and comparison of performance data on 2011 and 2012 New York State ELA exams. DISCUSSION Perfect attendance for 92% of students enrolled in a mandatory summer program is unusually high3 and suggests that MS 223’s unique, integrated arts approach to improving student achievement during summer months has promise. Additional data from the student focus group, student surveys, and the principal visitations to MS 233 suggest that Summer Bridge may be a model for integrated arts middle school summer programming that should be further developed and expanded. Summer Bridge integrates half-day arts programming with half-day academic programming for incoming 6th graders that provides an orientation to high academic and arts expectations for students. Summer Bridge also extends the summer school experience beyond the school walls by inviting local artists and community groups in and taking students out of the building weekly to cultural institutions in the city. Data from the focus group and surveys suggest that students overall feel positive about Summer Bridge’s impact on their reading (90% indicating that their reading level has improved and that they plan to read at least 4 more books before the start of the school year) and readiness for middle school. Longitudinal data will be needed to draw 3

                                                                                                                                                       

While trends in summer school attendance in NYC have shown some improvement (reported to be 60% in 2003, 69% in 2010, and 75% in 2011 by the New York Times and New York Daily News), city averages for summer school attendance fall far below Summer Bridge attendance data for 2011.

Summer Bridge 2011 Program Evaluation

11

conclusions about the Summer Bridge Program gains throughout the school year and beyond. These preliminary data, however, may suggest that extending the summer program to four weeks will provide more opportunity for these short-term attitudinal gains. Providing students at least 30 hours of arts programming over the summer (the equivalent of one year of arts course credit) may also positively impact student attitudes and achievement if the program is expanded. Gender differences in the data regarding art programming in Summer Bridge stand out as well. Boys were more likely to hate art and less likely to see the importance of art in their lives or want to learn more about the arts. Girls’ overwhelming interest in the arts and engagement in the integrated arts programming at Summer Bridge should be studied further. The data also show some of the ways language may be impacting student achievement and attitudes regarding the arts and academics. English language learners were more likely to feel that art had an important role in their lives and that they wanted to learn more about the arts and attend specialized high schools in the arts than their native English speaking peers. The data also point to a potentially important link between arts and academics as 61% of English language learners felt that the arts would help them in other academic areas—as compared to 44% of students speaking native English. Having the notification of funds 2-3 months prior to the start of the program would allow MS 223 to recruit the best possible staff and meet family and staff vision for a longer 4-week program more likely to impact student achievement and the depth of student learning in the arts.4

4

                                                                                                                                                       

The principal in his self-evaluation noted the difficulties presented by the timing of state test data and funding notification. Without ELA test score data from the 2010-2011 school year, the Summer Bridge Program was limited in program evaluation design for this evaluation, unable to compare high and low proficiency scores with attitudinal survey data towards academics and the arts. The lateness of funding notification also limited MS 223’s ability to recruit staff and students, thus resulting in larger class sizes and a shortened 3-week program (instead of a four-week program that could meet state standards as a separate academic semester).

Summer Bridge 2011 Program Evaluation

12

RECOMMENDATIONS  Funding sources should be secured by the start of May 2012 to insure that proper staff can be recruited and trained for the 2012 Summer Bridge Program.  MS 223 should seek funding to expand the program to 3 weeks of programming for 6th graders and 4 weeks of programming for incoming 6th graders.  MS 223 should continue to seek funding to function as a laboratory site for integrated arts education during summer and out-of-school time, inviting school leaders to study the integrated arts model of Summer Bridge at MS 223.  MS 223 should ensure ongoing assessment of its Summer Bridge Program.  MS 223 should maintain its daily two-hour blocks for visual and performing arts.  MS 223 should maintain its commitment to field trips to cultural institutions and seek funding to increase the number of field trip opportunities, especially for visual arts. MS 223 should also invite parent volunteers on field trips to further extend the school-home connection between the arts and academic success.  Professional development for all program staff should begin in early June, so that all arts and academic teachers can collaboratively plan curriculum and align student performance and procedural expectations during the summer program.