Total Literacy, 2003-04

End of Year Report:  Hartford, CT

Implementation, Findings and Discussion

2003-04 was designed as a pilot year for Total Literacy in Hartford, CT.  The initiative was two-pronged.  There was a thread of professional development workshops for the 27 elementary Unified Arts teams throughout the year, with a goal of introducing integrated arts collaborations between the three arts, and deepening understanding of Total Literacy basic concepts in preparation for potential expansion in coming years.  In addition, based on promising findings from an eight-week pilot with three classes the previous year, McDonough School continued an intensive pilot Total Literacy initiative.

McDonough School

McDonough School was the site of teacher training during the summer for about 1/3 of the staff, then each of the four subsequent SFA cycles introduced a different treatment or pilot group.

Over the course of the year, several different implementation models and modifications were explored across the school from Grades 1-6, in classrooms of trained teachers.  The eight-week SFA cycles provided opportunities to assess progress, evaluate each implementation strategy, and modify accordingly.  We utilized:

• Professional development days;
• Model classes and team teaching by/with consultants;
• Weekly teacher conferences ranging from 15 to 40 minutes in length with focus on literacy concepts/skills and/or comprehension strategies including CMT strands;
• Rotation with the unified arts teachers during SFA reading time for classes of “stuck” readers ranging from three-days a week,1 ½ hour rotation through all three arts to one arts discipline three days a week, each art for two weeks;
• Visiting artist workshops;
• Creation of packets for KIDS Time centers; and
• Teacher self-assessment guides and rubrics.

In addition to the variety of implementation strategies, we worked with different student groups over the year, including

• Very low Roots[1];
• “Stuck” at Roots 16, 21, and 32;
• Bilingual moving from Spanish Roots 48 to English 16;
• Roots Level IV, both English and bilingual;
• Wings 2.1 “stuck” readers;
• Wings from grades 3-6, English and bilingual;
• Grade 1 KIDS Time

Data collection was systematic at the end of cycles 2, 3, and 4, and additionally the CMT and DRA scores.  Qualitative data has been provided through the evaluation of Charles Bruckerhoff (Hartford’s assessor), Bruce Wilson and Dick Corbett (IDEAS assessors), and through teacher journaling and conference discussions.  The quantitative data shows that Total Literacy has made a significant difference in student reading achievement, particularly in classrooms where the teachers embraced the strategies and used them frequently and well.  The qualitative data indicates that the initiative has many benefits beyond test scores, including social and emotional well-being of students and teachers, increased attendance and decreased behavior problems, teacher growth, and an increase in differentiated learning strategies and collaborative staff work.

Quantitative Data

Cycle 2 Quantitative Data

At the end of the second cycle, comparison was made to test scores the prior year, before the Total Literacy pilot began.  The Roots data was analyzed using two different scales, one (Absolute Gains Measure) of which is used for report cards (1.1 to 1.9) counting approximately five books as a level; and the other scale (Bucket Measure) used by the district, citing approximately each 15 books as a level.

The analyses yield the following broad results:

• Average growth in Total Literacy trained teachers’ classrooms was higher before the pilot began. 
• Across the entire school, the percentage of students who moved up at least one level, and average growth per class, went up from February 2003 to February 2004.
  o       Absolute Gains Measure – 21.8% change from Year 1 to Year 2.
  o       Bucket Measure – 22.4% change from Year 1 to Year 2.
• In Year 2, students in Total Literacy classrooms continued to achieve at a higher rate than those in non-Total Literacy classrooms.  There was an additional percentage of increase in Year 2 that can be attributed to Total Literacy training and implementation.
  o        With the absolute scale, the Total Literacy classrooms had a higher percent of students moving up than non-Total Literacy (p<.05).  Year 1 showed an 8.3 % difference.  Year 2 showed a 16.5% difference.
  o       With the bucket scale, the Total Literacy classrooms had a higher percent of students moving up.  Year 1 showed a 7.3% difference.  Year 2 showed an 8.2 % difference.  Total Literacy classrooms had .6% greater change than non Total Literacy classrooms, even though they started from a higher baseline.
• Gains in Wings (2.1 and up) were significantly (p<.05) higher for the Total Literacy trained teachers’ students.
  o       Absolute Gains Measure – difference of 15.4% between Total Literacy and non-Total Literacy classroom percentage of change from prior year.
  o       Bucket Measure – difference of 15.4% between Total Literacy and non-Total Literacy classroom change from prior year.
  o       These are both significant at the .05 level.
• There is some evidence of a positive effect for Total Literacy on Roots bilingual students, and Wings monolingual students.
• Although some cultural sub-groups are very small, there is a large percentage difference in students improving between Total Literacy and non-Total Literacy classrooms during the second cycle of 2003-04:  for Overall African American (34.7% difference), Overall Hispanic (9.7% difference), Roots Hispanic (16.3% difference), Wings African American (42.3% difference), Wings Hispanic (9.6% difference).
• As expected, Special education students in Total Literacy have reading difficulties, where non-Total Literacy special education students are most likely reading fairly well.  Therefore, the figures showing a smaller percent of Special Education Total Literacy students improving is attributed to their reading difficulties.  The fact that 26% of Wings and 22% of Roots special education “stuck” students improved is a tribute to the strategies and the teachers who implemented them.

Using the absolute scale, significant differences occurred both from year to year, and between Total Literacy and non-Total Literacy groups.  Using the bucket measure, Total Literacy showed higher results than non-Total Literacy classrooms, however these numbers are only significant for Wings.  

Regardless of which measure is used, the quantitative results are promising and suggest further exploration. 

Cycle 3 Quantitative Data

The initial data for the third eight-week cycle of 2003-2004 was analyzed with the following methodology.  In each classroom the percent of students advancing one or more levels during the eight-week cycle was tallied.  A simple chi-square (Χ²) analysis with 1df was run on the data, comparing the changes in TL versus non-TL classrooms from the 3^rd cycle of 2003 to the 3^rd cycle of 2004.  The changes in the non-TL rooms were used as a baseline to obtain an expected effect in the Total Literacy rooms (a null hypothesis number predicting how many students expected to improve if there were no TL effect).  This expected frequency gave the data to run the Χ² analysis.

Results:

Data was analyzed with three different scales (to satisfy three different interested parties).  With the most rigorous scale, the null hypothesis (no effect) yielded an expected 72 out of the 233 Total Literacy students advancing.  101 students improved, translating to a total school Χ² value of 12.15, significant at the P=.001 level.  This means that we can say with 99.9% confidence that Total Literacy is having a positive effect on the percent of children improving.  There are significant findings for sub-groupings as well.

While the percentage of students at or above grade level declined school-wide at the end of Cycle 3, students in the classrooms of Total Literacy trained teachers showed gains.  Both Wings and bilingual Total Literacy sub-groups showed sizeable gains compared to non-TL trained teachers’ students.

Cycle 4 Quantitative Data

Fourth cycle data was analyzed with the same methodology as the 3^rd cycle.  Again, the Χ²   value for advancing Total Literacy students was significant at the .001 level regardless of the scale applied.  In Roots, using any of the three scales, the percent of non-TL students advancing decreased from the prior year.  In constrast, the percent of TL students increased.  The same difference was found for Wings students, with a larger margin. 

DRA scores

DRA scores are gathered every January and May in Grades 1-3, to determine the percentage of students on grade level, and those “substantially deficient” by scoring at least a year behind grade level.  The goal of reading on grade level is primarily desirable for student benefit, it also indicates that a student does not require summer school.  This is a fiscal concern for a district where many readers are substantially deficient.

Analysis was done by homeroom to match the grade level scoring.  In second grade, the most intensely trained and practicing Total Literacy teachers, the percentage of students substantially deficient (more than a year below grade level) was 11.5%.  This is compared to both first and third grades, where the percentage was 33%.

Bilingual students in the Total Literacy classrooms made extraordinary gains.

After a year of data collection, there is trend established.  Students in classrooms of Total Literacy trained teachers achieve at a greater rate than students in classrooms of non-Total Literacy trained teachers.  The data shows that we have a 99.9% rate of confidence that Total Literacy is having a positive effect on the percentage of children improving.

Note:  CMT scores (Connecticut Mastery Tests) were from beginning of the year testing at grades 3 and up, and therefore provide a baseline for comparison the following year.

Qualitative Data

Teachers

The enthusiasm of Total Literacy trained teachers has been documented throughout the year, and their level of confidence rose as they met and planned, and observed model lessons with their classes.  Their self-assessments showed growth in all but 3 trained teachers, and those three chose not to be fully involved.  Based on a five-year change model, the expectations for the first full year of implementation are that teachers will mostly imitate, and require substantial guidance.

When this guidance and support was provided, the teachers began to demonstrate understanding of how to select materials and ask for suggestions of brain-based activities that linked the CMT Strands and their selected texts through developmentally appropriate strategies.

Some teachers worked with their SFA reading texts, and others with the KIDS Time literacy block.

The teachers continually provided suggestions for modifying or enhancing the implementation, contributing enormously to refinements.

Particular mention of the Unified Arts (music, art, and PE) teachers is necessary as they took positions of leadership during lesson planning and implementation.  The body of work that was developed is most impressive, and outside observers stated that their lessons “should be videotaped as models of excellence in arts-infused literacy for the whole country.”  They truly have provided a standard for this work.

Students

Interviews with students toward the end of each cycle indicated that the students enjoyed the learning, felt successful and challenged, and especially enjoyed learning to work together in collaborative teams.  In particular, the 2.1 class that participated in the rotation during the third cycle was a disparate group of learners with little capacity to listen, attend, work together, or respect classmates, teachers and materials.  By the end of the cycle they were working together to solve visual, aural, and kinesthetic problems, then applying those new-found skills to reading of a fairly complex book.

Classrooms teachers reported the following:

• Parents calling to find out why the students were so excited;
• Students attending and participating more than before;
• Much less stress, and a willingness of students to try learning activities without the threat of failure;
• Student understanding through visual, aural, and kinesthetic reinforcement of concepts;
• Students attending school on days of the arts rotation;
• Less discipline problems during Total Literacy activities.

Rigorous Lesson Content

Across Total Literacy classrooms, the level of attention and depth of learning has been enhanced by actively involving students in creating, performing, and responding in and through the arts.  They have been engaged in drawing and other art strategies, reading and playing of music, exploration of space through creative and structured dance, and dramatizing story components. 

The Total Literacy strategies were specifically designed to build student understandings and skills where they were lacking.  For example:  If students were not attending during listening comprehension then clipboards and blank journals were provided so students could draw as they listened.  (The research suggests that it is the act of drawing that creates the condition for critical listening, but does not specify what the content of the drawing must be.)  

Total Literacy strategies were also designed to focus on CMT Strands and the SFA focus of the particular group, whether it was a comprehension concept or a phonics/word construction element.  Layered into the strategy was a related arts concept or skill. 

Confounding Factors

As with any action research, there were many confounding variables that impacted outcomes as we progressed through the year.  Some unique to this pilot are listed here.

• There is a 40% yearly turnover rate in the school.
• There is as much variance across trained Total Literacy teachers as there is between trained and untrained teachers.  The results for Total Literacy teachers who used the strategies frequently and with integrity are extremely high.
• The principal was under scrutiny most of the year, and left in early May.
• The district reading facilitators were concerned about rigorous implementation of the reading curriculum, resulting in reevaluation of TL interactions with the reading program.
• New versions of the reading instruction package introduced an additional variable.
• There is a question about the validity and reliability of the SFA reading tests, particularly that the testing format is not the same as the format in which the testing is prepared through instruction.
• Testing by computer rather than pencil and paper began in the 4^th cycle, leading to lower scores for many Wings students.

Recommendations

The findings suggest that Total Literacy has enhanced the reading curriculum, and reading instruction.  It also enhances the classroom cultures in which trained teachers are planning and implementing instruction.  For the upcoming year, or for other implementations, the following suggest strategies that will aid in success and replicability.

• A new year or new implementation could begin with clear expectations, goals, assessment models, and training.  If a new site is chosen, it might be treated as a magnet situation that allows flexibility in the current programs in order to give Total Literacy the opportunity to demonstrate success when fully implemented.  In lieu of setting expectations and goals beforehand, the first month or cycle could be spent developing a clear plan and communication system for the year.
• Consider a clear and consistent communication system to minimize mixed or confusing messages, and maximizes the effective teaching practices promoted through Total Literacy. 
• Administrative support is critical to successful implementation.  A strategy should be developed for systematic and frequent dissemination of information about brain-based, research-based, differentiated strategies to reach building and district decision makers, so these strategies will become embraced and expanded, and available to every teacher and student.  Perhaps a newsletter or regular update could fill this need.
• Strong arts programs will help Total Literacy be most effective.  The strand that supports arts/essential teachers in developing strong curricula and teaching strategies, as well as understanding how their curricula teach and support literacy, is a comparatively small but critical component. 
• Goals and assessments IN the arts disciplines would be a powerful addition to this implementation.  Initially done through checklists, this is something that could emerge as standards-based arts curricula are developed district-wide.  This practice would reinforce the essential role of the arts in learning, as well as strengthen learning and literacy in the arts.
• Total Literacy should be positioned so it is not exclusively linked to any specific reading program, but rather becomes a delivery system for carefully chosen content, concepts, and skills.  There is a recommendation that at McDonough the emphasis for the coming year be on the Guided Reading (KIDS Time) program and the CMT Strands, rather than on SFA.  While continuing the “stuck reader” program that has already been developed, this approach would broaden possibilities for teachers and students.
• Assessment questions should broaden and deepen the discussion about what is important for children to know and be able to do in literacy and reading.  If Total Literacy can move stuck readers to the same level as their unstuck peers while providing the social/emotional scaffolding to build confident thinkers, this measures as success for the goal of moving “stuck” readers.  However, Total Literacy is raising the bar for stuck and unstuck readers, and has been shown to be particularly effective in Wings classes.  It offers hands-on, brains-on learning; higher order thinking, and literacy in multiple human languages, and therefore could be considered for all teachers and classrooms.
• In addition, as we plan future Total Literacy lessons, social/emotional goals might be added to the explicit goals.  In schools with bullying and other threatening student behaviors, ignoring this component leads to disrespectful interactions that inhibit learning.  Total Literacy helps build respectful learning communities where student achievement can increase.
• There will be a week of summer training in August of 2004 for district personnel new to Total Literacy.  In addition, some key trained personnel will attend a training of trainers in early August, to begin preparing for district sustainability and growth after grant funding diminishes.

2003-2004 has been a year of learning, challenge, and success.  We hope to build on this foundation to both deepen/broaden the initiative at McDonough School, and to expand to one or more schools.