Print Page
Blueprints logo

Head Start REDI

An enrichment program for low-income preschoolers and their families that supplements the Head Start preschool program.

Fact Sheet

Program Outcomes

  • Antisocial-aggressive Behavior
  • Close Relationships with Non-Parental Adults
  • Conduct Problems
  • Emotional Regulation
  • Positive Social/Prosocial Behavior
  • Preschool Communication/ Language Development
  • Prosocial with Peers
  • School Readiness

Program Type

  • Early Childhood Education
  • School - Individual Strategies
  • Skills Training
  • Social Emotional Learning
  • Teacher Training

Program Setting

  • School

Continuum of Intervention

  • Universal Prevention

Age

  • Early Childhood (3-4) - Preschool

Gender

  • Both

Race/Ethnicity

  • All

Endorsements

Blueprints: Promising

Program Information Contact

Karen Bierman, Ph.D.
Penn State University
Department of Psychology
251 Moore Building
University Park, PA 16801
kb2@psu.edu
sites.psu.edu/redi/

Program Developer/Owner

Karen L. Bierman
Pennsylvania State University


Brief Description of the Program

The Head Start REDI program is designed as an enrichment intervention that can be integrated into the existing framework of Head Start programs that are already using the High/Scope or Creative Curriculum. The intervention is delivered by classroom teachers and integrated into their ongoing classroom programs. It includes curriculum-based lessons, center-based extension activities, and training and weekly classroom coaching in "teaching strategies" to use throughout the day. It is focused primarily on social-emotional skill enrichment using the PATHS Preschool curriculum and language/emergent literacy skill enrichment. Parents also receive take-home materials describing the importance of positive support, emotion coaching, and interactive reading, with parenting tips and learning activities to use at home. In addition, REDI-P (Bierman et al., 2015) adds parent training intended to extend benefits to children for a longer period through parental support.

The Head Start REDI (Research-Based, Developmentally Informed) program is delivered by classroom teachers and integrated into their ongoing Head Start classroom programs that use the High/Scope or Creative Curriculum. It includes curriculum-based lessons, center-based extension activities, and training in "teaching strategies" to use throughout the day.

Social-emotional skill enrichment. The program promotes children's social-emotional skills using the 33-session PATHS Preschool curriculum. The curriculum targets four domains: (a) prosocial friendship skills, (b) emotional understanding and emotional expression skills, (c) self-control (e.g., the capacity to inhibit impulsive behavior and organize goal-directed activity), and (d) problem-solving skills, including interpersonal negotiation and conflict resolution skills. The curriculum is divided into 33 lessons that are delivered by teachers once per week during circle time. These lessons include modeling stories and discussions and use of puppet characters, photographs, and teacher role-play demonstration. Each lesson includes extension activities (e.g., cooperative projects and games) that provide children with opportunities to practice the target skills with teacher support. Teachers teach one lesson and conduct one extension activity each week. Generalized teaching strategies are encouraged with mentoring, including positive classroom management, use of specific teacher praise and support, emotion coaching, and induction strategies to promote appropriate self-control.

Language/emergent literacy skill enrichment. Four language and emergent literacy skills are targeted in REDI: (a) vocabulary, (b) syntax, (c) phonological awareness, and (d) print awareness. Three program components were developed to target these skills, including an interactive reading program, a set of "Sound Games," and print center activities. The curriculum includes two books per week, which are scripted with interactive questions. Each book has a list of targeted vocabulary words, presented with the aid of physical props and illustrations. In addition to presenting these materials in a systematic way during the week, teachers receive mentoring in the use of "language coaching" strategies, such as expansions and grammatical recasts, to provide a general scaffold for language development in the classroom. The overall goal is to improve teacher's strategic use of language in ways that would increase child oral language skills, including vocabulary, narrative, and syntax.

Teachers are provided with curricula materials to promote phonological awareness and print knowledge. A set of Sound Games is based primarily on the work of Lundberg and colleagues. The games are organized developmentally, moving from easier to more difficult skills during the course of the year (e.g., listening, rhyming, alliteration, words and sentences, syllables, phonemes). Teachers are asked to use a 10- to 15-minute Sound Game activity at least three times per week.

In addition, teachers are provided with a developmentally sequenced set of activities and materials to be used in their alphabet centers, including letter stickers, a letter bucket, materials to create a Letter Wall, and craft materials for various letter-learning activities. Each child is encouraged to visit the alphabet center several times per week and teachers are given materials to track the children's acquisition of letter names.

Training and professional development support. Teachers receive detailed manuals and kits containing all materials needed to implement the intervention. A 3-day professional training is conducted in August, prior to initiating the intervention, and a one-day booster training session is conducted in January. Teachers also receive weekly mentoring support provided by local education consultants ("REDI trainers"), experienced master teachers who are supervised by two project-based senior educational trainers. The weekly consultations are intended to enhance the quality of implementation through modeling, coaching, and providing ongoing feedback regarding program delivery. REDI trainers spend an average of three hours per week in each classroom observing, demonstrating, or team teaching lessons. They also meet with the head and assistant teacher for one hour each week outside of class.

Parent take-home materials. Three "take-home" packets are mailed to parents during the course of the year, each containing a modeling videotape, with parenting tips and learning activities to use at home. In addition, the PATHS curriculum includes handouts for parents, with suggestions for home activities. Children also take home letter stickers and compliment pages to prompt their parents to ask them about their school day and to provide positive support at home.

REDI-P (Bierman et al., 2015) adds additional home visits for Head Start parents, enriching these visits with evidence-based learning activities and support strategies. In addition to receiving take-home materials, parents receive 10 home visits during the spring of the Head Start pre-kindergarten year and 6 booster sessions after the start of kindergarten. Parents receive additional encouragement in these home visits to utilize materials and learn strategies that extend the benefits of the Head Start REDI program in the home.


A central objective of REDI is to maximize the integration of the social-emotional and language/emergent literacy intervention components that comprise the enrichment program. Each week, one of the books used in the interactive reading program focuses on the PATHS theme for that week (e.g., friendship, feelings, self-control, social problem solving), and feeling words are included in the vocabulary prompts. Conversely, PATHS extension activities incorporate language and emergent literacy skills.

Outcomes

Study 1

Bierman, Domitrovich et al. (2008), Bierman, Nix et al. (2008), Bierman et al. (2014), Nix et al. (2013; 2016)

  • Parents and teachers reported less aggression among those in the intervention group when compared to the control group at posttest and follow-up.
  • Students in the intervention group showed more improvements in emergent literacy skills than control students at posttest and, for a measure of phonemic decoding, at follow-up.
  • Children in the intervention group showed more improvements on child vocabulary and on parent reports of communication and language use at home at posttest.
  • At the 4-year follow-up, intervention subjects showed more positive trajectories for several measures of socio-emotional functioning such as social competence, aggressive-oppositional behavior, and peer rejection.
  • At the 6-year follow-up, intervention children, compared to control children, demonstrated significantly better social adjustment.
  • At the 8- and 10-year follow-ups, intervention children, compared to control children, demonstrated significantly fewer conduct problems.
  • At the 8- and 10-year follow-ups, intervention children, compared to control children, demonstrated significantly fewer emotional symptoms.

Effects on Risk and Protective Factors

  • Students in the intervention group showed more improvement than those in the control group on emotional understanding at posttest and social problem-solving skills at posttest and follow-up.
  • At the 4-year follow-up, intervention subjects showed more positive trajectories for several measures of socio-emotional functioning such as learning behavior, attention problems, and student-teacher closeness.
  • At the 6-year follow-up, intervention children showed significantly higher academic engagement.
  • At the 6-year follow-up, intervention children showed significantly higher parent involvement.

Study 2

Bierman et al. (2015) REDI-P home visit program

At posttest, there was a significant intervention effect for standardized tests of language and literacy skills in:

  • Emergent literacy
  • Academic performance

In addition, there was a significant intervention effect for teacher reports of social-emotional adjustment in:

  • Self-directed learning
  • Social competence

Risk and Protective Factors

At posttest, there was a significant intervention effect for parent reports of parent support for learning in:

  • Reading quality
  • Conversations

Study 3

Bierman et al. (2017) found that in second grade, children in the REDI-C school-only program, compared to children in the control group, had significantly

  • higher teacher-rated social competence
  • higher teacher-rated classroom participation
  • higher student-teacher relationship quality
  • fewer teacher-rated peer problems

Children in the school and parent program, compared to children in the REDI-C school-only program, showed significantly

  • higher student-reported social competence
  • fewer student-reported peer problems
  • better academic outcomes (sight word reading, reading skills, math skills).

Brief Evaluation Methodology

Study 1

The Head Start REDI study (Bierman, Domitrovich et al., 2008; Bierman, Nix et al. 2008; Bierman et al. 2014; Nix et al., 2013; Nix et al., 2016) was designed as a randomized trial that involved 25 centers with 44 classrooms. Centers were randomly assigned to either the treatment condition (REDI) or to the usual practice of Head Start. The classrooms were drawn from three counties in Pennsylvania and involved a mix of large and small towns. In total, 356 4-year-olds participated. They were assessed pre-intervention, post-intervention, 1 year after the intervention, and annually up to 10 years after the intervention. Children were observed during playtime and assessed by their parents and teachers.

Study 2

An additional study (Bierman et al., 2015) was conducted to evaluate the added benefit of home visits with REDI-P. This randomized trial involved 200 families randomly assigned to either the treatment of additional home visits or the control of the standard Head Start REDI program. The study recruited families from 24 Head Start centers in three urban and rural Pennsylvania counties during the 2008-2009 and 2009-2010 academic years. The study conducted assessments in the fall of prekindergarten and the end of kindergarten. Children completed standardized tests, teachers reported on student achievement and attitudes, parents and children were observed by researchers, and parents reported their own behavior and attitudes.

Study 3

Bierman et al. (2017) combined the two samples from Study 1 and Study 2 in a randomized controlled trial of 556 Head Start children assigned to three conditions: usual practice Head Start (control), REDI-C school-only intervention, and REDI-C plus the parent intervention. Assessments of social relations and academic outcomes in second grade occurred more than two years after the program end.

Blueprints Certified Studies

Study 1

Bierman, K. L., Nix, R. L., Heinrichs, B. S., Domitrovich, C. E., Gest, S. D., Welsh, J. A., & Gill, S. (2014). Effects of Head Start REDI on children's outcomes 1 year later in different kindergarten contexts. Child Development, 85(1), 140-159.


Bierman, K., Domitrovich, C., Nix, R., Gest, S., Welsh, J., Greenberg, M., . . . Gill, S. (2008). Promoting academic and social-emotional school readiness: The Head Start REDI program. Child Development, 79(6), 1802-1817.


Welsh, J. A., Bierman, K. L., Nix, R. L., & Heinrichs, B. N. (2020). Sustained effects of a school readiness intervention: 5th grade outcomes of the Head Start REDI program. Early Childhood Research Quarterly, 53, 151-160. doi:10.1016/j.ecresq.2020.03.009


Bierman, K. L., Heinrichs, B. S., Welsh, J. A., & Nix, R. L. (2020). Reducing adolescent psychopathology in socioeconomically disadvantaged children with a preschool intervention: A randomized controlled trial. American Journal of Psychiatry. Advance online publication. doi:10.1176/appi.ajp.2020.20030343


Risk and Protective Factors

Risk Factors

Individual: Antisocial/aggressive behavior*

Family: Low socioeconomic status

Protective Factors

Individual: Clear standards for behavior, Problem solving skills*, Prosocial behavior, Skills for social interaction*

Family: Opportunities for prosocial involvement with parents, Parental involvement in education*, Parent social support, Rewards for prosocial involvement with parents

School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school


* Risk/Protective Factor was significantly impacted by the program

See also: Head Start REDI Logic Model (PDF)

Race/Ethnicity/Gender Details

Race/Ethnicity/Gender Details
Nix et al. (2013) reported that the intervention effects on posttest outcomes generally did not differ significantly across European-American, African-American, or Latino-American children, although there were a few differences in indirect effects.

Training and Technical Assistance

REDI program training is usually done on site, for a Head Start program or set of preschools/child-care centers serving a common community. The initial training workshop consists of three days. Ideally, the first two days are scheduled together before teachers begin the program, with the third day scheduled about 2 months later. The first two days provide teachers/trainees with theory, research background, lessons modelled by the trainer, practice to prepare teachers to use PATHS Curriculum lessons with fidelity, along with the interactive reading, Sound Games, and print center activities. Implementation planning also occurs during this initial training. Teachers then begin to teach the REDI curriculum. Over the first two months of implementation, teachers become more familiar and comfortable with the general flow of the REDI lessons. During the third day of training, about 2 months after they have started training, they are ready to discuss teaching strategies that will optimize program impact. The third day of training is more interactive, and moves beyond the pragmatics of REDI implementation to talk about high-quality learning interactions, and how to support children's self-regulated learning. Trainer and teachers discuss advanced curriculum issues, trade ideas and engage in problem solving; teachers also model interactive lessons. Another option is to schedule training for three consecutive days.

Training is also available for local REDI coaches. These can be program supervisors or master teachers, who have time allotted to provide mentoring/coaching to teachers as they implement the REDI program. There is a one-day training for coaches that is scheduled before the first teacher training. Coaches typically observe classrooms and meet with teachers to support teachers and discuss program implementation and teaching strategies. The teaching strategies emphasize positive classroom management and enriched language use. Coaching typically requires about one hour per week per classroom during the first year of REDI implementation. Regular conference calls with a national trainer provide coaches with the support they need to oversee program implementation and support teachers.

For optimal implementation, sites should consider booster training/technical assistance activities in subsequent years of program implementation. Ongoing consultation and booster visits are available and are often desired by comprehensive, long-term implementations. The trainer can provide a booster visit each year (one day in length) to meet with the staff and provide continued professional development. One day of fidelity visits is another option, in which the trainer visits schools, observes lessons, provides constructive feedback on how to help the program move forward in year two and beyond. The trainer can also provide ongoing consultation by means of regularly scheduled phone calls/conference calls and on-call email consultation with the local REDI coach or coordinator.

Training Certification Process

The first step to becoming a REDI Trainer is to attain Preschool PATHS Trainer certification. The PATHS Training Program is designed to develop highly experienced, high quality trainers who are fully competent to provide training in the PATHS Curriculum to their local educational entity. Trainers can include staff (teachers, support staff, staff developers) from local school districts/boards, Local Education Agencies (LEAs) and non-profit agencies focused on the promotion of children's early learning, mental health and youth development. PATHS Education Worldwide LLC trains these qualified "educators" to conduct school-based or regional workshops for the preparation of teachers and school support staff who plan to implement PATHS Curricula within these educational entities. Once certified, PATHS Trainers conduct workshops and provide follow-up technical assistance and coaching services for their district or regional personnel in accordance with the PATHS workshop training materials, agenda and guidelines. REDI training builds upon Preschool PATHS training to add competency in training the interactive reading, sound games, and print center activities that are part of the REDI program, along with Preschool PATHS.

To be considered as an Affiliate Trainer for REDI requires meeting the following prerequisites:

  • Attendance at a Preschool PATHS Workshop/REDI Workshop
  • High Quality Performance for at least two years as a Preschool PATHS/REDI teacher or PATHS/REDI Coach
  • Master's degree (or comparable credentials)
  • Classroom experience with students in a learner role (teaching, administration, and school counseling preferred)
  • Training experience with educators

After meeting the pre-requisites above, the requirements to be certified as a trainer include participation in the following four-step training/certification process. The AT candidate(s) receive four days of coaching from a PATHS Master Trainer in addition to participation in an Observation Workshop and two Shared Workshops. The first day of coaching follows the Observation Workshop. The second day precedes the Shared Workshop. The third day follows the Shared Workshop in preparation for the second Shared Workshop. The fourth day follows the second Shared Workshop in preparation for certification as a Preschool PATHS/REDI trainer. The primary purpose of the coaching days are to provide detailed and personalized instruction in how to conduct the PATHS/REDI workshop and to observe and provide feedback on candidates' training skills. This formalized process includes benchmarks of performance for all perspective affiliate trainers. Candidates who successfully complete the program are certified as Affiliate Trainers.

Benefits and Costs

Source: Washington State Institute for Public Policy
All benefit-cost ratios are the most recent estimates published by The Washington State Institute for Public Policy for Blueprint programs implemented in Washington State. These ratios are based on a) meta-analysis estimates of effect size and b) monetized benefits and calculated costs for programs as delivered in the State of Washington. Caution is recommended in applying these estimates of the benefit-cost ratio to any other state or local area. They are provided as an illustration of the benefit-cost ratio found in one specific state. When feasible, local costs and monetized benefits should be used to calculate expected local benefit-cost ratios. The formula for this calculation can be found on the WSIPP website.

Program Costs

Start-Up Costs

Initial Training and Technical Assistance

Initial teacher training is available through PATHS Worldwide, LLC: www.pathseducation.com/

$6000 for 3-days of initial training ($2000 per day) plus trainer travel costs, for up to 40 teachers. This is usually divided into an initial 2-day training and a mid-year 1-day booster. If local coaches are trained, there is an added 1 day of training for coaches before the teacher training ($2000).

Curriculum and Materials

Costs per classroom are as follows:

$839 - Preschool PATHS Curriculum
$ 45 - Language and Literacy Curriculum (printing cost)
$ 50 - Lakeshore Learning Alphabet Sounds Photo Library
$425 - Books for Interactive Reading Program

To use REDI, the Preschool PATHS manual must be purchased from Channing-Bete http://shop.channing-bete.com/onlinestore/storeitem.html?iid=179169. The additional language and literacy intervention components can be downloaded from the REDI web-site at no cost and printed. Information available at http://sites.psu.edu/redi/. Like the PATHS lessons, these lessons provide detailed information and illustrative scripts for teachers to help them teach lessons with fidelity. These additional curriculum components are formatted in a manner parallel to Preschool PATHS, with weekly manual inserts containing the REDI language and literacy intervention components that can be inserted into (or used with) the PATHS manual.

Licensing

None

Other Start-Up Costs

A local coach is recommended for at least the first year. Coaches are usually teachers or program supervisors who have received special training. Cost for coach training and support are indicated in the training section.

The program is implemented in the classroom, so no additional space is required. In most cases, initial planning can be done via telephone and/or email consultation. No cost, unless extensive consultation is required.

Intervention Implementation Costs

Ongoing Curriculum and Materials

All curriculum materials are re-usable in subsequent years. $100 per year per classroom is needed for photocopying PATHS Curriculum activity sheets, and purchasing program material replacements (e.g., feeling faces, turtle stickers). Templates are provided for the site to make their own feeling faces for cost savings.

Staffing

Program is delivered by classroom teachers. REDI requires about 3.5 hours per week (30-60 minutes per day.) PATHS lessons are taught once per week (20 minutes), interactive reading occurs 5 times per week (20 minutes each lesson), sound games are taught 3 times per week (10-15 minutes each lesson), and print centers are open at least 3 times per week.

To sustain REDI over time, we recommend the training of local coaches (program supervisors, master teachers) to support teachers in program administration, and sustain the program when teachers turn over. This coaching can be integrated with other responsibilities, but requires additional time in training and for program review with teachers. A full-time coach can typically support 10-12 classrooms.

Other Implementation Costs

In the first year, a local coach should expect to spend about 1 hour per week per classroom supporting teacher's program implementation.

Implementation Support and Fidelity Monitoring Costs

Ongoing Training and Technical Assistance

If desired, booster trainings are available at $2000 per day plus trainer travel costs. Technical assistance by email and phone is available from PATHS Education Worldwide, LLC. When coaches are used, funds should be budgeted for annual site visits by national trainers at a cost of $4000 for two consecutive days plus travel.

Fidelity Monitoring and Evaluation

The local coach takes the lead in fidelity monitoring. If a site does not have a local coach, a local coordinator should be designated who is responsible for fidelity monitoring.

There is a set of implementation measures that are designed to guide a teacher in the implementation process and support self-monitoring, as well as measures designed for a coach or trained administrator to use to monitor REDI implementation fidelity and quality.

Ongoing License Fees

None

Other Implementation Support and Fidelity Monitoring Costs

No information is available

Other Cost Considerations

The size of implementation is key to lowering costs. Training many teachers at one time is most cost effective. There are also bulk discounts on the Preschool PATHS Curriculum.

Year One Cost Example

This example assumes a Head Start center implements the REDI Classroom Program in three classrooms, each with 17 students. A local coach, in a 25%-time position, is trained to monitor fidelity to the program and promote future sustainability. The coach is supported with a site visit by a national trainer, timed to coincide with the booster training.

Teacher Training: 2-day initial, 1-day booster $6,000.00
Coach Training: 1-day $2,000.00
Trainer Travel: initial and booster $2,000.00
Coach Salary: .25 FTE $12,500.00
Preschool PATHS Curriculum: $839 x 3 $2,517.00
Language and Literacy Curriculum printing: $45 x 3 $135.00
Lakeshore Learning Alphabet Sounds Photo Library printing: $50 x 3 $150.00
Books for Interactive Reading Program: $425 X 3 $1,275.00
Site Visit by National Trainer: 2-day $4,000.00
Total One Year Cost $30,577.00

With 3 classrooms of 17 students each, the REDI program would serve 51 students in the first year. The Year One cost per student would be $599.55.

Funding Strategies

Funding Overview

The major costs for establishing REDI are in the first year, when the curriculum is purchased and initial training takes place. Thereafter, the major resource needed to maintain the program is a commitment from teachers and administrators to implement it consistently and with fidelity. When local administrators are trained as REDI coaches, it increases the likelihood that a program will sustain REDI with high fidelity over time. Hence, funding needs are primarily for the initial year of program implementation.

Funding Strategies

Improving the Use of Existing Public Funds

Opportunities may arise through federal or state funding to support the initiation of the REDI program in Head Start, private preschools or child-care centers, or public school pre-kindergartens. Recent new initiatives have devoted public funds to the expansion and enrichment of early learning and prekindergarten programs, emphasizing the implementation of evidence-based programs. REDI is evidence-based and has documented long-term effects through third grade, and hence is competitive for this kind of funding. Likely sources include the Department of Education and SAMSHA (Department of Health and Human Services).

Medicaid Funding is not applicable to REDI.

Allocating State or Local General Funds

State and local funds may be allocated to purchase the initial training and curriculum. State departments of education or health may also allocate state funds toward evidence-based early learning programs and administer them to school districts or local administrative units competitively or through formula.

Foundation Grants and Public-Private Partnerships

Because program costs are primarily a "one time" expense, foundations and public-private partnerships are potential funders. They may fund the initial curriculum purchase and training if programs commit to on-going implementation support.

Evaluation Abstract

Program Developer/Owner

Karen L. BiermanPennsylvania State UniversityDept. of Psychology251 Moore BuildingUniversity Park, PA 16802kb2@psu.edu sites.psu.edu/redi/

Program Outcomes

  • Antisocial-aggressive Behavior
  • Close Relationships with Non-Parental Adults
  • Conduct Problems
  • Emotional Regulation
  • Positive Social/Prosocial Behavior
  • Preschool Communication/ Language Development
  • Prosocial with Peers
  • School Readiness

Program Specifics

Program Type

  • Early Childhood Education
  • School - Individual Strategies
  • Skills Training
  • Social Emotional Learning
  • Teacher Training

Program Setting

  • School

Continuum of Intervention

  • Universal Prevention

Program Goals

An enrichment program for low-income preschoolers and their families that supplements the Head Start preschool program.

Population Demographics

This program targets 3-4-year-olds in Head Start programs for at-risk children.

Target Population

Age

  • Early Childhood (3-4) - Preschool

Gender

  • Both

Race/Ethnicity

  • All

Race/Ethnicity/Gender Details

Nix et al. (2013) reported that the intervention effects on posttest outcomes generally did not differ significantly across European-American, African-American, or Latino-American children, although there were a few differences in indirect effects.

Other Risk and Protective Factors

The program focuses on improving socio-emotional and literacy skills, academic engagement.

Risk/Protective Factor Domain

  • Individual
  • School
  • Peer
  • Family

Risk/Protective Factors

Risk Factors

Individual: Antisocial/aggressive behavior*

Family: Low socioeconomic status

Protective Factors

Individual: Clear standards for behavior, Problem solving skills*, Prosocial behavior, Skills for social interaction*

Family: Opportunities for prosocial involvement with parents, Parental involvement in education*, Parent social support, Rewards for prosocial involvement with parents

School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school


*Risk/Protective Factor was significantly impacted by the program

Brief Description of the Program

The Head Start REDI program is designed as an enrichment intervention that can be integrated into the existing framework of Head Start programs that are already using the High/Scope or Creative Curriculum. The intervention is delivered by classroom teachers and integrated into their ongoing classroom programs. It includes curriculum-based lessons, center-based extension activities, and training and weekly classroom coaching in "teaching strategies" to use throughout the day. It is focused primarily on social-emotional skill enrichment using the PATHS Preschool curriculum and language/emergent literacy skill enrichment. Parents also receive take-home materials describing the importance of positive support, emotion coaching, and interactive reading, with parenting tips and learning activities to use at home. In addition, REDI-P (Bierman et al., 2015) adds parent training intended to extend benefits to children for a longer period through parental support.

Description of the Program

The Head Start REDI (Research-Based, Developmentally Informed) program is delivered by classroom teachers and integrated into their ongoing Head Start classroom programs that use the High/Scope or Creative Curriculum. It includes curriculum-based lessons, center-based extension activities, and training in "teaching strategies" to use throughout the day.

Social-emotional skill enrichment. The program promotes children's social-emotional skills using the 33-session PATHS Preschool curriculum. The curriculum targets four domains: (a) prosocial friendship skills, (b) emotional understanding and emotional expression skills, (c) self-control (e.g., the capacity to inhibit impulsive behavior and organize goal-directed activity), and (d) problem-solving skills, including interpersonal negotiation and conflict resolution skills. The curriculum is divided into 33 lessons that are delivered by teachers once per week during circle time. These lessons include modeling stories and discussions and use of puppet characters, photographs, and teacher role-play demonstration. Each lesson includes extension activities (e.g., cooperative projects and games) that provide children with opportunities to practice the target skills with teacher support. Teachers teach one lesson and conduct one extension activity each week. Generalized teaching strategies are encouraged with mentoring, including positive classroom management, use of specific teacher praise and support, emotion coaching, and induction strategies to promote appropriate self-control.

Language/emergent literacy skill enrichment. Four language and emergent literacy skills are targeted in REDI: (a) vocabulary, (b) syntax, (c) phonological awareness, and (d) print awareness. Three program components were developed to target these skills, including an interactive reading program, a set of "Sound Games," and print center activities. The curriculum includes two books per week, which are scripted with interactive questions. Each book has a list of targeted vocabulary words, presented with the aid of physical props and illustrations. In addition to presenting these materials in a systematic way during the week, teachers receive mentoring in the use of "language coaching" strategies, such as expansions and grammatical recasts, to provide a general scaffold for language development in the classroom. The overall goal is to improve teacher's strategic use of language in ways that would increase child oral language skills, including vocabulary, narrative, and syntax.

Teachers are provided with curricula materials to promote phonological awareness and print knowledge. A set of Sound Games is based primarily on the work of Lundberg and colleagues. The games are organized developmentally, moving from easier to more difficult skills during the course of the year (e.g., listening, rhyming, alliteration, words and sentences, syllables, phonemes). Teachers are asked to use a 10- to 15-minute Sound Game activity at least three times per week.

In addition, teachers are provided with a developmentally sequenced set of activities and materials to be used in their alphabet centers, including letter stickers, a letter bucket, materials to create a Letter Wall, and craft materials for various letter-learning activities. Each child is encouraged to visit the alphabet center several times per week and teachers are given materials to track the children's acquisition of letter names.

Training and professional development support. Teachers receive detailed manuals and kits containing all materials needed to implement the intervention. A 3-day professional training is conducted in August, prior to initiating the intervention, and a one-day booster training session is conducted in January. Teachers also receive weekly mentoring support provided by local education consultants ("REDI trainers"), experienced master teachers who are supervised by two project-based senior educational trainers. The weekly consultations are intended to enhance the quality of implementation through modeling, coaching, and providing ongoing feedback regarding program delivery. REDI trainers spend an average of three hours per week in each classroom observing, demonstrating, or team teaching lessons. They also meet with the head and assistant teacher for one hour each week outside of class.

Parent take-home materials. Three "take-home" packets are mailed to parents during the course of the year, each containing a modeling videotape, with parenting tips and learning activities to use at home. In addition, the PATHS curriculum includes handouts for parents, with suggestions for home activities. Children also take home letter stickers and compliment pages to prompt their parents to ask them about their school day and to provide positive support at home.

REDI-P (Bierman et al., 2015) adds additional home visits for Head Start parents, enriching these visits with evidence-based learning activities and support strategies. In addition to receiving take-home materials, parents receive 10 home visits during the spring of the Head Start pre-kindergarten year and 6 booster sessions after the start of kindergarten. Parents receive additional encouragement in these home visits to utilize materials and learn strategies that extend the benefits of the Head Start REDI program in the home.


A central objective of REDI is to maximize the integration of the social-emotional and language/emergent literacy intervention components that comprise the enrichment program. Each week, one of the books used in the interactive reading program focuses on the PATHS theme for that week (e.g., friendship, feelings, self-control, social problem solving), and feeling words are included in the vocabulary prompts. Conversely, PATHS extension activities incorporate language and emergent literacy skills.

Theoretical Rationale

The goal of REDI is to maximize the impact of Head Start on child school readiness outcomes by considering a comprehensive approach that addresses both the cognitive and social-emotional skill development of children affected by poverty and disadvantage. By integrating support for improved instruction and teaching across these domains, children benefit in a broader array of outcome areas.

Theoretical Orientation

  • Skill Oriented
  • Biological - Neurobiological
  • Person - Environment

Brief Evaluation Methodology

Study 1

The Head Start REDI study (Bierman, Domitrovich et al., 2008; Bierman, Nix et al. 2008; Bierman et al. 2014; Nix et al., 2013; Nix et al., 2016) was designed as a randomized trial that involved 25 centers with 44 classrooms. Centers were randomly assigned to either the treatment condition (REDI) or to the usual practice of Head Start. The classrooms were drawn from three counties in Pennsylvania and involved a mix of large and small towns. In total, 356 4-year-olds participated. They were assessed pre-intervention, post-intervention, 1 year after the intervention, and annually up to 10 years after the intervention. Children were observed during playtime and assessed by their parents and teachers.

Study 2

An additional study (Bierman et al., 2015) was conducted to evaluate the added benefit of home visits with REDI-P. This randomized trial involved 200 families randomly assigned to either the treatment of additional home visits or the control of the standard Head Start REDI program. The study recruited families from 24 Head Start centers in three urban and rural Pennsylvania counties during the 2008-2009 and 2009-2010 academic years. The study conducted assessments in the fall of prekindergarten and the end of kindergarten. Children completed standardized tests, teachers reported on student achievement and attitudes, parents and children were observed by researchers, and parents reported their own behavior and attitudes.

Study 3

Bierman et al. (2017) combined the two samples from Study 1 and Study 2 in a randomized controlled trial of 556 Head Start children assigned to three conditions: usual practice Head Start (control), REDI-C school-only intervention, and REDI-C plus the parent intervention. Assessments of social relations and academic outcomes in second grade occurred more than two years after the program end.

Outcomes (Brief, over all studies)

Study 1

Bierman, Domitrovich et al. (2008), Bierman, Nix et al. (2008), Bierman et al. (2014), Nix et al. (2013; 2016), Welsh et al. (2020), Bierman et al. (2020): A study that integrated REDI and language emergent literacy skills into a Head Start program already using the High/Scope or Creative curriculum demonstrated the following outcomes. At posttest, statistically significant differences in aggression were reported by teachers, and marginal differences were reported by parents, with those in the intervention group showing less aggression. No differences in aggression were reported by observers. Additionally, intervention students were better on various skills targeted by the intervention, including language and emerging literacy skills and social-emotional skills and task orientation (i.e., learning engagement in school) rated by observers.

At the 1-year follow-up, the intervention showed significant main effects on five outcomes: phonemic decoding, teacher-rated learning behaviors, competent problem solving, teacher-rated aggression, and parent-rated aggression. At the 4-year follow-up, intervention subjects showed more positive trajectories for several measures of socio-emotional functioning such as social competence, aggressive-oppositional behavior, and peer rejection.

At the 6-year follow-up (Welsh et al., 2020), intervention children, compared to control children, demonstrated significantly higher social adjustment, academic engagement, and parent involvement.

At the 8- and 10- year follow-ups (Bierman et al., 2020), intervention children, compared to control children, demonstrated significantly fewer conduct problems and emotional symptoms.

Study 2

Bierman et al. (2015): At posttest, children in the home visit treatment group had significantly improved language and literacy test scores, teachers reported significant improvement in measures of social-emotional learning, and parents improved on several measures of support for learning, as compared to the control group.

Study 3

Bierman et al. (2017): In second grade, children in the REDI-C school-only program showed significantly better teacher-rated social competence, classroom participation, student-teacher relations, and peer problems than the control group. Children in the school and parent program showed significantly better student-reported social competence and peer problems and significantly better scores on academic outcomes (sight word reading, reading skills, math skills) than children in the school-only program.

Outcomes

Study 1

Bierman, Domitrovich et al. (2008), Bierman, Nix et al. (2008), Bierman et al. (2014), Nix et al. (2013; 2016)

  • Parents and teachers reported less aggression among those in the intervention group when compared to the control group at posttest and follow-up.
  • Students in the intervention group showed more improvements in emergent literacy skills than control students at posttest and, for a measure of phonemic decoding, at follow-up.
  • Children in the intervention group showed more improvements on child vocabulary and on parent reports of communication and language use at home at posttest.
  • At the 4-year follow-up, intervention subjects showed more positive trajectories for several measures of socio-emotional functioning such as social competence, aggressive-oppositional behavior, and peer rejection.
  • At the 6-year follow-up, intervention children, compared to control children, demonstrated significantly better social adjustment.
  • At the 8- and 10-year follow-ups, intervention children, compared to control children, demonstrated significantly fewer conduct problems.
  • At the 8- and 10-year follow-ups, intervention children, compared to control children, demonstrated significantly fewer emotional symptoms.

Effects on Risk and Protective Factors

  • Students in the intervention group showed more improvement than those in the control group on emotional understanding at posttest and social problem-solving skills at posttest and follow-up.
  • At the 4-year follow-up, intervention subjects showed more positive trajectories for several measures of socio-emotional functioning such as learning behavior, attention problems, and student-teacher closeness.
  • At the 6-year follow-up, intervention children showed significantly higher academic engagement.
  • At the 6-year follow-up, intervention children showed significantly higher parent involvement.

Study 2

Bierman et al. (2015) REDI-P home visit program

At posttest, there was a significant intervention effect for standardized tests of language and literacy skills in:

  • Emergent literacy
  • Academic performance

In addition, there was a significant intervention effect for teacher reports of social-emotional adjustment in:

  • Self-directed learning
  • Social competence

Risk and Protective Factors

At posttest, there was a significant intervention effect for parent reports of parent support for learning in:

  • Reading quality
  • Conversations

Study 3

Bierman et al. (2017) found that in second grade, children in the REDI-C school-only program, compared to children in the control group, had significantly

  • higher teacher-rated social competence
  • higher teacher-rated classroom participation
  • higher student-teacher relationship quality
  • fewer teacher-rated peer problems

Children in the school and parent program, compared to children in the REDI-C school-only program, showed significantly

  • higher student-reported social competence
  • fewer student-reported peer problems
  • better academic outcomes (sight word reading, reading skills, math skills).

Mediating Effects

One mediation analysis (Nix et al., 2013) treated changes in five outcome measures from baseline to posttest as mediators: 1) vocabulary, 2) emergent literacy skills, 3) emotional understanding, 4) competent social problem solving, and 5) positive social behavior. The intervention significantly improved all five of these mediating outcomes, with effect sizes ranging from .25 to .49.

The analysis examined three kindergarten outcomes: 1) reading achievement, 2) learning engagement, and 3) positive social behavior. The five mediators and three outcomes defined 15 possible indirect effects.

The results demonstrated that gains made during preschool predicted kindergarten functioning. For reading achievement, the intervention had significant indirect effects via four of the five mediators. For learning engagement, the intervention had significant indirect effects via three of five posttest measures. For positive social behavior, the intervention had significant indirect effects via one of five posttest measures (positive social behavior). Gains made during Head Start thus continued to predict functioning one year later, in kindergarten.

Effect Size

At posttest (Bierman, Domitrovich et al., 2008), effect sizes ranged from .15 to .39 for child skill outcomes. At follow-up (Bierman et al., 2014), effect sizes ranged from .22 (small) to .40 (small medium). In most cases, these effect sizes were equivalent to or larger than those at posttest. In comparing trajectories through the 4-year follow-up, Nix et al. (2016) reported odds ratios in the small-medium range (1.6 to 1.9).

Bierman et al. (2015) reported small to small-medium standardized, mean-centered coefficients (comparable to Cohen's d of .25-.29).

Welsh et al. (2020) reported small to medium effect sizes (d = .24 - .34).

Bierman et al. (2020) reported small effect sizes (d = .20 - .25).

Effect sizes in Study 3 (Bierman et al., 2017) ranged widely from small (d = .24) to large (d = .78).

Generalizability

The program has been examined in children from Head Start centers in Pennsylvania.

Potential Limitations

Study 1

Bierman,Domitrovich et al. (2008), Bierman, Nix et al. (2008), Bierman et al. (2014), Nix et al. (2013, 2016)

  • The analysis appears to have dropped some families who withdrew from the program.
  • Models measured intervention at the classroom level when it appears that centers, typically with multiple classrooms, were randomized; however, subsequent analysis requested by Blueprints showed minimal impact from adding center as a clustering factor.
  • Tests for baseline differences did not include sociodemographic characteristics.
  • Six of 13 outcome measures were missing pretest scores.
  • Measures of executive functions, although important for development of skills and behaviors, had limited relationships with the intervention in analyses of outcomes, moderation, or mediation.

Welsh et al. (2020)

  • One outcome measure missing a baseline control
  • One significant differential attrition effect

Bierman et al. (2020)

  • Some evidence of differential attrition

Study 2

Bierman et al. (2015)

  • Some measures came from reports of parents, who helped to deliver the intervention
  • One outcome measure not equivalent at baseline

Study 3

Bierman et al. (2017)

  • Incomplete tests and some evidence of differential attrition

Endorsements

Blueprints: Promising

Program Information Contact

Karen Bierman, Ph.D.
Penn State University
Department of Psychology
251 Moore Building
University Park, PA 16801
kb2@psu.edu
sites.psu.edu/redi/

References

Study 1

Bierman, K. L., Nix, R. L., Greenberg, M. T., Blair, C., & Domitrovich, C. E. (2008). Executive functions and school readiness intervention: Impact, moderation, and mediation in the Head Start REDI program. Development and Psychology, 20(3), 821-843.

Certified Bierman, K. L., Nix, R. L., Heinrichs, B. S., Domitrovich, C. E., Gest, S. D., Welsh, J. A., & Gill, S. (2014). Effects of Head Start REDI on children's outcomes 1 year later in different kindergarten contexts. Child Development, 85(1), 140-159.

Certified Bierman, K., Domitrovich, C., Nix, R., Gest, S., Welsh, J., Greenberg, M., . . . Gill, S. (2008). Promoting academic and social-emotional school readiness: The Head Start REDI program. Child Development, 79(6), 1802-1817.

Nix, R. L., Bierman, K. L., Domitrovich, C. E., & Gill, S. (2013). Promoting children's social-emotional skills in preschool can enhance academic and behavioral functioning in kindergarten: Findings from Head Start REDI. Early Education and Development, 24(7), 1000-1019.

Nix, R. L., Bierman, K. L., Heinrichs, B. S., Gest, S. D., Welsh, J. A., & Domitrovich, C. E. (2016). The randomized-controlled trial of Head Start REDI: Sustained effects on developmental trajectories of social-emotional functioning. Journal of Consulting and Clinical Psychology, 84(4), 310-322.

Certified

Welsh, J. A., Bierman, K. L., Nix, R. L., & Heinrichs, B. N. (2020). Sustained effects of a school readiness intervention: 5th grade outcomes of the Head Start REDI program. Early Childhood Research Quarterly, 53, 151-160. doi:10.1016/j.ecresq.2020.03.009

Certified

Bierman, K. L., Heinrichs, B. S., Welsh, J. A., & Nix, R. L. (2020). Reducing adolescent psychopathology in socioeconomically disadvantaged children with a preschool intervention: A randomized controlled trial. American Journal of Psychiatry. Advance online publication. doi:10.1176/appi.ajp.2020.20030343

Study 2

Bierman, K. L., Welsh, J. A., Heinrichs, B. S., Nix, R. L., & Mathis, E. T. (2015). Helping Head Start parents promote their children's kindergarten adjustment: The research-based developmentally informed parent program. Child Development, 86,(6), 1877-1891.

Study 3

Bierman, K. L., Heinrichs, B. S., Welsh, J. A., Nix, R. L., & Gest, S. D. (2017). Enriching preschool classrooms and home visits with evidence-based programming: Sustained benefits for low-income children. Journal of Child Psychology and Psychiatry, 58(2), 129-137. doi:10.1111/jcpp.12618

Study 1

Intervention: Teachers delivered the 33 lesson Preschool PATHS curriculum once per week to promote children's social-emotional skills. REDI targeted vocabulary, syntax, phonological awareness, and print awareness with three program components. Teachers received weekly mentoring support provided by local education consultants (REDI trainers), experienced master teachers who were supervised by two project-based senior educational trainers. The program was integrated into Head Start centers that were implementing the High/Scope or Creative Curriculum.

Evaluation Methodology

Design: The study consisted of a randomized sample taken of 44 Head Start classrooms in 25 (Nix et al., 2013, 2016) centers across three counties in Pennsylvania. Half of the participating classrooms came from a large, fairly densely populated county in the southeastern part of the state, which included an urban community surrounded by smaller communities. The other classrooms came from two smaller counties in the central part of the state, characterized by small towns and rural areas. Classrooms were stratified on location, length of program, and student demographics to assure even representation in the intervention and comparison conditions. In the more urban county, classrooms were stratified into three groups, in terms of urban (vs. nonurban) location, percentage of minority students served, and use of Spanish in the classroom. In the two more rural counties, classes were also stratified into three groups, which varied in terms of being full-day or half-day programs and located in small towns versus rural locations.

Classrooms in the same center were always assigned to the same condition, to avoid inadvertent contamination of condition within centers. Within stratified groups, centers were randomly assigned to intervention or comparison conditions. This process resulted in 14 classrooms (67%) in each condition that were located in small centers (1-2 classrooms) and 8 classrooms in each condition located in four larger centers (containing 3-5 classrooms), with children from ethnic minority groups fairly evenly spread across condition (39% of the intervention group and 45% of the control group.) A total of 356 children took part in the study.

Children were recruited across two years. Each year, teachers were studied as they implemented the intervention for the first time, and 4-year-old children were assessed after receiving one year of REDI intervention and then one year later at the end of kindergarten. Teachers in the comparison classrooms continued to conduct the curriculum "as usual." These control classrooms contained about 14 children, typically one-third three-year olds and two-thirds four-year olds. To recruit participants for the evaluation trial, brochures describing the study were distributed to parents of all 4-year-old children in participating classrooms. Overall, only 14 eligible families declined to participate in the evaluation process, but an additional 21 primary caregivers were very difficult to reach and failed to complete the preassessment. Two children were dropped because they had a twin or sibling in the study, and 19 families withdrew early from Head Start and completed only parts of the assessment procedures. Hence, study participants represented 86% of the initially eligible population.

Assessments occurred early in the school year (pretest) and at the end of the school year (posttest). At the 1-year follow-up, the subjects transitioned to 202 kindergarten classes at 82 schools in 33 school districts. The follow-up assessment occurred in March-April, near the end of kindergarten. The 4-year follow-up assessed the children each year through the end of third grade. The 6-year follow-up assessed the children during 5th grade. The 8- and 10-year follow-ups assessed the children during 7th and 9th grade, respectively.

Among the 356 children, attrition was only 3% at posttest and only 5% at follow-up. At the 4-year follow-up, attrition reached only 9%. At the 6- and 10-year follow-ups, attrition was 19%.

Sample: Participants in this study included two cohorts of 4-year old children. They were 17% Latino, 25% Black, and 58% white. Also, 54% were girls and 46% were boys. Primary caregivers included 89% mothers, 4% fathers, 4% grandparents, and 3% other, such as stepparents or foster parents. In terms of education of the primary caregivers, 31% had less than a high school education, 60% graduated from high school, 8% completed a technical degree, and 2% completed a college degree. The authors note that the sample is typical of the socioeconomically disadvantaged families and children for whom Head Start is intended.

Measures: A multi-method, multi-measure assessment battery included child assessments, teacher ratings, parent ratings, and direct observations. Outcome measures at posttest represented six core domains: (a) language skills, (b) emergent literacy skills, (c) emotional understanding and social-cognitive skills, (d) social-emotional behaviors, (e) learning engagement at school, and (f) learning engagement at home. Bierman, Domitrovich et al. (2008) said posttest observational measures were obtained from observers naive to condition, and Bierman et al. (2014, p. 156) said with reference to the 1-year follow-up measures that child interviewers and kindergarten teachers did not know intervention group status. However, Head Start teachers both implemented the program and provided some measures at posttest.

Language skills. Three tests were administered directly to children to assess their language skills. In the Expressive One-Word Picture Vocabulary Test, children gave the word that best described pictures they were shown. The Grammatical Understanding subtest of the Test of Language Development assessed syntax comprehension. Children listened to a sentence and chose one of the four pictures that "best matched" the meaning of the sentence. The Sentence Imitation subtest assessed syntax expression. Children repeated sentences read aloud by the interviewer. Scores reflected the number of increasingly complex sentences a child imitated correctly.

Emergent literacy skills. Three subscales assessing emergent literacy skills were drawn from the Test of Preschool Early Literacy (TOPEL). The Blending subtest assessed phonological processing. Children were asked to combine different parts of a word, such as "hot" and "dog" or "b" and "air" and point to the correct picture or say the full word. On the Elision subtest of TOPEL, children deconstructed compound words and pointed to the correct picture. On the Print Knowledge subtest of the TOPEL, children identified pictures of letters or words and named letters.

The 1-year follow-up (Bierman et al., 2014) used somewhat different measures at the end of kindergarten. Instead of the Blending measure, the follow-up used a measure of Phonemic Decoding Efficiency based on the number of words sounded out accurately. Instead of the Elision measure, the follow-up used a measure of Sight Word Efficiency based on the number of words read accurately. Instead of the Print Knowledge measure, the follow-up used a measure of letter-word identification from the Woodcock-Johnson Test of Achievement.

Emotional understanding and social-cognitive skills. Two measures assessed emotional understanding. On the Assessment of Children's Emotion Skills, children determined whether the facial expressions in 12 photographs reflected happy, mad, sad, scared, or no feelings. The score was the total number correctly identified. On the Emotion Recognition Questionnaire children listened to 16 stories describing characters in emotionally evocative situations and identified their feeling by pointing to pictures of happy, mad, sad, or scared faces. Children received a score of 2 for correctly identifying the feeling and a score of 1 for correctly identifying the valence. In addition, social-problem solving skills were assessed using a variation of the Challenging Situations Task. Children were presented with pictures of four peer scenarios (e.g., a peer knocking down blocks, being hit, entering a group, a peer taking a ball). After each scenario, children were asked what they would do in the situation. Their open-ended responses were coded as competent (i.e., appropriately asserting oneself or calmly negotiating a solution), aggressive (i.e., responding with verbal or physical antagonism, intimidation, or force), or inept.

Social-emotional behaviors. Teacher ratings, observer ratings, and parent ratings assessed social competence and aggressive-oppositional behavior. The 13 items of the Social Competence Scale were rated on a 6-point Likert scale (never to almost always) and included prosocial behaviors such as sharing, helping, and understanding other's feelings, as well as self-regulatory behaviors, such as resolving peer problems independently. Ratings provided by lead and assistant teachers were averaged.

Seven items from the Teacher Observation of Child Adaptation-Revised (TOCA-R) assessed overt aggression (e.g., stubborn, yells, fights). Six items from the Preschool Social Behavior Scale-Teacher Form assessed relational aggression (e.g., "Tells other kids he/she won't be their friend unless they do what he/she wants"). Items were rated on a 6-point Likert scale (almost never to almost always). Ratings from lead and assistant teachers were averaged and overt and relational ratings were combined to form a total aggression score. Parents and observers completed the seven items from TOCA-R only.

Learning engagement at school. Teacher ratings were used to assess learning engagement at school, using an eight-item inventory developed for this study. Items were rated on a 6-point Likert scale (strongly disagree to strongly agree), and reflected self-regulation (e.g., "Has the self-control to do well in school" and "Can follow the rules and routines that are part of the school day"), learning motivation and involvement (e.g., "Seems enthusiastic about learning new things"), and compliance (e.g., "Is able and willing to follow teacher directions"). Lead and assistant teacher ratings were averaged.

Observers completed the Adapted Leiter-R Assessor Report to assess task orientation at school. After the child assessments, interviewers rated the child's participation and involvement, using 13 items (e.g., "Shows pleasure in accomplishment and active mastery," "Careful and interested in accuracy," "Alert and interactive"). Each item was rated on a 4-point scale. Scores from the two assessment sessions were averaged.

Teachers also completed the ADHD Rating Scale. Based on the Diagnostic and Statistical Manual symptoms of attention deficit hyperactivity disorder, this scale includes 14 items reflecting attention problems (e.g., "Is easily distracted," "Has trouble following directions"), each rated on a 4-point scale. Lead and assistant teacher ratings were averaged and used in this study to represent a continuous dimension reflecting difficulties with impulse control, distractibility, and sustained attention.

Learning engagement at home. Parents completed the ADHD Rating Scale. In addition, they responded to five questions about children's language and communication at home (e.g., "How many times in a typical week do you and your child have a conversation that lasts 10 minutes or more?", "How often does your child volunteer to tell you about something that happened when you were not with him or her?"). Finally, parents answered six questions about children's engagement in reading at home (e.g., "When was the last time you and your child read a book together?", "How many books did you read at that time?").

Mediation. Bierman, Nix et al. (2008) examined three of the 12 skill measures and four of the 11 behavior measures. Unique to their study, they examined five measures of executive function as outcomes, moderators, and mediators: 1) Backward Word Span measured the ability to repeat a list of words in backward order; 2) Peg Tapping measured the ability of children to tap their peg twice when the interviewer tapped once and vice versa (alpha for 16 trials = .87); 3) Dimensional Change Card Sort (DCCS) measured the ability of children to correctly sort cards by, for example, color after initially sorting them by, for example, size (alpha for six trials = .93); 4) Walk-a-Line Slowly measured the ability of children to walk on a straight line at a slow pace; and 5) Task Orientation measured the ability to focus on a task during a testing session (ratings of multiple dimensions had an alpha of .93).

Follow-up. Outcome measures at 1-year follow-up (Bierman et al., 2014) included 13 items in four domains: a) language and emergent literacy skills, b) learning engagement, c) social competence, and d) aggressive behavior. The follow-up used 10 of the 23 measures used at pretest and posttest. As described above, three other measures of emergent literacy skills were new to the follow-up but were related to the pretest and posttest measures.

Although not treated as outcomes, three measures of kindergarten context served as potential moderators in the follow-up. Observers rated kindergarten classrooms of the study subjects on the quality of student-teacher interactions and the quantity of reading instruction. In addition, standardized tests measured the percent of third-grade scores below the basic category in reading and math proficiency.

In the mediation analysis (Nix et al., 2013), posttest outcome measures served as mediators, and differed in some ways from posttest measures used in other articles. These mediating measures included 1) vocabulary, 2) emergent literacy skills based on a combination of the blending and Elision measures, 3) emotional understanding based on a combination of measures of emotional skills and emotional recognition, 4) competent problem solving, and 5) positive social behavior based on a combination of observer, teacher, and parent ratings of social competence and aggressive behavior (reverse coded).

The mediation analysis used several final outcome measures obtained at the kindergarten follow-up. They included 1) reading achievement based on three measures, 2) learning engagement based on a combination of measures of intellectual curiosity, self-discipline, and attention, and 3) positive social behavior based on the same items as at posttest.

6-year follow-up (Welsh et al., 2020)

At the 6-year follow up, the children's fifth grade teachers completed measures of the children's social adjustment, problem behaviors, academic engagement, and parent involvement. The teachers were unaware of the intervention assignment made in preschool and provided independent ratings.

Social adjustment was assessed using the Social Competence Scale (α = .94 - .96 across data collection waves), four items from the Excluded by Peers subscale of the Child Behavior Scale (α = .83 - .88), and the closeness subscale of the Student-Teacher Relationship Scale (α = .88 - .92) These three scales were averaged to form a composite measure of social adjustment (α = .72 - .77).

Problem behaviors were assessed using 7 items from the Authority Acceptance subscale of the Teacher Observation of Classroom Adaptation - Revised Scale (α = .89 - .93) and 6 items from the Children's Social Behavior Scale - Teacher Version (α = .94 - .96). These two measures were significantly correlated at each time point (r = .50 - .66) and were averaged.

Academic engagement was assessed using the School Readiness Questionnaire (α = .93 - .97) and the Inattentive-Impulsive subscale of the ADHD Rating Scale (α = .93 - .96). These two measures were significantly correlated at each time point (r = .72 - .81) and were averaged.

Parent involvement was measured using the Parent-Teacher Involvement Questionnaire (α = .90 - .95).

8- and 10-year follow-ups (Bierman et al., 2020)

When the children were in 7th and 9th grade, their teachers rated them using the Strengths and Difficulties Questionnaire. This scale contained subscales for conduct problems, emotional symptoms, hyperactivity-inattention, and peer problems. Alphas for these measures across the two time points ranged from .64 to .86.

Analyses: To examine preintervention differences between the intervention and the comparison groups, hierarchical linear models (HLM) were estimated, accounting for the nesting of children within classrooms. Child sex and race were included as Level 1 covariates and center site, cohort, and intervention status were included as Level 2 covariates in each of these models.

The first set of analyses to examine postintervention group differences used HLM to examine the 11 measures of child language, emergent literacy, emotional understanding, and social problem-solving skills that were targeted directly by the intervention and were measured using direct assessments of child skill knowledge. Because the intervention was delivered at the classroom level, analyses accounted for the nonindependence of data at that level. Child sex and race were included as Level 1 covariates and site, cohort, and intervention status were included as Level 2 covariates. Preintervention scores were available for all these measures and also were included as Level 1 covariates. For ease of interpretation, all measures were standardized with a mean of 0 and a standard deviation of 1. By doing this, the coefficient for the intervention effect represents the difference in average expected scores between children in the intervention and control groups as a proportion of a standard deviation.

HLM analyses were also conducted to assess intervention effects on the 12 behavioral ratings provided by teachers, observers, and parents. As before, the HLM nested children within classrooms, included child sex and race as Level 1 covariates and included site, cohort, and intervention status as Level 2 covariates. In this case the researchers only had preintervention parent ratings and preintervention observing ratings of task orientation as Level 1 covariates.

The follow-up analysis used cross-classified hierarchical models in which Level-2 random effects accounted for clustering within both preschool classrooms and kindergarten school districts. The Level-1 equation controlled for baseline outcomes (or something similar when measures changed at follow-up). With outcome measures standardized, the coefficients are comparable to Cohen's d.

The mediation analysis used full information maximum likelihood estimation to minimize any bias that comes from missing data. It also used multilevel models that nested children within their Head Start classrooms. The models controlled for baseline values of the mediating variables by treating them as residualized gain scores, but it appears that the models did not use baseline controls for the kindergarten outcomes.

With data on five time points (posttest, kindergarten, first grade, second grade, and third grade), the 4-year follow-up (Nix et al., 2016) compared developmental trajectories across conditions. For each of six outcomes, latent class growth models identified from two to four trajectory patterns and each child was assigned to the trajectory that he or she most likely followed. The test of the intervention effect was based on the significance of differences in the proportion of children in each developmental trajectory who had been in Head Start REDI versus Head Start as usual. Odds ratios for intervention children following a particular developmental trajectory relative to control children showed the magnitude of the program effect. The analysis was done at the individual level without adjustment for clustering within classrooms (though the ICC = .003) or schools.

For the 6-year follow-up outcomes (Welsh et al., 2020), growth curve models were estimated for each outcome. The models allowed for both random slopes and random intercepts for time within individuals but not for preschool centers, the unit of assignment, because clustering within centers was found to be negligible. Mean growth curves were examined for children in the intervention and control groups from posttest through the 6-year follow-up. Initial growth curves allowed for linear, quadratic, and cubic effects of time. In all growth models, all of the interactions of time with intervention status were nonsignificant or failed to improve model fit and were trimmed. Models controlled for study design features (e.g., cohort, county site), demographics (e.g., child sex, age, race, and family income-to-needs ratio), direct assessments of child skills (e.g., IQ, vocabulary, executive functions, emotional understanding). To control for baseline outcomes, models included baseline parent ratings of scales of social adjustment, aggressive problem behaviors, and attention problems. There was not a comparable measure of parent involvement at baseline.

For the 8- and 10-year follow-ups (Bierman et al., 2020), intervention effects were examined using three-level hierarchical linear models. Level 1 included fixed effects for time (7th or 9th grade assessments), Level 2 included fixed effects for intervention status and covariates (study design features, demographics, baseline assessments of child skills, and similar though not identical baseline measures of outcomes), and Level 3 nested participants within their Head Start classroom. All analyses were conducted with 40 imputed datasets that imputed missing covariate and outcome data.

Outcomes

Implementation Fidelity: Teachers reported completing most lessons, and trainers rated the average fidelity and quality of implementation as between adequate and strong.

Baseline Equivalence: No significant pretreatment intervention-control group differences emerged on the 11 measures of child skills or the 6 measures of child behavior for which pretreatment scores were available. The study did not report on equivalence of sociodemographic characteristics.

At follow-up (Bierman et al., 2014), intervention and control subjects in kindergarten were compared on measures of teacher-student interaction quality, curriculum emphasis on reading instruction, and school-level achievement. The conditions did not differ significantly on these three measures, thus confirming that on entering kindergarten, intervention students were not selected into better schools. Welsh et al. (2020) reported that there were no significant differences at baseline on a number of study design features, demographics, and assessments of child skills. Bierman et al. (2020) reported no significant differences on study design features, demographics, or outcomes at baseline.

Attrition: Attrition was low - only 3-5%. But Nix et al. (2013, p. 1003) stated that there were "few significant differences" between those retained and not retained. Nix et al. (2016) presented no information on differential attrition for the 9% lost by the 4-year follow-up. Welsh et al. (2020) reported 19% attrition at the 6-year follow up. They reported one significant predictor of attrition: African American participants were more likely to drop out. However, their rates of attrition did not differ by condition. Bierman et al. (2020) reported a 19% attrition rate at 10-year follow up. Retention was significantly higher in the control group (88%) compared to the intervention group (78%) and among White non-Latino children (87%) compared to Latino children (72%).

Posttest: In Bierman, Domitrovich et al. (2008), there were significant treatment effects for 7 of 11 skills directly targeted by the intervention and marginally significant intervention effects for two others. Three of 12 behavioral ratings provided by teachers, observers, and parents significantly favored intervention students and five measures showed marginally significant trends favoring the intervention students. Statistically significant differences in aggression were reported by teachers, with those in the intervention group showing less aggression. There were marginally significant differences reported by parents, and no significant differences reported by observers. Students who participated in the intervention showed statistically significant improvements on direct assessments of the children's emotional understanding and social problem-solving skills and emergent literacy skills compared to the control group. When looking at language skills, those in the intervention group had more improvement on measures of child vocabulary and on parent reports of communication and language use at home. Observers also reported more improvements in task orientation among those in the intervention group compared to those in the control group.

Omnibus tests or overall tests of the significance grouping the individual measures in the conceptual domains were requested by Blueprints and subsequently delivered by the evaluators, finding significance in all but one (language skills) of the six domains (emergent literacy skills, social cognitions, social-emotional behaviors, learning engagement at school, and learning engagement at home).

Long-term: Bierman et al. (2014) examined the effects of the intervention on the 1-year follow-up measures (i.e., at the end of kindergarten). The intervention showed significant main effects on 5 of 13 outcome measures: phonemic decoding, teacher-rated learning behaviors, competent problem solving, teacher-rated aggression, and parent-rated aggression. Effect sizes ranged from .22 to .40. In most cases, these effect sizes were equivalent to or larger than those at posttest, and in most cases, they reflected benefits in the social-emotional domain.

Omnibus tests or overall tests of the significance grouping the individual measures in the conceptual domains were requested by Blueprints and subsequently delivered by the evaluators, finding significance in all but one (language/emergent literacy) of four domains (social competence, aggression, and learning engagement).

Significant moderation (p < .05) by kindergarten context emerged for 5 of 39 tests. The authors noted that, even in the absence of main effects, the interaction effects indicated program benefits for subsets of subjects. Specifically, the results indicated that the intervention had stronger effects and greater benefits when there were many low-achieving children in the school on four outcomes: teacher-rated attention problems, parent-rated attention problems, teacher-rated social competence, and teacher-rated aggression in schools. While these four interaction effects showed greater benefits in less positive kindergarten contexts, one other showed greater benefits in a positive kindergarten context. The results revealed stronger intervention effects on competent problem solving in classes with high-quality teacher-student interaction.

To isolate the changes unique to kindergarten, a final analysis added controls for posttest scores. Only one of the main effects of the intervention remained significant, and two were marginally significant. The weaker results suggest that most benefits came during preschool but that these gains were maintained in kindergarten. The moderating effects of kindergarten context, in contrast, were largely maintained with the controls for posttest scores.

Executive Function: In Bierman, Nix et al. (2008), the analysis examined the influence of measures of executive function on the skill and behavioral outcomes and the influence of the executive function measures as moderators of the intervention, as outcomes of the intervention, and as mediators of the intervention.

To determine if the program benefit varied by level of executive function, the models added product terms of treatment by each of the five executive function measures. Two of 35 interaction tests (five moderators by seven outcomes) reached statistical significance and three more reached marginal significance (p < .10). For these five interactions, children with low baseline levels of executive functioning benefitted more from the intervention; children with high baseline levels did well regardless of the intervention.

The intervention significantly improved (p < .05) posttest scores on one of the five executive function measures - Task Orientation - and came close (p = .06) on one more - DCCS. Effect sizes equaled .28 and .20, respectively, for the two outcomes.

Mediation tests examined the two executive function measures that were significantly or near significantly influenced by the program (Task Orientation and DCCS). Of the 14 tested mediation effects (two mediators by seven outcomes), three proved significant. Task Orientation significantly mediated the program effect on print awareness, with the relationship between the program and print awareness declining by 33% with controls for the two mediating variables. Task Orientation significantly mediated the program effect on observer-rated social competence and observer-rated aggression. The reductions due to the mediators equaled 29% for observer-rated social competence and 43% for observer-rated aggression.

In summary, 1 of 5 tests showed that executive function was significantly affected by the intervention, 2 of 35 tests showed that executive function moderated the intervention impact, and 3 of 14 tests showed that executive function mediated the intervention impact. Overall, 11% of the 54 tests involving the measures of executive function reached significance.

Developmental Trajectories: Tests of the latent class growth models showed program benefits for each of six outcomes tested over the five time points between posttest and 4-year follow-up: social competence, aggressive-oppositional behavior, learning behavior, attention problems, student-teacher closeness, and peer rejection. Compared to control children, intervention children had higher odds of belonging to positive trajectories and lower odds of belonging to negative trajectories.

Mediating Effects : The mediation analysis (Nix et al., 2013) treated changes in five outcome measures from baseline to posttest as mediators: 1) vocabulary, 2) emergent literacy skills, 3) emotional understanding, 4) competent social problem solving, and 5) positive social behavior. The intervention significantly improved all five of these mediating outcomes, with effect sizes ranging from .25 to .49.

The analysis examined three kindergarten outcomes: 1) reading achievement, 2) learning engagement, and 3) positive social behavior. The five mediators and three outcomes defined 15 possible indirect effects.

The results demonstrated that gains made during preschool predicted kindergarten functioning. For reading achievement, the intervention had significant indirect effects via four of the five mediators. For learning engagement, the intervention had significant indirect effects via three of five posttest measures. For positive social behavior, the intervention had significant indirect effects via one of five posttest measures (positive social behavior). Gains made during Head Start thus continued to predict functioning one year later, in kindergarten.

However, the study also found that the two conditions did not differ significantly on reading achievement and learning engagement in kindergarten. These results differ from Bierman et al. (2014), perhaps because Nix et al. (2013) used broader versions of the outcome measures.

6-year follow-up (Welsh et al., 2020)

The models did not show significant condition differences in growth over time from posttest to the follow-up but showed overall condition differences for three of the four outcomes. Over the six years following the intervention, children in the intervention condition, compared to the control condition, demonstrated significantly higher social adjustment (d = .34), academic engagement (d = .24), and parent involvement (d = .24). In summary, the results indicated that benefits of the intervention at posttest were maintained through the 6-year follow-up.

8- and 10-year follow-up (Bierman et al., 2020)

Across the 8- and 10-year follow-ups, children in the intervention condition, compared to the control condition, exhibited significantly fewer conduct problems (d = -.20) and emotional symptoms (d = -.25). Additional tests for clinical risk categories affirmed these effects.

Study 2

Evaluation Methodology

Design:

Recruitment: The study recruited families from 24 Head Start centers in three urban and rural Pennsylvania counties during the 2008-2009 and 2009-2010 academic years. Each year, the study sent letters describing the study to all prekindergarten children in participating classrooms. Letters were sent to approximately 400 eligible households, of which 52% elected to participate.

Assignment: The study randomly assigned 95 households to the treatment condition and 105 households to the control condition. Treatment families received the Head Start REDI materials in addition to home visits, while control conditions received only the Head Start REDI materials.

Attrition: The pretest assessment occurred in the fall of the preschool year and the posttest occurred in the spring of the kindergarten school year. The study reported that 4% of households left the study.

Sample: The sample was comprised of 55% white, 26% black, and 19% Hispanic children. The sample was 56% male and averaged 4.45 years of age. The median annual family income among the population was $18,000, 39% single parent, and 54% unemployed.

Measures: All but 2 of 11 measures came from independent observers - teachers and researchers who were unaware of condition. Two measures came from reports of parents, who helped deliver the program. Reliability was acceptable for most measures but was low (alpha = .56) for one of the parent-reported measures.

The study used four measures of child language and emergent literacy skills. First, the study used the Expressive One-Word Picture Vocabulary Test, which assessed vocabulary. Second, the study used the Letter-Word Identification scale of the Woodcock-Johnson Tests of Achievement III-revised, which tests emergent literacy skills. Third, the study used the Test of Word Reading Efficiency, which tests reading fluency. Finally, teachers, who were blind to condition, rated children's academic performance with the 12-item performance rating scale.

The study also used three teacher-rated measures to assess social-emotional adjustment. First, teachers used the School Readiness Questionnaire, which measured child self-directed learning. Second, teachers rated children on 13 items describing prosocial behavior. Finally, teachers rated children on the Teacher Observation of Child Adaptation-Revised, which describes aggressive behavior.

Finally, parent support for learning was assessed by four measures using parent reports, videotaped observations of parent-child interactions, and observations conducted in the home. Parents first described the degree to which they read interactively with their children, using five items from the Participation subscale of the Reading Belief Inventory and second described the quality of conversation with their children. Third, trained coders rated videotapes of parent-child interactions during structured tasks. Finally, observers rated parents with a modified version of the Post-Visit Inventory to measure parental warmth and support.

Analysis: The study used cross-classified hierarchical linear models, nesting children within their Head Start classrooms and elementary school districts. The models included pretest outcomes as covariates but the vocabulary measure served as the control for the other language and literacy measures that were obtained only in kindergarten. Other controls included kindergarten school quality, county, cohort, age, sex, race, aggression, single-parent status, caregiver depressive symptoms, and family income-to-needs ratio.

Intent-to-Treat: The study included all available data, excepting only the reported 4% attrition.

Outcomes

Implementation Fidelity: The study reported that families completed an average of 12 of 16 total visits, and visitors reported a parent-engagement score of 2.27 out of a possible 3.

Baseline Equivalence: The study reported that one outcome measure (parent support for learning) favored the comparison group at pretest, but all other sociodemographic and outcome measures were equivalent across the intervention and control groups at pretest.

Differential Attrition: With less than 4% attrition, the study did not provide additional analysis of attritors.

Posttest: A significant intervention effect was found for 2 of 4 measures of language and literacy skills. Children in the treatment group had significantly better scores in emergent literacy and academic performance at posttest as compared to children in the control group. In addition, the study found a significant intervention effect for 2 of 3 measures of social-emotional learning. Children in the treatment group had better scores at posttest in teacher-rated self-directed learning and social competence.

Finally, the study found a significant intervention effect for 2 of 4 risk and protective factors in parent support for learning. Parents in the treatment group reported higher scores for reading quality and conversations. Observer-rated scores on parent support did not differ across conditions.

Long-Term: The study did not report a long-term follow-up.

Limitations:

  • Some measures came from reports of parents, who helped to deliver the intervention
  • One outcome measure not equivalent at baseline

Study 3

This evaluation combined the samples from Study 1 and Study 2 but also extended the follow-up period to second grade and directly compared two versions of the program. One version located all program activities at school and the other version located activities at both school and home.

Evaluation Methodology

Design:

Recruitment: The combined sample of two previous studies included 556 children who were recruited from Head Start centers in three counties in Pennsylvania. The children were 4-5 years old and eligible to start kindergarten in the following year. The participating Head Start centers came from one large county with both an urban community and smaller surrounding communities and from two smaller counties with small towns and rural areas.

While most eligible children (n=356; 86%) participated in the first study of the school program, only 52% (n=264) of the eligible parents agreed to participate in the second study of the school and parent program. The two samples thus differed, and as noted below, the authors weighted cases to control for selection into the second sample. Still, the representativeness of the two samples may differ.

Assignment: In the first study, 356 families were cluster randomized by Head Start center to the Head Start with REDI-C school intervention (n = 192) or the usual practice Head Start control group (n = 164). The randomization stratified the 25 Head Start centers on location, length of program (e.g., half day vs. full day), and student demographic characteristics (e.g., percent underrepresented minority).

In the second study, 200 families were individually randomized within classrooms by child to the combined REDI-C school and parent intervention (n = 95) or the school REDI-C comparison group (n = 105). The school-only parents received home materials via mail but in a different domain. Because all children participated in the school program, randomization of children could occur within classrooms and centers.

The full sample used in Study 3 thus consisted of three groups: usual practice Head Start (control), REDI-C school intervention only, and REDI-C plus the parent intervention. Note that the condition sample sizes listed in the CONSORT diagram do not match those listed in the text (p. 131). For example, the CONSORT diagram shows 95 in the school and parent program, while the text reports 105.

Assessments/Attrition: The follow-up assessment occurred near the end of second grade, 3.5 years after baseline (the start of the prekindergarten year) and more than two years after the program end. According to the CONSORT diagram, completion rates were 89% in sample 1 and 83% in sample 2.

Sample:

The combined sample averaged 4.48 years of age at baseline, had 49% males, and was 55% European American, 25% African American, and 19% Latino. Families were low-income (median annual income of $16,800), and 38% were single parents.

Measures:

The outcome measures came from researchers, students, and teachers. Trained research assistants visited schools, distributed rating forms to teachers, and conducted standardized child assessments during individual pull-out sessions. The research assistants who conducted the assessments and the second-grade teachers who provided ratings were unaware of the condition membership. The measures used previously validated instruments, and the scales had good reliabilities for the study sample.

The study divided the behavioral outcomes into two domains. In the domain labeled mental health, there were seven outcomes related to prosocial relationships with peers (e.g., social competence, peer problems) and risk and protective factors (e.g., classroom participation, student-teacher relationship quality). In the academic domain, there were five outcomes related to academic performance (e.g., letter-word identification, math skills).

Analysis:

The analysis used regression models with covariates for baseline outcomes, child and family demographic characteristics, preintervention child functioning, and study design features. The models presented separate tests for condition differences across the two samples, with the first tests comparing the school program to the control group, and the second tests comparing the combined school and parent program to the school program alone.

The analysis combined the two samples. Because consent differed widely between the two samples, the analysis weighted the second unrepresentative sample by propensity scores to match the characteristics of the first representative sample. The propensity scores came from a logistic regression using 44 covariates from the initial baseline assessment. After weighting, the two samples were equivalent within 0.2 standard deviations on all baseline measures.

Intent-to-Treat: The authors briefly noted that "Multiple imputation was used to estimate missing data" and that the intent-to-treat analysis included all families as randomized.

Outcomes

Implementation Fidelity:

Parents received (on average) 12 of the 16 planned home visits, and only a small number of families (13%) participated minimally, accepting 0-3 home visits.

Baseline Equivalence:

The study did not test for baseline equivalence, but information from each of the original and separate studies applies. The first sample (Study 1 above) showed no significant condition differences for the outcomes but did not examine sociodemographic measures. The second sample (Study 2) above showed one significant condition difference for the outcomes and sociodemographic measures (parent support for learning favored the control group.

Differential Attrition:

The study did not test for differential attrition, although attrition rates varied by condition. In sample 1, the attrition rate of 16% in the intervention group exceeded the attrition rate of 6% in the control group. In sample 2, the attrition rates were similar, 18% in the school-only condition and 16% in the school and parent condition. Most dropouts (73%) had moved or were unavailable, while the other 27% refused.

Posttest:

Not examined.

Long-Term:

In comparing the school program to the usual practice, five of seven tests of social relations outcomes were significant. The school intervention group did significantly better than the control group on teacher-reported measures of classroom participation, social competence, teacher-student relations, and peer problems and on student-reported perceived social competence. Effect sizes ranged from .24 to .39. However, none of the five academic outcomes differed significantly across conditions.

In comparing the combined school and parent program to the school-only program, two of seven tests of social relations outcomes were significant. The combined intervention group reported greater student-reported outcomes of perceived social competence (d = .43) and lower perceived peer problems (d = -.47). The combined intervention also did better for academic outcomes, significantly improving three of five outcomes: sight word reading, reading skills, and math skills. Effect sizes ranged from .30 to .50.

Figures 2 and 3 added the effect sizes from both samples, which ranged from .23 to .78.