In-service professional development and continuing education are considered essential for educators to become proficient and sustain expertise in their teaching professions (Donovan and others 1999; Guskey 2002; Guskey 2014). According to Darling-Hammond et al. (2009), “well-designed professional learning helps teachers master content, hone teaching skills, evaluate their own and their students’ performance, and address changes needed in [their] teaching and learning” (p. 7). Yet, many teachers and educators consider themselves ill prepared for their professions (Akiba et al., 2007; Lewis et al., 1999). At least one reason for teachers’ judgments of their lack of preparedness is the type of continuing professional development either offered or procured as part of in-service training opportunities.

As part of a status report on teacher development, Darling-Hammond et al. (2009)found that attendance at one-time workshops, conferences, or training sessions are the primary types of in-service professional development for nearly all teachers and that other types of in-service training considered more effective are experienced much less often. Similar findings were reported by Lewis et al. (1999)as part of their analyses of the in-service experiences associated with teacher preparedness and quality. Darling-Hammond et al. (2009)concluded their review of teacher professional development by stating, “We found that well-designed professional development is still relatively rare, and few of the nation’s teachers have access to regular opportunities for intensive learning” (p. 19).

The purpose of the metasynthesis described in this paper was to ascertain the extent to which studies of in-service professional development that included key characteristics and core features of in-service training considered effective by professional development specialists were associated with changes and improvements in educator and student outcomes (Desimone 2009; Guskey, 2002). This was accomplished by identifying research syntheses of in-service professional development and by both coding and systematically analyzing the types of in-service afforded teachers and educators to determine whether the inclusion of key characteristics and core features were, in fact, related to positive teacher and student outcomes. The term metasynthesis refers to the “bringing together and breaking down of [quantitative and qualitative] findings, examining them, discovering essential features, and combining phenomena into a transformed whole”  (Schreiber et al., 1997).

The investigators of the research syntheses included in the metasynthesis either explicitly or implicitly employed a framework similar to the one shown in Figure 1. The framework is based on those proposed by Desimone (2009) and Guskey (2002)for designing and researching in-service professional development. According to the model, professional development that includes key characteristics and core features is expected to increase or improve teacher knowledge, skills, and practices and, in turn, be related to improved student and child outcomes. 

 

 

The characteristics and features that have been identified as important for in-service training to be effective include professional development specialists’ explicit explanations and illustrations of specific content knowledge and practice to  be  learned  (Archibald  et  al., 2011; Desimone, 2009; Donovan et al., 1999; Dunst and Trivette 2009; Garet et al., 2001; Wei et al., 2009), active and authentic teacher learning opportunities (Archibald et al., 2011; Desimone 2009; Donovan et al., 1999; Dunst and Trivette, 2009; Garet et al., 2001; Guskey 2002; Wei et al., 2009), explicit inclusion of different types of practices for engaging teachers in reflection on their understanding and mastery of content knowledge or practice (Archibald et al., 2011; Desimone 2009; Donovan et al., 1999; Dunst and Trivette, 2009; Garet et al., 2001; Wei et al., 2009), coaching, mentoring, and performance feedback during the in-service training (Archibald et al., 2011; Donovan et al., 1999; Garet et al., 2001; Guskey 2002; Wei et al., 2009), ongoing follow-up supports to reinforce in-service learning (Archibald et al., 2011; Donovan et al., 1999; Dunst and Trivette 2009; Guskey 2002; Wei et al., 2009), and professional development of sufficient duration and intensity to provide repeated opportunities to become proficient in the use of content knowledge and practice (Archibald et al., 2011; Desimone, 2009; Dunst and Trivette 2009; Garet et al., 2001; Guskey 2002; Wei et al., 2009). Accordingly, in-service professional development that included the majority of these key characteristics and features was expected to be associated with positive teacher and student outcomes.

A multiple case design was used to analyze the research syntheses in the metasynthesis (Riedl, 2007; Yin, 2014). According to Yin (2014), multiple case research is grounded in a theoretical or conceptual framework that provides a foundation for testing hypothesized relationships between independent and dependent variables in order to establish causal or explanatory inferences. Therefore, each research synthesis was considered a separate case, and the extent to which the relationships between in-service professional development and teacher and student outcomes were the same or very similar in the research syntheses was the focus of analysis.

Search strategy

Research syntheses were located using the following search terms: (in-service OR in-service) AND (professional development OR staff development OR continuing education OR training) AND (literature review OR narrative review OR systematic review OR meta-analysis OR summative review OR traditional review) AND (teacher OR educator OR practitioner) AND (early intervention OR early childhood OR preschool OR elementary OR secondary). Follow-up searches were conducted using controlled vocabulary, key word, and natural language searches as alternative terms were identified from retrieved publications and reports.

ERIC, PsychInfo, MEDLINE, Academic Search Complete, CINAHL, and Health Source were searched to identify research syntheses.  These   were   supplemented  by  searches  of  Infotrac, ProQuest, WorldCat, Google Scholar, and Google. The reference sections of retrieved journal articles, book chapters, books, dissertations, and other published and unpublished reports and papers were examined to identify additional reviews.

Research syntheses were included if in-service professional development was the main focus of a literature review, there was an explicit attempt to identify the characteristics of and conditions under which in-service training was effective, and sufficient information was included in the reports to code and conduct secondary analyses of the relationships between the key characteristics of in-service professional development and findings in the research syntheses. Literature reviews were excluded if any of the three inclusion criteria were not met, or after an initial review of a research synthesis, it was determined that insufficient information was reported to be able to ascertain the scope of in-service training.

 

Search results

More than 25,000 abstracts (including duplicate abstracts in different databases) were generated from searches. These were reviewed to determine which were literature reviews and research syntheses, and which included studies or evaluations of in-service professional development in early childhood, elementary, or secondary education. This resulted in a preliminary list of 36 reviews that were then examined to determine if they met the inclusion criteria. Eighteen reviews were initially considered relevant for the metasyntheses. Three reviews were subsequently excluded because they included either too little information about in-service training (Cornelius and Nagro, 2014; Solomon et al., 2012)or the in-service training in the studies in the review was limited in terms of the characteristics of the professional development afforded the teachers (Gersten et al., 2014). Eight of the research syntheses were published in peer reviewed journals and seven syntheses were unpublished government or professional organization reports.

 

Metasynthesis coding

Table 1 includes the in-service professional development features that were coded and used to conduct the secondary analyses of the reviews as well as the description or definitions of the five sets of characteristics. The core features were developed based on characteristics described by a number of professional development specialists as essential for in-service professional development to be effective (Bransford et al., 2000; Darling-Hammond et al., 2009; Desimone, 2009; Guskey, 2002). 

 

 

Focus of training. The focus of training included both learner objectives and the content knowledge or practice that was the focus of in-service professional development. According to Desimone (2009), in-service professional development is most likely to be effective if it emphasizes specific content knowledge and the instructional practices used by teachers to promote student/learner understanding and use of the knowledge.

In-service setting. The settings in which the in-service training was conducted were coded as either or both the teachers’ classrooms or early childhood intervention settings (job embedded) or settings other than those where teachers or early childhood practitioners taught students or worked with young children (non- job embedded). The settings in which in-service training was conducted were coded as either primary or secondary based on how much of the professional development was conducted in either of the two types of settings (Table 1).

In-service characteristics. Desimones’ (2009)core features of professional development,  findings in How People Learn (Donovan et al., 1999), and recommendations in other sources (Guskey, 2014; Zaslow, 2014)were used to operationalize and code six different characteristics of the in-service training afforded the teachers. These included the methods used by professional development specialists to introduce the content knowledge or practice to the teachers and the methods used to illustrate or demonstrate the use and importance of the content knowledge or practice. The teachers’ role in learning the content knowledge or practice was coded in terms of type of active involvement (authentic or real-life opportunities, simulations, etc.) in learning to use the content knowledge or practice and the methods used to engage the teachers in reflection on their understanding and mastery of the content knowledge or practice. In-service support was coded in terms of coaching or mentoring to promote and strengthen the teachers’ confidence and competence during the in-service training or direct performance feedback on how well the teachers applied content knowledge or used an intervention or instructional practice.  

Each of the six characteristics was coded as either a primary or secondary focus of the in-service training based on information in the research syntheses.

Research syntheses outcomes. The research syntheses were coded in terms of both teacher and student/child outcomes. The teacher outcomes included changes or improvements in attitudes or beliefs, changes or improvements in content/subject area know-ledge, and changes or improvements in the use of instructional or behavioral practices. The student or child outcomes included improvements in student knowledge or academic performance, child skill acquisition, and changes in student or child behavior.

Metasynthesis findings. The dosage of in-service training was ascertained in terms of in-service duration (number of sessions, hours or length of training). Follow-up training was coded in terms of the types of ongoing extended supports provided to reinforce in-service learning in the teachers’ schools, classrooms, or early childhood intervention settings.

The results from the research syntheses were ascertained from the findings reported by each research synthesist or by secondary analysis of the results in the literature reviews in terms of the in-service professional development characteristics associated with positive teacher and student outcomes.

 

Interrater agreement

Two of the investigators independently abstracted and coded information for the in-service features in Table 1 as well as background information about the studies in the research syntheses (e.g., type of synthesis, research designs, number of studies). Interrater agreement of the Table 1 features was attained on 87 to 100% of the 15 research synthesis characteristics. Interrater agreement for the three metasynthesis findings, for example, was 93% for in-service duration, 100% for ongoing extended supports, and 93% for the research synthesis findings. Disagreements were resolved through repeated reviews of the research reports until 100% agreement was reached on all information by both metasynthesists.

 

Method of analysis

A replication logic was used to ascertain if the presence of different in-service professional development features and characteristics was associated with the same or similar results in each of the research syntheses (Hak and Dul, 2010b; Riedl, 2007; Yin, 2014). According to Hak and Dul (2010a)and Yin (2014), replication is demonstrated when the characteristics of each case (research synthesis) are much the same and are associated with similar results, and the nature of the relationships among independent and dependent variables allow causal or explanatory inferences. As noted by Eisenhardt and Graebner (2007), the use of replication logic in case study research contributes to theory building which in the case of in-service professional development research either confirms or disconfirms the hypothesized relationships between the core features of in-service training and teacher and student outcomes (Desimone, 2009; Guskey, 2002). 

Research syntheses

Selected characteristics of the 15 research syntheses and study participants are shown in Table 2. Five of the syntheses were traditional narrative reviews, four were meta-analyses, three were systematic reviews, and three were summative reviews (Davies, 2000). Seven syntheses included only group design studies, and six syntheses included a mixture of group design studies and either descriptive case studies or single subject studies. The majority of group design studies included experimental, quasi-experimental, and pre-experimental investigations or program evaluations (Shadish et al., 2002). One research synthesis included only experimental studies (Dunst et al., 2010),  and two research syntheses included only experimental and quasi-experimental studies (Blank and De las Alas, 2009; Yoon et al., 2007). The investigators of two research syntheses did not include information in their reports about the types of studies in their reviews (Joyce and Showers, 1995; Saylor and Johnson, 2014).

The 15 research syntheses included more than 550 studies. The participants included PreK or K to  grade  12 teachers (N = 8 reviews), K to grade 5, 6, or 8 teachers (N = 3 reviews), early childhood practitioners (N = 3 reviews), or both PreK to grade 12 teachers and other non-educators (N = 1 review). The research syntheses that included the number of participants or where the number could be estimated from information in the research reports found that the studies included more than 43,000 teachers, educators, and other adult learners. Based on information in the research syntheses that did not include the number of participants, it was conservatively estimated that the 15 reviews included as many as 50,000 early childhood, elementary, and secondary education teachers and students/children.

 

Focus of in-service training

Eleven research syntheses included studies of in-service professional development to promote use of different types of instructional or behavioral practices, two research syntheses included studies to promote teacher understanding and use of content knowledge or skills, and two research syntheses included studies of in-service training to promote teacher or practitioner use of different job-related practices or to support teachers’ confidence in their teaching practices. The content areas of in-service training included mathematics or science (N = 5 reviews), teacher-child interactions (N = 1 review), teacher praise (N = 1 review), teacher confidence (N = 1 review), or a mixture of different content knowledge and practice (N = 7 reviews).

 

In-service training context

Five of the research syntheses included studies where the preponderance of in-service professional development was provided in non- job-embedded settings, and seven syntheses included studies where all or most of the in-service training was provided in teachers’ classrooms or schools, childcare or preschool settings, or other work environments. Three research syntheses included studies where in-service professional develop-ment was provided in both job-embedded and non-job-embedded settings in about equal amounts.

 

Characteristics of the in-service training

Table 3 shows the particular characteristics of in-service professional development that were included in the majority of studies in the research syntheses. All of the research syntheses included studies that incorporated at least 4 of the 6 characteristics as either primary or secondary practices (Mean  =  5.20,  SD  =  0.77).  Eighty percent of the research syntheses (N = 12) included practices for 5 or 6 of the characteristics. All of the research syntheses included both professional development specialist descriptions (introduction) of content knowledge or practice constituting the focus of in-service training and some type of authentic teacher learning opportunities. Most of the research syntheses included the majority of key characteristics and features considered necessary for in-service professional development to be effective (Desimone, 2009; Donovan  et  al.,  1999; Ericsson and Charness, 1994; Guskey, 2002; Zaslow, 2014).

 

 

 

Research synthesis outcomes

Acquisition or improvements in teacher instructional or behavioral practices were the primary outcomes in 14 research syntheses. Content knowledge mastery and use were the outcomes in eight research syntheses, and changes in teacher attitudes or beliefs were also the outcomes in eight syntheses. Five research syntheses included all three teacher outcomes (practice, knowledge, attitudes). Eleven of the research syntheses included 2 of the 3 teacher outcomes.

Nine research syntheses included student academic performance, knowledge acquisition, or skill development as the primary child outcome measures, and four research syntheses included student or child behavioral outcome measures. Three research syntheses included both types of child outcomes.

Twelve research syntheses included both teacher instructional practices and student or child outcome measures. Five research syntheses included both teacher content knowledge and instructional practice outcomes and student or child outcome measures. Five research syntheses included only teacher outcome measures, and one research synthesis included only student outcome measures.

 

Metasynthesis findings

Table 4 shows the findings from each research synthesis in terms of the dose of the in-service professional development, the extended supports provided to the teachers to reinforce in-service learning, and the findings from the research syntheses. The patterns of results are remarkably similar regardless of type of research synthesis, types of studies included in the syntheses, and types of content knowledge or practice. Taken together, the metasynthesis indicated that in-service professional development was effective when it included most of the key characteristics and core features described in Table 2, was of sufficient duration and intensity, and included extended follow-up supports and opportunities to reinforce the use of content knowledge or practice.

In-service dose. Fourteen of the research syntheses included information about the duration or amount of in-service training afforded the teachers. Eight of the research syntheses included explicit descriptions of “how much” in-service professional development was associated with positive teacher or student outcomes (Blank and De las Alas, 2009; Blank et al., 2008; Dunst and Trivette, 2012; Joyce and Showers, 1995; Saylor and Johnson, 2014; Snow-Renner and Lauer, 2005; Yoon et al., 2007; Zaslow et al., 2010). The number of hours of in-service training associated with positive effects ranged between 15 and 80+. In a number of reviews, it was stated that multiple in-service sessions distributed over weeks or months of professional development was a factor contributing to positive and significant effects (Dunst and Trivette, 2012; Fukkink and Lont 2007; Isner et al., 2011; Joyce and Showers, 1995).

The dose of in-service professional development reported in three research syntheses was similar in terms of the hours, intensity, or number of sessions although no relationships between dose and teacher or student outcomes were reported, nor could they be discerned from information in the synthesis reports (Blank and De las Alas 2009; Fukkink and Lont 2007; Isner and others 2011). Nonetheless, it could be surmised that the similar doses were factors likely contributing to positive outcomes.

The fact that different doses of in-service professional development were found to be associated with positive outcomes was neither surprising nor unexpected. As noted by Zaslow et al. (2010), smaller dosages of professional development may suffice for discrete practices, whereas larger dosages may be necessary for broader-based and comprehensive sets of practices.

Ongoing supports. All of the research synthesis included information about the nature and extent of follow-up supports afforded teachers after the completion of the initial in-service professional development. Ten investigators explicitly stated that ongoing follow-up supports were a factor that reinforced in-service training, whereas three investigators made statements, or it could be surmised, that follow-up supports contributed to positive outcomes (Cavanaugh, 2013; Ingersoll and Kralik, 2004; Saylor and Johnson, 2014). In the majority of cases, the conclusions or statements made by the research synthesists permitted inferences about the importance of extended supports as a factor associated with, or contributing to, positive teacher and student outcomes.

Inferences about the links between extended supports and positive outcomes derive from the fact that the same or similar statements were made by many research synthesists (Capps et al., 2012; Ingersoll and Strong 2011; Kretlow and Bartholomew 2010; Zaslow et al., 2010), or it was possible to discern the conditions under which extended supports were associated with positive outcomes (Cavanaugh 2013; Saylor and Johnson, 2014).

Blank and De las Alas (2009), for example, explicitly stated “the importance of continuing learning reinforcement activities after the initial period of teacher training” (p. 24) as a factor contributing to positive student outcomes. This type of inferential statement was echoed by many research synthesists (Blank et al., 2008; Ingersoll and Strong, 2011; Kretlow and Bartholomew, 2010).

Research synthesis results. Investigators of all 15 research syntheses reported or described the characteristics of and conditions under which in-service professional development was most effective. What is reported in the table is the particular in-service  professional development characteristics that the research synthesists or the metasynthesists found associated with positive teacher or child outcomes.

Most research synthesists concluded that the in-service professional development afforded the study participants “produced strong evidence,” “showed significant effects,” “was most effective,” “provided empirical support,” etc. when it included trainer introduction, demonstration, and explanation of the benefits of mastering content knowledge or practice; active and authentic teacher learning experiences together with opportunities to engage in reflection on the use of the content knowledge or practice; and coaching, mentoring, or performance feedback during both the in-service professional development and follow-up sessions in the settings where the teachers used the content knowledge or practice.

Thirteen of the research synthesists included explicit statements or conclusions about the key characteristics and core features of in-service professional development that were found to be associated with positive teacher or child outcomes. The statements or conclusions in Table 4 are either direct quotations or paraphrased descriptions in the research syntheses reports. The results from two of the research syntheses are summarizations of findings which contain information about the particular in-service practices that were found to be associated with positive outcomes (Yoon et al., 2007; Zaslow et al., 2010).

The patterns of results, taken together, provide strong evidence for the relationship between specific in-service professional development characteristics and core features and teacher and student outcomes. The fact that the results were the same or similar in the different types of research syntheses for different types of practices bolsters contentions about the necessary, but not the sufficient, conditions for in-service training to be effective. 

 

 

 

The metasynthesis   described   in    this    paper    used replication logic to determine the extent to which research on in-service professional development that included an attempt to identify which in-service training characteristics under which conditions were associated with positive teacher or student outcomes. A multiple case design was used to select cases (research syntheses) that included the same or similar in-service professional development characteristics and core features and to determine the extent to which the use or presence of these characteristics or features was related to the same or similar teacher or student outcomes. As noted by Yin (2002), a multiple case study design is analogous to the ability to conduct multiple experiments on the same or related topics or practices.

The focus of analysis in the metasynthesis was the extent to which there was literal replication of the results (pattern matching) between the use of the key characteristics and core features of in-service professional development and either the results reported by the research synthesists or those ascertained by the metasynthesists. The following three sets of characteristics were used to determine the extent to which literal replication (Yin, 2014)was demonstrated: (1) the characteristics of in-service professional development used to promote teacher, educator, and early childhood practitioner understanding and use of content knowledge or instructional practices, (2) the extended supports that were used to reinforce in-service learning, and (3) in-service training of sufficient duration and intensity to ensure in-service recipients had sufficient time and opportunity to learn and become proficient in the knowledge or practices constituting the focus of in-service professional development.

Results showed that replication was demonstrated in all 15 research syntheses for the in-service professional development characteristics (100%), in 13 research syntheses for extended follow-up supports (87%), and in 12 research syntheses for in-service duration and intensity (80%). Taken together, the three sets of findings (evidence) provide support for the contentions made by Donovan et al. (1999), Desimone (2009), Guskey (2002), and others (e.g., Zaslow 2014)with regard to planning and conducting in-service professional development so it includes key characteristics and core features to increase the probability of the effectiveness of in-service training. The fact that nearly all the synthesists of the reviews included in our metasynthesis independently came to the same or similar conclusions about what matters most in terms of effective in-service professional development highlights the importance of the particular characteristics identified as most important in terms of changes in teacher and student outcomes. In each of the reviews, the synthesists attempted to identify a subset of studies that yielded positive  teacher  or  student  outcomes,  and then proceeded to unpack and disentangle which in-service characteristics under which conditions were associated with positive effects and outcomes.

The metasynthesis, however, was not able to determine whether changes in teacher learning were associated with improvements in student outcomes as purported by a number of research synthesists. As noted in the introduction, a number of research synthesists either explicitly or implicitly hypothesized the types of relationships depicted in Figure 1. This framework and conceptual model constituted the theory-of-change that guided the analysis of the 15 research syntheses described in this paper. There were, however, no attempts to explicitly ascertain the relationships between changes in teacher knowledge, practices, or attitudes and beliefs to changes or improvements in student academic performance, knowledge, or behavior either because the investigators of the primary studies did not do so or the research synthesists did not attempt to relate teacher and student outcomes. This was most certainly a shortcoming of many if not most of the research syntheses and in turn is a shortcoming and limitation of the metasynthesis.

The need for studies, and research syntheses of those studies, where the kinds of relationships depicted in Figure 1 are an explicit focus of analysis are clearly needed if advances are to be made in terms of a more complete understanding of which in-service characte-ristics implemented under which conditions (e.g., setting, duration, follow-up supports) are directly and indirectly related to teacher and student outcomes. As noted by Yoon et al. (2007), “to substantiate the empirical link between professional development and [student outcomes], studies should ideally establish two points. One is that there are links among professional development, teacher learning and practice, and student learning. The other is that the empirical evidence is of high quality--that the study proves what it claims” (p. 3).

The types of linkages that Yoon et al. (2007)call for are those that implementation science methodologists (Kelly and Perkins, 2012)consider necessary for demonstrating the direct effects of implementation practices (in-service professional development) on the use of intervention practices (e.g., teacher instructional methods), the direct effects of intervention practices on learner outcomes (e.g., student achievement), and the indirect effects of implementation practices on learner outcomes mediated by intervention practices (Dunst et al., 2013; Rudnick et al., 2012). These types of studies, and research syntheses of the studies, are the next generation of research that is likely to shed light on how in-service professional development influences and is related to both teacher and student outcomes. Advances in an understanding of how in-service professional development is associated with teacher  and  student  benefits  is most likely to occur if different key characteristics and core features of in-service training are measured and related to outcomes of interest

 

Implications for practice

The findings reported in this paper provide additional empirical support for the professional development frameworks described by Browder et al. (2012), Desimone (2009), Dunst and Trivette (2009), Guskey (2014), and others (Gall and Vojtek 1994; Glazer and Hannafin 2006; Joyce and Showers 2002). These frameworks constitute particular ways for  planning and conducting in-service training to promote and improve teacher acquisition of content knowledge and instruc-tional practices and, in turn, to enhance child and student learning and competence. Each of the frameworks includes methods and strategies, guidelines and activities, and suggestions for ensuring that in-service professional development includes key characteristics and core features. These characteristics and features include, but are not limited to, the methods and procedures to introduce and illustrate or demonstrate content knowledge or practice to teachers, authentic teacher learning opportunities and teacher reflection on knowledge and skills acquisition, in-service professional development specialist coaching, mentoring, or feedback during the in-service training, extended and ongoing follow-up supports to reinforce in-service learning, and in-service professional development of sufficient duration and intensity to promote teacher mastery and continued use of the content knowledge or practice constituting the focus of in-service training.

A particular finding in the metasynthesis that deserves special attention in planning and conducting in-service professional development is the appropriate dose of in-service teacher training and the need to explicitly include distributed teacher learning opportunities with enough time between opportunities to reflect on and internalize knowledge and skill acquisition and to receive ongoing supports to reinforce teacher mastery. As noted by Zaslow et al. (2010), the dose necessary to produce observable and sustained effects is likely to differ depending on the complexity of the knowledge or practice of in-service professional development, but regardless of complexity, effective in-service professional development includes multiple teacher learning opportunities rather than in-service training in only one or a few sessions. 

The author(s) have not declared any conflict of interests.

The preparation of the metasynthesis described in this paper was supported, in part, by funding from the U.S. Department of Education, Office of Special Education Programs (H235B120004). The opinions expressed, however, are those of the authors and do not necessarily reflect the opinions or positions of either the Department or Office.