Educational Research and Reviews

  • Abbreviation: Educ. Res. Rev.
  • Language: English
  • ISSN: 1990-3839
  • DOI: 10.5897/ERR
  • Start Year: 2006
  • Published Articles: 2008

Full Length Research Paper

Factors affecting higher order thinking skills of students: A meta-analytic structural equation modeling study

Prayoonsri Budsankom1, Tatsirin Sawangboon1, Suntorapot Damrongpanit2* and Jariya Chuensirimongkol3
1Faculty of Education, MahaSarakham University, Thailand. 2Faculty of Education, Chiang Mai University, Thailand. 3Faculty of Nursing, Navamindrahiraj University, Thailand.
Email: [email protected]

  •  Received: 28 June 2015
  •  Accepted: 05 October 2015
  •  Published: 10 October 2015


The purpose of the research is to develop and identify the validity of factors affecting higher order thinking skills (HOTS) of students. The thinking skills can be divided into three types: analytical, critical, and creative thinking. This analysis is done by applying the meta-analytic structural equation modeling (MASEM) based on a database of 166 primary empirical studies. The research results assert the theories and bring conceptual and empirical clarity to the factors affecting HOTS of students and also give readers an understanding of the magnitude and significance of relationships among the variables in the model. MASEM results confirm that classroom environment, psychological and intellectual characteristics of students have direct effects on HOTS (96.8% explained variance). Whereas, the family characteristic had insignificant effects on HOTS but they had indirect effects on HOTS through psychological characteristic. Furthermore, we show that the most direct effects on HOTS were psychological characteristic, classroom environment and intellectual characteristic, respectively. This study provided a holistic view on the relationship of factors affecting HOTS and proposed a direction for future research and practice.

Key words: Higher order thinking skills, meta-analytic structural equation modeling, classroom environment, family characteristic, psychological characteristic, intellectual characteristic.


Higher Order Thinking Skills (HOTS) is a thinking process, which consists of complicated procedures and needs to be based on various skills such as analysis, synthesis, comparison, inference, interpretation, assessment,  and   inductive   and   deductive   reasoning  to  be employed to solve unfamiliar problems (Smith, 1992; Zohar and Dori, 2003). The characteristics of students with HOTS are open-mindedness for risk-taking, curiosity, keen on fact discovery, planning and indicating the most suitable method, have a systems thinking process, think carefully, use evidence to think rationally and frequent self-monitoring (Shari et al., 1993). The students with HOTS are able to create new knowledge and make appropriate and logical decisions. Information and technology advancement greatly influences the current society. Consequently, learning management must be adapted to the current situation/society and focus on improving HOTS of students.

There are many concepts of HOTS applied to the educational development of students and these concepts have been studied for years and used for teaching and learning in the classroom and the research of factors contributing to students’ HOTS development (Noble and  Powell, 1995; Rajendran, 2001; O’Tuel and Bullard, 2001;  Marshall , Robert and Horton,  2011;  Magno, 2011;  Fischer et al., 2011;  Kondak and Ayden, 2013). Within the thinking process literature, there are many factors affecting HOTS: classroom environment, family charac-teristic, psychological characteristic, and intelligence (Horan, 2007; Silvia, 2008; Pannells and Claxton, 2008; Lim and Smith, 2008; Chini et al., 2009; Pascarella et al., 2013; Fearon et al., 2013; Lather et al. ., 2014). These factors are related and mutually supported. Thus, the aforementioned factors should be included in the teaching model, which will be of benefit in supporting and promoting the development of HOTS. However, despite more than a decade of studies in this area and a variety of models proposed to explain the factors affecting HOTS, the extant factor affecting HOTS literature remains as follows; 1) the lack of systematic integration among those variables, 2) the researcher can not specified all relationships by a theory needed to be included in each primary study 3) some relations that are inconsistent or contradict one another across studies (Montea and Siu, 2002; Brink, 2003;  Cheung and Chan, 2005; Montazemi and Hamed, 2015). It is difficult to draw conclusion from these studies. Hence, it is necessary to have a summary of research findings of the increased studies, variety of concepts on factors affecting HOTS. These will be studied further for a clear conclusion and a similar direction to get the most benefit from the information and to develop the most effective practical application. A systematic meta-analysis is likely to help us solve these problems.

For the research methodology of data collection and conclusion, the researcher employed a research synthesis method to data collection and applied statistical procedures to draw conclusion and solutions of the problems (Light and Pillemer, 1984). During the first stage, the descriptive method, traditional vote-counting methods and commutation of p-values are used for quantitative synthesis. Later, the meta-analysis is used by integrating the effect size to the analysis process to acquire better synthesis results and to delete the disadvantages of traditional synthesis which give subjective  results  (Kulik and Kulik, 1989). More recently, the meta-analytic structural equation modeling approach (MASEM) has been developed for advanced statistics from more complex variance models which give research conclusions in terms of the causal relationship from different research. This also resulted in affirming or denying the theoretical relationship structure. Moreover, it will provide a powerful means for testing broader, richer, and more complex theories that are unlikely to be feasibly tested in any single primary study (Viswesvaran and Ones, 1995; Hunter and Schmitdt, 2004). Additionally, the research results indicated a causal relationship model of both direct and indirect effects in real situations (Bamberg, 2007; Yu and Chiu, 2007). In conclusion, the research indicates that the MASEM is more practical and informative than the traditional meta analytic method.

From the problems and major issues mentioned above, it is obvious that there has not been conclusion from analyzing the factors affecting HOTS by applying MASEM in Thailand. The problems could be caused by the following; 1) unfamiliarity with the analysis method, 2) few applications for education and 3) some factors are overlooked. Moreover, some repeated problems are caused by education systems and policies, which reflect the limitations of research analysis due to the same results provided with no differences, thus the result of research database is not enough to analyze by MASEM (Borenstein et al., 2009). Therefore, the author is aware of the advantage and the application of MASEM and HOTS for research in human and social sciences. Consequently, the educationists use the information to determine what factors directly resulted in HOTS development and what elements relate to them. The outcomes of this analysis will contribute to effective development of students’ HOTS. Additionally, it is also considered to be an extension of the knowledge application of MASEM analysis for the future studies.




Higher order thinking skills; HOTS

The definition of HOTS is the ability and expertise to find answers or achieve target goals through various forms of thinking processes. It is necessary for students to learn and practice this ability in order to acquire answers, to make decisions, and to solve problems (Lewis and Smith. 1993; King et al., n.d.).  Educators have an assortment of HOTS that include several concepts. Krulik and Rudnick (1993) state that HOTS includes 1) recall thinking, 2) basic thinking, 3) critical thinking, and 4) creative thinking. Byrnes (1996) classifies HOTS into 4 levels; 1) the application level, 2) the analysis level, 3) the synthesis level, and 4) the evaluation level. Anderson and Krathwohl (2001) propose the concepts of Bloom’s Taxonomy Revised, and  classify  cognitive  approaches  to  learning into six levels; 1) remembering, 2) understanding, 3) applying, 4) analyzing, 5) evaluating, and 6) creating. Based on the national standards of educational management and basic curriculum of Thailand, the key of these concepts related to HOTS development are the main focus for the development of characteristics in students’ thinking skills. Moreover, they are the variables the author use in this study; 1) Analytical thinking: AnT is the ability of individuals to classify objects logically, assessing the relationships of certain elements, how they contribute, how they relate to each other, how they work, and what the most important parts are (Bloom et al., 1956; Marzano, 2001).  2) Critical Thinkingical Thinking; CriT refers to the ability to evaluate and consider things by searching for reliable and sufficient information before making decisions, solving problems, evaluating situations and taking action on any tasks with the most appropriate and accurate ways (Ennis, 2002; Black and Black, 2006; Ellis, 2009). 3) Creative Thinking; CreT refers to thinking competency in using previous knowledge to create new knowledge for discovering or innovating new things. This often results in more valuable outcomes, which can be used or applied to problem solving or effective performance (Sternberg and Lubart, 1999; Harvey, 2010).

Meta-analytic structural equation modeling; MASEM

Meta-analytic structural equation modeling (MASEM) is the most recently developed quantitative synthesis technique, which combines two research methodologies. Meta – analytic (MA) is the statistical analysis of analysis results from individual studies for the purpose of integrating the findings in form of effect size. Structural equation modeling (SEM) is a technique used to verify or test theoretical causal models (Glass, 1976; Hunter and Schmidt, 2004; Cheung, 2008). For the first phase, meta-analysis was synthesized to draw a conclusion of the effect size as an index of the direction and magnitude of the association between two variables, which includes Pearson correlations (r) and standardized mean difference (g). In order to conclude the effect size of more complex variables, the effect size on a series of correlation matrices is used to create a pooled correlation matrix, which can then be analyzed using SEM (Viswesvaran and Ones, 1995; Shadish, 1996; Cheung and Chan, 2005; Hafdahl, 2009).                               

Landis (2013) states that there are at least two primary approaches that serve as a foundation for integrating MA and SEM. 1) The analysis model proposed by Viswesvaran and Ones (Colquitt et al., 2000; Earnest et al., 2011; Robbins et al., 2009) is applied when no study provided full information of all variables indicated in the models. 2) The two-stage SEM (TSSEM) proposed by Cheung and Chan (2005) is  a  preferable  alternative  for the author to apply when there is at least one study provided full information. In the present paper the author conducted MASEM by following a two-stage procedure of Viswesvaran and Ones (1995), which was considered to be the most suitable method for the data in this study. The concepts of analysis consist of two models that have continuous processes related to each other: the measurement model and the casual model. The five steps of the measurement model for theory testing are 1) identifying important constructs and relationships, 2) identifying different measurements used to operationalize each construct, 3) indicating all relating statistics and all of their importance in of studies, 4) processing the meta- analyzing and estimating the real value of the relationship of the measurement, 5) using factor analysis to test the measurement models. For casual models, there are the processes of measurement as following: 6) estimating the correlation value between structures from different structures, and 7) using path analysis with the estimated true value of correlation to test the proposed theories.

Classroom environment; ClEnv

The previous studies of classroom environment revealed the factors affecting  the environment to enhance the effective teaching and learning processes are learning achievement, desirable characteristics of students,  and processes of skill development including HOTS (Brown and Freeman, 2000; Dorman, 2002; Fisher and Khine, 2006; Wolf and Fraser, 2008; Galton et al., 2009; Pascarella et al., 2013). Even there were results indicating that factors concerning classroom environment were differed and variety, but from the author synthesis  the variables of classroom environment affecting HOTS can be divided  into three factors; 1) Classroom climate; ClCli refers to learning environment for both physical atmosphere such as  tidiness, cleanliness, light, and size, and psychological atmosphere such as safety, warmness and good relationship, and freedom in expressing ideas and feelings (Moos, 1979; Dunn and Dunn, 1992; Brand et al., 2003; Ambrose et al., 2010; Wanekezi and Iruloh, 2012)., 2) Teaching and learning methods; TeM refers  to principles, methods, patterns, and techniques that teachers apply to manage students’ learning and to achieve classroom management goals (Jones et al., 1987; Alberta Learning, 2002)., and 3) Teacher behavior; TeB refers to the actions of teachers in classrooms to motivate, facilitate, and encourage students for performing their efficient works (Dorman, 2009).

Family characteristic; FaCh

Family is a basic social unit where parents insert their love, cares, values, attitudes, and life experiences for students. Therefore, this factor is considered a foundation for every dimension of students’ development as well as an influent element affecting students’ learning outcomes and thinking skills, which showed the individual differences (Jackson, 2003; Wade, 2004; Campbell and Gilmore, 2007). Regarding the previous studies, the results show that there are two major factors of the family characteristic; 1) Democratic parenting style; Dmo refers to the method used by parents to take care of their children informally, but remain the rules with reasonably and democratically acceptances (Baumrind, 1966; Maccoby, 1992; Steinberg, 2001)., 2) parental support; Sup refers to the assistance, support, encouragement, and conveniences provided to children to live and learn including the learning environment to enhance students to gain new experiences and develop more advance skills (Ghate et al., 2000; Patricia et al., 2004).

Psychological characteristic; PsyCh

The psychological characteristic refers to the personality trait or behavioral characteristic which affects the learning strategy and the thinking process of individual to express students’ feelings to contribute to their different learning and thinking skills (Lahey, 2001; Sternberg and Willium, 2001; Woolfolk, 2004; Santrock, 2009). The studies show that the two major factors of psychological characteristic are 1) Attitude toward learning; Atti refers to the student’s ability to show satisfaction, and the agreement and disagreement toward classroom environment, teachers, learning activities, classmates and curriculum (Zimbardo, 1999; Bernstein et al., 2006)., 2) Achievement motivation; Moti refers to students’ willingness, intention, enthusiasm, and attempt to achieve learning objectives with high performance (McClelland, 1961; Woolfolk, 2004)., and 3) Internal locus of control; Loc refers to students’ self-awareness competency in working and achieving the goals, or even when they fail on their tasks, they keep their focus and effort to be successful (Rotter, 1990; Stajkovic and Luthans, 2003).

Intellectual characteristic; IntCh

According to the literature reviews, the findings show that intellectual characteristic also covers intellectual competency, solving problems and reasoning to change learning behavior, and differences of thinking process skills of individuals (Kane et al., 2004; Kim, 2005; Horan, 2007; Silvia, 2008). The results of synthesis show two major factors of intellectual competency, which are 1) Intelligence quotient; IQ refers to competency in learning, solving problems, and adjusting to new environments and problems (Feldman, 1992; Woolfolk, 2004)., 2) Reasoning abilities;  Reas   refers  to  the  ability  in  transferring  previous knowledge to new experiments through thinking processes, solving problems, and finding relationships of things  to make decisions based on the current information and problems (O’Daffer, 1990).        


To develop and assess the validity of a structural equation model of factors affecting HOTS through meta-analytic structural equation modeling.


The research hypotheses are given in Table 1. The theoretical models of the factors affecting HOTS are shown in Figure 1.






To identifying the studies relevant for our MASEM consisted of using the internet search from ThaiLis Digital Collection and the electronic theses online system of 71 higher education institutions of Thailand. The studies are composed of quantitative research, experimental and correlational research, which focus on factors relating to the family characteristic, the intellectual characteristic, the psychological characteristic, and the classroom environment, which affect students’ HOTS. The thinking skills consist of three factors; analytical thinking, critical thinking, and creative thinking published during 1999-2013, which was the period when the Thai educational system was renovated and there was more emphasis on students’ thinking skills development. Search keywords include the following terms: 1) classroom climate, 2) teaching and learning methods, 3) teacher behavior, 4) democratic parenting style,  5) parental support, 6) attitude toward learning, 7) achievement motivation, 8) internal locus of control, 9) intelligence quotient, 10) reasoning abilities, 11) analytical thinking 12) critical thinking, 13) creative thinking and 14) higher order thinking skills. The search initially yielded 300 primary studies from 35 educational institutions matching our keywords. The studies were then examined for inclusion in our study, using the inclusion criteria.

Selecting the studies

Not all the studies were appropriate for inclusion in our analysis. Rosenthal (1995) and Wolfswinkel et al. (2013) recommended that researchers should assess the quality of the primary studies before analyzing the establishing criteria for the inclusion of the primary studies by using a multiple-rater technique to evaluate data from the primary studies, and assessing inter-rater reliability. Therefore, the author processes the research as follows;

Inclusion criteria

The studies would be included in the present meta-analysis if it satisfied the following inclusion criteria. 1) At least two of the constructs included in our hypothetical model were analyzed in the studies. 2) The sample in each primary studies are the students of the government schools  3)  Both  bivariate  Pearson  correlations(r) and sample size were reported in the studies. 4) The sufficient data to compute effect sizes according to Glass’ formula were reported in the studies. Abstracts of these papers were examined in greater detail. After closer inspection of the full papers, only 166 studies from 22 educational institutions satisfied all the above criteria and were retained to create a pooled correlation matrix for the MASEM analysis.

Intercoder reliability

The author examined all collected primary studies and recoded information on each study’s demographic and  substantive  features to ensure the literature search processed reliability (Cooper and Hedges, 1994).

The studies were coded by 3 authors independently, consisting of two research advisers and the author, reaching an intercoder agreement of 95%. The level of agreement reached was highly satisfactory. Disagreements in coding were resolved through discussion for consensus.

Data analysis

Two steps of Viswesvaran and Ones (1995) were employed for this


In step 1; the addition of the pooled correlations matrix based on

Hedges and Olkin (1985) method consisted of three steps; 1.1) Transformed correlation coefficients into a standard normal metric using Fisher's r-to-Z transformation  before calculating a weighted average of these transformed scores in fixed-effects model. (Fisher, 1921; Hedges and Olkin's, 1985) 1.2) Next, we tested the homogeneity of correlations from 1.1. Hedges and Olkin’s Q statistic was applied to test the homogeneity of the correlations for each component. The fixed-effects model is appropriate for calculating the pooled correlation matrix when the heterogeneity tests are insignificant. Whereas, the random-effects model is proper when these tests indicate heterogeneity (Hedges and Vevea, 1998; Hunter and Schmidt, 2004). 1.3) After that, we transformed the weighted average Fisher's Z-to-r correlation for each pair of all variables back to the standard correlational form to the more interpretable effects size for reporting. This resulted in a matrix of meta-analytic correlations between all variables in the hypothetical model. The Comprehensive Meta-Analysis computer program was used to perform the data analysis (Borenstein et al., 2009)

In step 2 of the MASEM, the pooled correlation matrix by the true-score population effect sizes of the variable pairs was subjected to the SEM technique using the Mplus version7. The criteria for assessing the validity of a structural equation model was a very good fit with the empirical data from the primary studies composed of the Comparative fit index; CFI, the Tucker - Lewis index; TLI, the Standardized root mean squared residual; SRMR, and the Root mean squared error of approximation; RMSEA. The goodness of fit statistics from structural validity shows that very good fitting model were CFI and TLI ≥ .95 SRMR and RMSEA ≤ .05 (Mclachlan and Pell, 2000; Muthén and Muthén, 2009; Byrne, 2012). For model sample size, we followed the recommendation of Viswesvaran and Ones (1995) to use the harmonic mean as the appropriate sample size because it tends to yield the least biased estimates of standard errors of parameter estimates.


Description of studies

Studies included in the meta-analysis were highly variable in terms of sample sizes that ranged from 411 to 30,163. The harmonic mean of the sample sizes was 655. For each effect size, the author used the following criteria to assess the effect size magnitudes: small (r< 0.30), moderate (0.30 ≤ r < 0.50), and large (r ≥ 0.50) (Cohen, 1988).

Among the 78 average weighted correlations obtained in the fixed effect model varied from small to large (0.060 to 0.669); a majority of correlations (40 out of 78) was the moderate, 25 correlations was the small, and 13 correlations was the large. Lower- and upper- bound effect sizes for confidence intervals of fixed effect model ranged from -0.003-0.683.  In the random effect model, the effect size varied from small to large (0.060-0.576), a majority of correlations (48) was the moderate, 26 correlations was the small, and 4 correlations was the large. Lower- and upper- bound effect sizes for confidence intervals of random effect model ranged  from -0.003-1.958 (show in Appendix A).

Results of the validity of a structural equation model of factors affecting HOTS 

According to the pooled correlation matrix of a structural equation model of factors affecting HOTS consisted of 78 effect sizes in the matrix (show in Appendix B). The result of the initial path analysis showed that the model was a very good fit with the empirical data from the primary studies with   = 0.035, df = 5, p-value = 1.000, TLI = 1.025, CFI = 1.000, SRMR = 0.001, RMSEA = 0.000 (Table 2).



Results of the validity of a structural equation model of factors affecting HOTS are shown in Figure 2.                

In accordance with Table 2 and Figure 2, the direction of effects is summarized as follows:

Direct effect factors are as follows.



The finding showed that three-fourths of the path of factors directly affecting HOTS significantly affected HOTS. The psychological characteristic (H7: 0.762**) indicated a higher effect size than the classroom environment (H1: 0.380*) and double in the intellectual characteristic (H8: 0.363*). The three latent factors explain the variance of 96.8%. However, the family characteristic insignificantly affected HOTS. Therefore, the study of the contribution of the psychological characteristic will enhance students’ HOTS more than the classroom environment, and double in the intellectual characteristic. If we compare the results with the family characteristic, the findings indicated a 7 times higher development in students’ HOTS.

The classroom environment (H2: 0.521**) had significant direct effects on the psychological characteristics equal to the family characteristic (H5: 0.414**). Therefore, in order to study or research, the development of students’ psychological characteristic must focus on the enhancement of the classroom environment and the family characteristic. Even though the effect size of variable in the classroom environment was higher, the result indicated that the variable of the psychological characteristic must be equally focused on.

The factors that directly affected the intellectual characteristic consisted of the classroom environment (H3 : 0.457**), which showed a higher value more than twice of the family characteristic (H6 : 0.208*). It can be conclude that, study in the contribution of the classroom environment will enhance students’ intellectual characteristic more than twice of the family characteristic.

The four paths of indirect factors affecting HOTS are as follows.

The family characteristic indirectly affected HOTS through the  psychological  characteristics of  students  (0.315**).

Example of the value of indirect effects (0.414×0.762 = 0.315) was developed by two paths. 1) The family characteristic  had  a  direct  effect  on  the  psychological characteristic of students (H4: 0.414**). (2) The psychological characteristics of students directly affected HOTS (H7:0.762**).  In conclusion, the family characteristic affects the improvement of the psychological characteristics and will enhance students’ HOTS.

The family characteristic had insignificant indirect effects on the intellectual characteristic. The study indicates that study or research in the contribution of the family characteristic affects the improvement of the intellectual characteristic will not enhance students’ HOTS.

The classroom environment indirectly affected HOTS through the psychological characteristics of students (0.397**). It is concluded that the classroom environment positively affects the psychological characteristics and also increases students’ HOTS.

The classroom environment indirectly affected HOTS through the intellectual characteristic (0.166**). In conclusion, the study shows that the classroom environment positively affects the intellectual characteristic and enhances students’ HOTS.


This study brings conceptual and empirical clarity to the factors affecting HOTS based on the MASEM method. Our study makes four major contributions to theory, as follows.

The psychological characteristic, the classroom environment, and the intellectual characteristic of students directly affect HOTS, which supports the hypothesis. However, the family characteristic insignificantly affects HOTS. The result may be caused by the development of the hypothesis, which determined only direct effects. When this path included in the model had a variety complex variables, it did not support the hypothesis (Ali and Hamed, 2015). The psychological characteristic had effects on HOTS more than any other. This may be explained because  these variables can be continuously developed by various techniques of the learning process, the classroom environment, and parents’ support (Hoang, 2007; Dorman, 2009; Baeten et al., 2013; Umo, 2013).

The classroom environment and the family characteristic directly affect the intellectual characteristic, which supports the hypothesis. Morris and Maisto (2002) assert that the elements of social environment affect the intellectual characteristic. The classroom environment and the family condition are considered to be parts of the social environment. Additionally, this study found that the classroom environment has more than twice an effect of the family characteristic. This may result from the classroom environment and learning management that aim to encourage students to learn and develop their intellectual characteristic. Moreover, classroom management can design situations or experiments practice students thinking skills in various ways during the period of learning (School Drug Education and Road Aware, 2013).

The psychological characteristic of students is an important mediator variable for HOTS. The study indicates two indirect   effects   through    students’   psychological characteristic: the classroom environment and the family characteristic. The results may be caused by the attribute of SEM analysis that able to analyze various effects including direct effect, moderating effect, and reverse effect. These also allow to identify linear and additive relationships of recursive and non- recursive model, as well as indirect effect through the mediator variable (Schumacker and Lomax, 2010; Barbara, 2012). The result of this research shows that the classroom environment has more an indirect effect than the family characteristic. The results may be caused by the effective instructional management, which benefits in organizing classroom environment to support the feelings, attitudes, knowledge, and thinking skills of students (Nelson and Debacker, 2008; Chini et al., 2009; Pascarella et al., 2013).

The family characteristic had insignificant indirect effects on the intellectual characteristic. The results may because the intellectual characteristic had several effects on both the classroom environment and the family characteristic. When this path included the complex model, it did not support the hypothesis. However, the family characteristic is also an important factor that is indirectly affected  through the psychological characteristic and increases students’ HOTS.


Within the organizational literature, the study of factors affecting HOTS has been conducted for many years. Researchers have chosen to study a variety of variables and proposed a variety of models based on their individual interests, and there is no systematic integration among them. Moreover, some research findings are inconsistent with other studies and have become difficult to draw conclusions from the literature reviews. To solve these problems, this study collects the variables affecting HOTS to synthesize and find the conclusion for MASEM. This research contributes systematic integration among the variables. The research findings confirm the concepts, theories and importance of factors based on the structural equation model of factors affecting HOTS. The model is systematically designed from various concepts and theories of HOTS, which makes powerful and strong results and gains a boarder conclusion than the conclusions of one single primary study (Hunter and Schmidt, 2004). Moreover, this study extends the concept of research synthesis by using advanced statistics to behavioral and social sciences study.
On the basis of the results of this study, we have several suggestions for future research and practical applications;
1. The findings indicate that the classroom climate, teaching and learning methods affect HOTS  of  students.
Additionally, the psychological factor should be considered and applied to the classroom environment. For example, with the positive learning activity management, the classroom climate should support positive thinking in learning, teaching behavior or personalities of teachers to support an attitude toward learning. Techniques to motivate students to learn and express ideas can enhance HOTS of student as well.
2. The democratic parenting style and support of the family will help students improve their attitudes towards learning, achievement motivation, and self-trust, which will affect HOTS. Therefore, the parents should take care of their children closely and fairly, and provide students with an opportunity to share their ideas, make decisions, and solve problems. Additionally, the parents should encourage their children to participate in activities in and out of the classroom.
3. There are many studies on social science, which contain various models and variables. It is possible to synthesize those variables and make conclusions. This strategy extends the limit of the study of some variables in social science study. However, even this research got the conclusion. Future research may apply this information for research and development in practice such as development of learning strategy for contributing to students’ HOTS. It is possible to apply the MASEM, which will not only develop students’ HOTS, but also to extend the area of the MASEM study. Additionally, this form of learning management, from research synthesis with the advance statistical method, will add more value to future research.
4. The results of Q statistics indicated that the hetero-geneity among effect size across studies (Hedges and Olkin, 1985). Therefore, future research should investigate the sources of this heterogeneity through moderator analyses, and the limitation of the process must be considered on the possibility of sufficient (Lipsey, 1994; Card, 2012).


The authors have not declared any conflicts of interest.


Akinsola EF (2011). Relationship Between Parenting Style, Family Type, Personality Dispositions and Academic Achievement of Young People in Nigeria. Ife PsychologI A; 19(2):246-267.
Alberta Learning (2002). Instructional Strategies. Edmonton, AB: Alberta Learning.
Ali RM, Hamed QS (2015). Factors affecting adoption of online banking: A meta-analytic structural equation modeling study. Inform. Manage. 52:210-226.
Amabile TN, Goldfarb P, Brackfield SC (1990). Social influences on creativity: Evaluation, co-action and surveillance. Creativity Res. J. 3:6-21.
Ambrose SA, Bridges MW, DiPietro M, Lovett MC, Norman MK (2010). How Learning Works: Seven Research-Based Principles for Smart Teaching. San Francisco, CA: Jossey-Bass.
Anderson LW, Krathwohl, DR (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Addison Wesley Logman.
Ari RB, Eliassy L (2003). The Differential Effects Of The Learning Environment On Student Achievement Motivation: A Comparison Between Frontal And Complex Instruction Strategies. Soc. Behav. Pers. 31(2):143-166.
Baeten M, Dochy F, Struyven K (2013). The effects of different learning environments on students' motivation for learning and their achievement. Br. J. Educ. Psychol. 83:484-501.
Bamberg S (2007). Twenty years after Hines, Hungerford, and Tomera: A new meta-analysis of psycho-social determinants of pro-environmental behavior. J. Environ. Psychol. 27:14-25.
Barbara, MB (2012). Structural equation modeling with Mplus: basic concepts, application, and programming. New York: Taylor and Francis Group.
Barkl S, Porter A, Ginns P (2012). Cognitive Training for Children: Effects on Inductive Reasoning, Deductive Reasoning, and Mathematics Achievement in an Australian School Setting. Psychol. Schools 49(9):828-842.
Baumrind D (1966). Effects of authoritative control on child behavior. Child Dev. 37:887-907.
Bernstein DA, Penner LA, ClarkeSA, Roy EJ (2006). Psychology (7th ed) Boston M.A. Houghton Mifflin Company.
Black H, Black S (2006). Teacher Manual and Lesson Plans: Building Thinking Skills. California: Midwest Publication: Critical Thinking Press and Software.
Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company.
Blumenfeld PC, Pintrich PR, Hamilton VL (1987). Teacher Talk and Students Reasoning about Morals, Conventions, and Achievement. Child Dev. 58:1389-1401.
Bong M (2005). Within-grade changes in Korean girls' motivation and perceptions of the learning environment across domains and achievement levels. J. Educ. Psychol. 97:656-672.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009). Introduction to meta-analysis. Chichester: Wiley edition. John Wiley & Sons.
Brand S, Felner R, Shim M, Seitsinger A, Dumas T (2003). Middle school improvement and reform: Development and validation of a school-level assessment of climate, cultural pluralism, and school safety. J. Educ. Psychol. 95(3): 570-588.
Brink NH (2003). Locus of control, creativity in late middle childhood. Retrived March 16, 2015 From
Brown MN, Freeman K (2000). Distinguishing Features of Critical Thinking Classrooms. Teach. Higher Educ. 5(3):301-309.
Byrne BM (2012). Structural Equation Modeling with Mplus: Basic Concept, Applications and Programming. New York: Taylor and Francis Group.
Byrnes JP (1996). Cognitive development and learning in instructional context. Boston, MA: Allynand Bacon.
Campbell J, Gilmore L (2007).Intergenerational continuities and discontinuities in parenting styles. Austr. J. Psychol. 59(3):140-150.
Card NA (2012). Applied meta-analysis for social science research. NY: Guilford.
Cheung MWL, Chan W (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychol. Methods 10(1):40-64.
Cheung MWL. (2008). A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychol. Methods 13(3):182-202.
Chini JC, Carmichael A, Rebello NS, Puntambekar S (2009). Does the Teaching Learning Interview Provide an Accurate Snapshot of Classroom Learning?. Proceedings of the 2009 Physics Education Research Conference, AIP Publications, July 29-30.
Cohen J (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Colquitt JA, LePine JA, Noe RA (2000). Toward an integrative theory of training motivation: A meta- analytic path analysis of 20 years of research. J. Appl. Psychol. 85:678-707.
Cooper HM, Hedges LV (1994). The handbook of research synthesis. New York: Russel Sage.
Dombusch SM, Ritter PL, Leiderman P, Roberts DF, Fraleigh, MJ (1987). The Relation of Parenting Style to Adolescent School Performance. Child Development, 58:1244-1257.
Dorman JP (2002). Classroom environment research: Progress and possibilities. Queensland J. Educ. Res. 18:112-140.
Dorman JP (2009). Associations between psychosocial environment and outcomes in technology- rich classrooms in Australian secondary schools. Res. Educ. 82(1):69-84.
Dunn R, Dunn K (1992a). Teaching elementary student through their individual learning styles. Boston: Allynand Bacon.
Earnest DR, Allen DG, Landis RS (2011). A meta-analytic path analysis of the mechanisms linking realistic job previews and turnover. Pers. Psychol. 64: 865-897.
Ellis D (2009). Critical Thinking. IL: Houghton Mifflin Company.
Ennis RH (2002). A Super-Streamlined Conception of Critical Thinking. California: Critical Thinking Press and Software.
Fearon DD, Copeland D, Saxon TF (2013). The Relationship Between Parenting Styles and Creativity in a Sample of Jamaican Children. Creativity Res. J. 25(1): 119-128.
Feldman RS (1992). Essentials of Understanding Psychology. New York: McGraw-Hill.
Fisher RA (1921). On the probable error of a coefficient of correlation deduced from a small sample. Metron, 1:3-32.
Fischer C, Bol L, Pribesh S (2011). An Investigation of Higher-Order Thinking Skills in Smaller Learning Community Social Studies Classrooms. Am. Secondary Educ. 39(2):5-25.
Fisher DL, Khine MS (2006). Contemporary Approaches to Research on Learning Environments, Singapore: World Scientific.
Fleith DS (2000). Teacher and student perceptions of creativity in the classroom environment. Roeper Rev. 22(2):148-158.
Galton M, Hargreaves L, Pell T (2009). Group work and whole-class teaching with 11 to14 year olds compared. Cambridge J. Educ. 39(1):119-140.
Ghate D, Shaw C, Hazel N (2000) Fathers and family centers: engaging fathers in preventive services. York: Joseph Rowntree Foundation/York Publishing Services.
Ginsburg G, Bronstein P (1993). Family factors related to children's intrinsic/extrinsic motivational orientation and academic performance. Child Dev. 64:1461-1471.
Glass GV (1976). Primary, secondary, and meta-analysis of research. Educ. Res. 5:3-8.
Gottfried AE, Fleming J, Gottfried AW (1994). Role of parental motivational practices in children's academic intrinsic motivation and achievement. J. Educ. Psychol. 86:104-113.
Hafdahl AR (2009). Improved Fisher z estimators for univariate random-effects meta-Analysis of correlations. Bri. J. Math. Stat. 62:233-261.
Harvey D (2010). The Enigima of Capital and the Crises of Capitalism. London: Profile Books.
Hedges LV, Olkin I (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
Hedges LV, Vevea JL (1998). Fixed-and random-effects models in meta-analysis. Psychol. Methods 3:486-504.
Hoang TN (2007). The relations between parenting and adolescent motivation. Int. J. Whole Schooling, 3(2):1-21.
Horan R (2007). The Relationship Between Creativity and Intelligence: A Combined Yogic- Scientific Approach. Creativity Res. J. 19(2–3): 179-202.
Houtenville AJ, Conway KS (2008). Parental Effort, School Resources and Student Achievement. J. Hum. Resour. 43(2):437-453.
Hunter J, Schmidt F (2004). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Newbury Park, CA: Sage.
Jackson AP (2003). The effects of family and neighborhood characteristics on the behavioral and cognitive development of poor black children: A longitudinal study. Am. J. Commun. Psychol. 32:175-186.
Jones BF, Palincsar AS, Ogle DS, Carr C (1987). Strategic Teaching and Learning: Cognitive Instruction in the Content Areas. Alexandria, Virginia: Association for Supervision and Curriculum Development.
Kane MJ, Hambrick DZ, Tuholski SW, Wilhelm O, Payne TW, Engle RW (2004). The generality of working memory capacity: A latent-variable approach to verbal and visuospatial memory span and reasoning. J. Exp. Psychol. General,133:189-217.
Kellan NC (2000). The Basic principles of psychology. New York: Holt Donil.
Kim KH (2005). Can Only Intelligent People Be Creative? A Meta-Analysis. J. Secondary Gifted Educ. 16(2/3):57-66.
King FJ, Goodson L, Rohani F (n.d.). Higher order thinking skills. Center for Advancement of Learning and Assessment. Retrived March 7, 2015 From:
Kondak EU, Ayden YC (2013). Predicting Critical Thinking Skills of University Students through Meta cognitive Self-Regulation Skills and Chemistry Self-Efficacy. Educational Sciences: Theory Pract. 13(1):666-670.
Krulik S, Rudnick AJ (1993). Reasoning and Problem Solving: A Handbook for elementary school teachers. USA: Allyn and Bacon A Division of Simon and Schuster, Inc.
Kulik JA, Kulik CLC (1989). Meta-analysis in educational research. Int. J. Educ. Res. 13: 221-340.
Lahey BB (2001). Psychology: An Introduction. 7th Edition. Boston: McGraw-Hill.
Landis RS (2013). Successfully combining meta-analysis and structural equation modeling: Recommendations and strategies. J. Bus. Psychol. 28:251-261.
Lather AS, Jain S, Shukla AD (2014). Student's Creativity in Relation to Locus of Control: a Study of Mysore University, India. Int. J. Indian Psychol. 2(1): 146-165.
Lee SM, Daniels MH, Kissinger DB (2006). Parental influences on adolescent adjustment: Parenting styles versus parenting practices. Family J. 14: 253-259.
Lewis A, Smith D (1993). Defining Higher Order Thinking. Theory Pract. 32(3):131-137.
Light RJ, Pillemer DB (1984). Summing up: The Science of Reviewing Research. Cambridge, Massachusetts: Harvard University Press.
Lim S, Smith J (2008). The Structural Relationships of Parenting Style, Creative Personality, and Loneliness. Creativity Res. J. 20(4):412-419.
Lipsey MW (1994). Identifying Potentially Interesting Variables and Analysis Opportunities, in Handbook of Research Synthesis, H. Cooper and L. V. Hedges (eds.), Russell Sage Foundation, New York.
Maccoby EE (1992). The role of parents in the socialization of children: An historical overview. Dev. Psychol. 28:1006-1017.
Magno C (2011). Assessing the Relationship of Scientific Thinking, Self-regulationin Research, and Creativity in a Measurement Model. Int. J. Res. Rev. 6(1):22-47.
Marshall JC, Robert M, Horton RM (2011). The Relationship of Teacher - Facilitated, Inquiry-Based Instruction to Student Higher-Order Thinking. School Science and Mathematics.
Marzano RJ (2001). Designing a New Taxonomy of Educational Objective. Thousand Oaks. California: Corwin Press, Inc.
McClelland DC (1961). The Achieving Society. New York: The Free Press.
McGinn LK, Cukor D, Sanderson WC (2005). The Relationship Between Parenting Style, Cognitive Style, and Anxiety and Depression: Does Increased Early Adversity Influence Symptom Severity Through the Mediating Role of Cognitive Style?. Cogn. Ther. Res. 29(2):219-242.
Mclachlan GJ, Pell D (2000). Finite Mixture Models. New York: John Wiley.
Mednick MT, Andrews FM (1967). Creative thinking and level of intelligence. J. Creative Behav. 1:428-431.
Miller BC, Gerard D (1979). Family influences on the development of creativity in children: An integrative review. Fam. Coordinator 28:295-312.
Moneta GB, Siu CMY (2002). Trait intrinsic and extrinsic motivations, academic performance, and creativity in Hong Kong college students. J. College Students Dev. 43:664-683.
Montazemi AR, Hamed SHQ (2015). Factors affecting adoption of online banking: A meta-analytic structural equation modeling study. Inform. Manage. 52:210-226.
Moos RH (1979). Evaluating educational environments. Washington: Jossey-Bass Publisher.
Morris CG, Maisto AA (2002). Psychology and Introduction. New Jersey: Upper Soldle River.
Muthén LK, Muthén BO (2009). Mplus User's Guide, Statistical Analysis With Latent Variables. 6rd ed. Los Angeles, CA: Muthén and Muthén.
Nelson RM, Debacker TK (2008). Achievement Motivation in Adolescents: The Role of Peer Climate and Best Friends. J. Exp. Educ. 76(2): 170-189.
Noble J, Powell DA (1995).Factors Influencing Differential Achievement of Higher-order Thinking Skills, as Measured by PLAN. ACT Research Report Series, 95-4. O' Daffer PG (1990). Inductive and deductive reasoning. In The mathematics teacher. Edited byNational Council of Teachers of Mathematics. Michigan: AMS Reprint Co.
O'Tuel FS, Bullard RK (2001). Developing Higher Order Thinking in the Content Areas K-12. USA, CA; Critical Thinking Press and Software.
Pannells TC, Claxton AF (2008). Happiness, Creative Ideation, and Locus of Control. Creativity Res. J. 20(1): 67-71.
Pascarella ET, Wang JS, Trolian TL, Blaich C (2013). How the instructional and learning environments of liberal arts colleges enhance cognitive development. Higher Educ. 66:569-583.
Patricia M, Deborah G, Amelia M (2004). What Works in Parenting Support? A Review of the International Evidence. Research report's the Department for Education and Skills, Policy Research Bureau.
Patrick H, Ryan A, Kaplan A (2007). Early adolescents' perceptions of the classroom social environment, motivational beliefs, and engagement. J. Educ. Psychol. 99:83-98.
Plucker JA, Renzulli JS (1999). Psychometric approaches to the study of human creativity. In: R. J. Sternberg (Ed.), Handbook of creativity. New York: Cambridge University Press.
Querido JG, Warner TD, Eyberg SM (2002). Parenting styles and child behavior in African American families of preschool children. J. Clin. Child Psychiatry 31:272-277.
Rajendran N (2001). The Teaching of Higher-Order Thinking Skills in Malaysia. J. Southeast Asian Educ. 2(1):42-46.
Richmond BO, Serna MDL (1980). Creativity and locus of control among Mexican college students. Psychol. Rep. 46:979-983.
Robbins SB, Oh I, Le H, Button C (2009). Intervention effects on college performance as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. J. Appl. Psychol. 94:1163-1184.
Rosenthal R (1995). Writing meta-analytic reviews. Psychol. Bull. 118:183-191.
Rotter JB (1990). Internal versus eternal control of reinforcement: A case history of a variable. Am. Psychol. 45(4):489-493.
Santrock JW (2009). Educational psychology. New York, NY: McGraw Hill.
School Drug Education and Road Aware (2013). Teaching and Learning Strategies. School Drug Education and Road Aware. Challenges and choices: resilience, drug and road safety education.Western Australia
Schumacker RE, Lomax RG (2010). A beginner's guide to structural equation modeling. (3rd Edition). New Jersey: Lawrence Erlbaum Associates.
Shadish WR (1996). Meta-analysis and the exploration of causal mediating process: A primer of examples, methods, and issues. Psychological Methods 1:47-65.
Shari T, Eileen J, David P (1993). Teaching Thinking Dispositions: From transmission to enculturation. Theory into Practice. 32(3): 147-153.
Shield B, Dockrell JE (2008). The effects of environmental and classroom noise on the academicattainments of primary school children. J. Acoustical Soc. Am. 123(1):133-144.
Silvia PJ (2008). Creativity and Intelligence Revisited: A Latent Variable Analysis of Wallach and Kogan (1965). Creativity Res. J. 20(1):34-39.
Smith F (1992). To think: In language, learning, and education. London: Routledge.
Stajkovic AD, Luthans F (2003). "Social cognitive theory and self-efficacy: Implications for motivation theory and practice" In L.W. Porter, G.A. Bigley and R.M. Steers (eds.), Motivation and work behavior. Boston: McGraw-Hill.
Steinberg L (2001). We know some things: Parent–adolescent relationships in retrospect and prospect. J. Res. Adolesc. 11:1-19.
Sternberg RJ, Lubart TI (1999). The concept of creativity: Prospects and paradigms. In R. J. Sternberg (Ed.), Handbook of creativity. Cambridge: Cambridge University Press.
Sternberg RJ, O'Hara LA (1999). Creativity and intelligence. In: R. J. Sternberg (Ed.), Handbook of creativity. New York: Cambridge University Press.
Sternberg RJ, Williams WM (2001). Educational psychology. Boston, USA: Allynand Bacon.
Torrance EP (1965). Rewarding creative behavior. Englewood Cliffs,NJ: Prentice Hall.
Umo UA (2013). Indiscipline, Parenting Style and Attitude to Learning of Students in Secondary Schools in Uyo Local Government Area of AkwaIbom State, Nigeria. J. Educ. Practice, 4(15):87-92.
Viswesvaran C, Ones DS (1995). Theory testing: Combining psychometric meta-analysis and structural modeling. Pers. Psychol. 48:865-885.
Wade SM (2004) Parenting influences on intellectual development and educational achievement, in: M. Houghughi and N. Long (Eds) Handbook of parenting. Theory and research for practice (London, Sage).
Wanekezi AU, Iruloh BR (2012). Assessing Learning Environment for Achieving Standard in Primary Education: Implication for Counseling for Human Capacity Development. An International Multidisciplinary J. 6(2):274-289.
Wang H (2014). The Relationship Between Parenting Styles and Academic and Behavioral Adjustment Among Urban Chinese Adolescents. Chinese Sociol. Rev. 46(4):19-40.
Wolf SJ, Fraser BJ (2008). Learning environment, attitudes and achievement among middle-school science students using inquiry-based laboratory activities. Res. Sci. Educ. 38:321-341.
Wolfswinkel JF, Furtmueller E, Wilderom CPM (2013). Using grounded theory as a method for rigorously reviewing literature. Euro. J. Inform. Syst. 22(1):45-55.
Woolfolk AE (2004). Educational psychology.9th ed. Boston: Pearson Education, Inc.
Yu L, Chiu CH (2007). Testing a Model of Stress and Health Using Meta-Analytic Path Analysis. J. Nurs. Res. 15(3):202-214.
Zimbardo PG (1999). Psychology, third ed. Addison Wesley Publishing Co., Reading, MA.
Zohar A (1994). Teaching a thinking strategy: Transfer cross domains and self learning versus class-like setting. Appl. Cogniti. Psychol. 8:549-563.
Zohar A, Dori YJ (2003). Higher Order Thinking Skills and Low Achieving Students: Are they Mutually Exclusive? J. Learn. Sci. 12:145-182.