Higher Order Thinking Skills (HOTS) is a thinking process, which consists of complicated procedures and needs to be based on various skills such as analysis, synthesis, comparison, inference, interpretation, assessment,  and   inductive   and   deductive   reasoning  to  be employed to solve unfamiliar problems (Smith, 1992; Zohar and Dori, 2003). The characteristics of students with HOTS are open-mindedness for risk-taking, curiosity, keen on fact discovery, planning and indicating the most suitable method, have a systems thinking process, think carefully, use evidence to think rationally and frequent self-monitoring (Shari et al., 1993). The students with HOTS are able to create new knowledge and make appropriate and logical decisions. Information and technology advancement greatly influences the current society. Consequently, learning management must be adapted to the current situation/society and focus on improving HOTS of students.

There are many concepts of HOTS applied to the educational development of students and these concepts have been studied for years and used for teaching and learning in the classroom and the research of factors contributing to students’ HOTS development (Noble and  Powell, 1995; Rajendran, 2001; O’Tuel and Bullard, 2001;  Marshall , Robert and Horton,  2011;  Magno, 2011;  Fischer et al., 2011;  Kondak and Ayden, 2013). Within the thinking process literature, there are many factors affecting HOTS: classroom environment, family charac-teristic, psychological characteristic, and intelligence (Horan, 2007; Silvia, 2008; Pannells and Claxton, 2008; Lim and Smith, 2008; Chini et al., 2009; Pascarella et al., 2013; Fearon et al., 2013; Lather et al. ., 2014). These factors are related and mutually supported. Thus, the aforementioned factors should be included in the teaching model, which will be of benefit in supporting and promoting the development of HOTS. However, despite more than a decade of studies in this area and a variety of models proposed to explain the factors affecting HOTS, the extant factor affecting HOTS literature remains as follows; 1) the lack of systematic integration among those variables, 2) the researcher can not specified all relationships by a theory needed to be included in each primary study 3) some relations that are inconsistent or contradict one another across studies (Montea and Siu, 2002; Brink, 2003;  Cheung and Chan, 2005; Montazemi and Hamed, 2015). It is difficult to draw conclusion from these studies. Hence, it is necessary to have a summary of research findings of the increased studies, variety of concepts on factors affecting HOTS. These will be studied further for a clear conclusion and a similar direction to get the most benefit from the information and to develop the most effective practical application. A systematic meta-analysis is likely to help us solve these problems.

For the research methodology of data collection and conclusion, the researcher employed a research synthesis method to data collection and applied statistical procedures to draw conclusion and solutions of the problems (Light and Pillemer, 1984). During the first stage, the descriptive method, traditional vote-counting methods and commutation of p-values are used for quantitative synthesis. Later, the meta-analysis is used by integrating the effect size to the analysis process to acquire better synthesis results and to delete the disadvantages of traditional synthesis which give subjective  results  (Kulik and Kulik, 1989). More recently, the meta-analytic structural equation modeling approach (MASEM) has been developed for advanced statistics from more complex variance models which give research conclusions in terms of the causal relationship from different research. This also resulted in affirming or denying the theoretical relationship structure. Moreover, it will provide a powerful means for testing broader, richer, and more complex theories that are unlikely to be feasibly tested in any single primary study (Viswesvaran and Ones, 1995; Hunter and Schmitdt, 2004). Additionally, the research results indicated a causal relationship model of both direct and indirect effects in real situations (Bamberg, 2007; Yu and Chiu, 2007). In conclusion, the research indicates that the MASEM is more practical and informative than the traditional meta analytic method.

From the problems and major issues mentioned above, it is obvious that there has not been conclusion from analyzing the factors affecting HOTS by applying MASEM in Thailand. The problems could be caused by the following; 1) unfamiliarity with the analysis method, 2) few applications for education and 3) some factors are overlooked. Moreover, some repeated problems are caused by education systems and policies, which reflect the limitations of research analysis due to the same results provided with no differences, thus the result of research database is not enough to analyze by MASEM (Borenstein et al., 2009). Therefore, the author is aware of the advantage and the application of MASEM and HOTS for research in human and social sciences. Consequently, the educationists use the information to determine what factors directly resulted in HOTS development and what elements relate to them. The outcomes of this analysis will contribute to effective development of students’ HOTS. Additionally, it is also considered to be an extension of the knowledge application of MASEM analysis for the future studies.

Higher order thinking skills; HOTS

The definition of HOTS is the ability and expertise to find answers or achieve target goals through various forms of thinking processes. It is necessary for students to learn and practice this ability in order to acquire answers, to make decisions, and to solve problems (Lewis and Smith. 1993; King et al., n.d.).  Educators have an assortment of HOTS that include several concepts. Krulik and Rudnick (1993) state that HOTS includes 1) recall thinking, 2) basic thinking, 3) critical thinking, and 4) creative thinking. Byrnes (1996) classifies HOTS into 4 levels; 1) the application level, 2) the analysis level, 3) the synthesis level, and 4) the evaluation level. Anderson and Krathwohl (2001) propose the concepts of Bloom’s Taxonomy Revised, and  classify  cognitive  approaches  to  learning into six levels; 1) remembering, 2) understanding, 3) applying, 4) analyzing, 5) evaluating, and 6) creating. Based on the national standards of educational management and basic curriculum of Thailand, the key of these concepts related to HOTS development are the main focus for the development of characteristics in students’ thinking skills. Moreover, they are the variables the author use in this study; 1) Analytical thinking: AnT is the ability of individuals to classify objects logically, assessing the relationships of certain elements, how they contribute, how they relate to each other, how they work, and what the most important parts are (Bloom et al., 1956; Marzano, 2001).  2) Critical Thinkingical Thinking; CriT refers to the ability to evaluate and consider things by searching for reliable and sufficient information before making decisions, solving problems, evaluating situations and taking action on any tasks with the most appropriate and accurate ways (Ennis, 2002; Black and Black, 2006; Ellis, 2009). 3) Creative Thinking; CreT refers to thinking competency in using previous knowledge to create new knowledge for discovering or innovating new things. This often results in more valuable outcomes, which can be used or applied to problem solving or effective performance (Sternberg and Lubart, 1999; Harvey, 2010).

Meta-analytic structural equation modeling; MASEM

Meta-analytic structural equation modeling (MASEM) is the most recently developed quantitative synthesis technique, which combines two research methodologies. Meta – analytic (MA) is the statistical analysis of analysis results from individual studies for the purpose of integrating the findings in form of effect size. Structural equation modeling (SEM) is a technique used to verify or test theoretical causal models (Glass, 1976; Hunter and Schmidt, 2004; Cheung, 2008). For the first phase, meta-analysis was synthesized to draw a conclusion of the effect size as an index of the direction and magnitude of the association between two variables, which includes Pearson correlations (r) and standardized mean difference (g). In order to conclude the effect size of more complex variables, the effect size on a series of correlation matrices is used to create a pooled correlation matrix, which can then be analyzed using SEM (Viswesvaran and Ones, 1995; Shadish, 1996; Cheung and Chan, 2005; Hafdahl, 2009).                               

Landis (2013) states that there are at least two primary approaches that serve as a foundation for integrating MA and SEM. 1) The analysis model proposed by Viswesvaran and Ones (Colquitt et al., 2000; Earnest et al., 2011; Robbins et al., 2009) is applied when no study provided full information of all variables indicated in the models. 2) The two-stage SEM (TSSEM) proposed by Cheung and Chan (2005) is  a  preferable  alternative  for the author to apply when there is at least one study provided full information. In the present paper the author conducted MASEM by following a two-stage procedure of Viswesvaran and Ones (1995), which was considered to be the most suitable method for the data in this study. The concepts of analysis consist of two models that have continuous processes related to each other: the measurement model and the casual model. The five steps of the measurement model for theory testing are 1) identifying important constructs and relationships, 2) identifying different measurements used to operationalize each construct, 3) indicating all relating statistics and all of their importance in of studies, 4) processing the meta- analyzing and estimating the real value of the relationship of the measurement, 5) using factor analysis to test the measurement models. For casual models, there are the processes of measurement as following: 6) estimating the correlation value between structures from different structures, and 7) using path analysis with the estimated true value of correlation to test the proposed theories.

Classroom environment; ClEnv

The previous studies of classroom environment revealed the factors affecting  the environment to enhance the effective teaching and learning processes are learning achievement, desirable characteristics of students,  and processes of skill development including HOTS (Brown and Freeman, 2000; Dorman, 2002; Fisher and Khine, 2006; Wolf and Fraser, 2008; Galton et al., 2009; Pascarella et al., 2013). Even there were results indicating that factors concerning classroom environment were differed and variety, but from the author synthesis  the variables of classroom environment affecting HOTS can be divided  into three factors; 1) Classroom climate; ClCli refers to learning environment for both physical atmosphere such as  tidiness, cleanliness, light, and size, and psychological atmosphere such as safety, warmness and good relationship, and freedom in expressing ideas and feelings (Moos, 1979; Dunn and Dunn, 1992; Brand et al., 2003; Ambrose et al., 2010; Wanekezi and Iruloh, 2012)., 2) Teaching and learning methods; TeM refers  to principles, methods, patterns, and techniques that teachers apply to manage students’ learning and to achieve classroom management goals (Jones et al., 1987; Alberta Learning, 2002)., and 3) Teacher behavior; TeB refers to the actions of teachers in classrooms to motivate, facilitate, and encourage students for performing their efficient works (Dorman, 2009).

Family characteristic; FaCh

Family is a basic social unit where parents insert their love, cares, values, attitudes, and life experiences for students. Therefore, this factor is considered a foundation for every dimension of students’ development as well as an influent element affecting students’ learning outcomes and thinking skills, which showed the individual differences (Jackson, 2003; Wade, 2004; Campbell and Gilmore, 2007). Regarding the previous studies, the results show that there are two major factors of the family characteristic; 1) Democratic parenting style; Dmo refers to the method used by parents to take care of their children informally, but remain the rules with reasonably and democratically acceptances (Baumrind, 1966; Maccoby, 1992; Steinberg, 2001)., 2) parental support; Sup refers to the assistance, support, encouragement, and conveniences provided to children to live and learn including the learning environment to enhance students to gain new experiences and develop more advance skills (Ghate et al., 2000; Patricia et al., 2004).

Psychological characteristic; PsyCh

The psychological characteristic refers to the personality trait or behavioral characteristic which affects the learning strategy and the thinking process of individual to express students’ feelings to contribute to their different learning and thinking skills (Lahey, 2001; Sternberg and Willium, 2001; Woolfolk, 2004; Santrock, 2009). The studies show that the two major factors of psychological characteristic are 1) Attitude toward learning; Atti refers to the student’s ability to show satisfaction, and the agreement and disagreement toward classroom environment, teachers, learning activities, classmates and curriculum (Zimbardo, 1999; Bernstein et al., 2006)., 2) Achievement motivation; Moti refers to students’ willingness, intention, enthusiasm, and attempt to achieve learning objectives with high performance (McClelland, 1961; Woolfolk, 2004)., and 3) Internal locus of control; Loc refers to students’ self-awareness competency in working and achieving the goals, or even when they fail on their tasks, they keep their focus and effort to be successful (Rotter, 1990; Stajkovic and Luthans, 2003).

Intellectual characteristic; IntCh

According to the literature reviews, the findings show that intellectual characteristic also covers intellectual competency, solving problems and reasoning to change learning behavior, and differences of thinking process skills of individuals (Kane et al., 2004; Kim, 2005; Horan, 2007; Silvia, 2008). The results of synthesis show two major factors of intellectual competency, which are 1) Intelligence quotient; IQ refers to competency in learning, solving problems, and adjusting to new environments and problems (Feldman, 1992; Woolfolk, 2004)., 2) Reasoning abilities;  Reas   refers  to  the  ability  in  transferring  previous knowledge to new experiments through thinking processes, solving problems, and finding relationships of things  to make decisions based on the current information and problems (O’Daffer, 1990).        

Objective

To develop and assess the validity of a structural equation model of factors affecting HOTS through meta-analytic structural equation modeling.

Hypothesis

The research hypotheses are given in Table 1. The theoretical models of the factors affecting HOTS are shown in Figure 1.

To identifying the studies relevant for our MASEM consisted of using the internet search from ThaiLis Digital Collection and the electronic theses online system of 71 higher education institutions of Thailand. The studies are composed of quantitative research, experimental and correlational research, which focus on factors relating to the family characteristic, the intellectual characteristic, the psychological characteristic, and the classroom environment, which affect students’ HOTS. The thinking skills consist of three factors; analytical thinking, critical thinking, and creative thinking published during 1999-2013, which was the period when the Thai educational system was renovated and there was more emphasis on students’ thinking skills development. Search keywords include the following terms: 1) classroom climate, 2) teaching and learning methods, 3) teacher behavior, 4) democratic parenting style,  5) parental support, 6) attitude toward learning, 7) achievement motivation, 8) internal locus of control, 9) intelligence quotient, 10) reasoning abilities, 11) analytical thinking 12) critical thinking, 13) creative thinking and 14) higher order thinking skills. The search initially yielded 300 primary studies from 35 educational institutions matching our keywords. The studies were then examined for inclusion in our study, using the inclusion criteria.

Selecting the studies

Not all the studies were appropriate for inclusion in our analysis. Rosenthal (1995) and Wolfswinkel et al. (2013) recommended that researchers should assess the quality of the primary studies before analyzing the establishing criteria for the inclusion of the primary studies by using a multiple-rater technique to evaluate data from the primary studies, and assessing inter-rater reliability. Therefore, the author processes the research as follows;

Inclusion criteria

The studies would be included in the present meta-analysis if it satisfied the following inclusion criteria. 1) At least two of the constructs included in our hypothetical model were analyzed in the studies. 2) The sample in each primary studies are the students of the government schools  3)  Both  bivariate  Pearson  correlations(r) and sample size were reported in the studies. 4) The sufficient data to compute effect sizes according to Glass’ formula were reported in the studies. Abstracts of these papers were examined in greater detail. After closer inspection of the full papers, only 166 studies from 22 educational institutions satisfied all the above criteria and were retained to create a pooled correlation matrix for the MASEM analysis.

Intercoder reliability

The author examined all collected primary studies and recoded information on each study’s demographic and  substantive  features to ensure the literature search processed reliability (Cooper and Hedges, 1994).

The studies were coded by 3 authors independently, consisting of two research advisers and the author, reaching an intercoder agreement of 95%. The level of agreement reached was highly satisfactory. Disagreements in coding were resolved through discussion for consensus.

Data analysis

Two steps of Viswesvaran and Ones (1995) were employed for this

MASEM:

In step 1; the addition of the pooled correlations matrix based on

Hedges and Olkin (1985) method consisted of three steps; 1.1) Transformed correlation coefficients into a standard normal metric using Fisher's r-to-Z transformation  before calculating a weighted average of these transformed scores in fixed-effects model. (Fisher, 1921; Hedges and Olkin's, 1985) 1.2) Next, we tested the homogeneity of correlations from 1.1. Hedges and Olkin’s Q statistic was applied to test the homogeneity of the correlations for each component. The fixed-effects model is appropriate for calculating the pooled correlation matrix when the heterogeneity tests are insignificant. Whereas, the random-effects model is proper when these tests indicate heterogeneity (Hedges and Vevea, 1998; Hunter and Schmidt, 2004). 1.3) After that, we transformed the weighted average Fisher's Z-to-r correlation for each pair of all variables back to the standard correlational form to the more interpretable effects size for reporting. This resulted in a matrix of meta-analytic correlations between all variables in the hypothetical model. The Comprehensive Meta-Analysis computer program was used to perform the data analysis (Borenstein et al., 2009)

In step 2 of the MASEM, the pooled correlation matrix by the true-score population effect sizes of the variable pairs was subjected to the SEM technique using the Mplus version7. The criteria for assessing the validity of a structural equation model was a very good fit with the empirical data from the primary studies composed of the Comparative fit index; CFI, the Tucker - Lewis index; TLI, the Standardized root mean squared residual; SRMR, and the Root mean squared error of approximation; RMSEA. The goodness of fit statistics from structural validity shows that very good fitting model were CFI and TLI ≥ .95 SRMR and RMSEA ≤ .05 (Mclachlan and Pell, 2000; Muthén and Muthén, 2009; Byrne, 2012). For model sample size, we followed the recommendation of Viswesvaran and Ones (1995) to use the harmonic mean as the appropriate sample size because it tends to yield the least biased estimates of standard errors of parameter estimates.

Description of studies

Studies included in the meta-analysis were highly variable in terms of sample sizes that ranged from 411 to 30,163. The harmonic mean of the sample sizes was 655. For each effect size, the author used the following criteria to assess the effect size magnitudes: small (r< 0.30), moderate (0.30 ≤ r < 0.50), and large (r ≥ 0.50) (Cohen, 1988).

Among the 78 average weighted correlations obtained in the fixed effect model varied from small to large (0.060 to 0.669); a majority of correlations (40 out of 78) was the moderate, 25 correlations was the small, and 13 correlations was the large. Lower- and upper- bound effect sizes for confidence intervals of fixed effect model ranged from -0.003-0.683.  In the random effect model, the effect size varied from small to large (0.060-0.576), a majority of correlations (48) was the moderate, 26 correlations was the small, and 4 correlations was the large. Lower- and upper- bound effect sizes for confidence intervals of random effect model ranged  from -0.003-1.958 (show in Appendix A).

Results of the validity of a structural equation model of factors affecting HOTS 

According to the pooled correlation matrix of a structural equation model of factors affecting HOTS consisted of 78 effect sizes in the matrix (show in Appendix B). The result of the initial path analysis showed that the model was a very good fit with the empirical data from the primary studies with   = 0.035, df = 5, p-value = 1.000, TLI = 1.025, CFI = 1.000, SRMR = 0.001, RMSEA = 0.000 (Table 2).

Results of the validity of a structural equation model of factors affecting HOTS are shown in Figure 2.                

In accordance with Table 2 and Figure 2, the direction of effects is summarized as follows:

Direct effect factors are as follows.

The finding showed that three-fourths of the path of factors directly affecting HOTS significantly affected HOTS. The psychological characteristic (H7: 0.762**) indicated a higher effect size than the classroom environment (H1: 0.380*) and double in the intellectual characteristic (H8: 0.363*). The three latent factors explain the variance of 96.8%. However, the family characteristic insignificantly affected HOTS. Therefore, the study of the contribution of the psychological characteristic will enhance students’ HOTS more than the classroom environment, and double in the intellectual characteristic. If we compare the results with the family characteristic, the findings indicated a 7 times higher development in students’ HOTS.

The classroom environment (H2: 0.521**) had significant direct effects on the psychological characteristics equal to the family characteristic (H5: 0.414**). Therefore, in order to study or research, the development of students’ psychological characteristic must focus on the enhancement of the classroom environment and the family characteristic. Even though the effect size of variable in the classroom environment was higher, the result indicated that the variable of the psychological characteristic must be equally focused on.

The factors that directly affected the intellectual characteristic consisted of the classroom environment (H3 : 0.457**), which showed a higher value more than twice of the family characteristic (H6 : 0.208*). It can be conclude that, study in the contribution of the classroom environment will enhance students’ intellectual characteristic more than twice of the family characteristic.

The four paths of indirect factors affecting HOTS are as follows.

The family characteristic indirectly affected HOTS through the  psychological  characteristics of  students  (0.315**).

Example of the value of indirect effects (0.414×0.762 = 0.315) was developed by two paths. 1) The family characteristic  had  a  direct  effect  on  the  psychological characteristic of students (H4: 0.414**). (2) The psychological characteristics of students directly affected HOTS (H7:0.762**).  In conclusion, the family characteristic affects the improvement of the psychological characteristics and will enhance students’ HOTS.

The family characteristic had insignificant indirect effects on the intellectual characteristic. The study indicates that study or research in the contribution of the family characteristic affects the improvement of the intellectual characteristic will not enhance students’ HOTS.

The classroom environment indirectly affected HOTS through the psychological characteristics of students (0.397**). It is concluded that the classroom environment positively affects the psychological characteristics and also increases students’ HOTS.

The classroom environment indirectly affected HOTS through the intellectual characteristic (0.166**). In conclusion, the study shows that the classroom environment positively affects the intellectual characteristic and enhances students’ HOTS.

This study brings conceptual and empirical clarity to the factors affecting HOTS based on the MASEM method. Our study makes four major contributions to theory, as follows.

The psychological characteristic, the classroom environment, and the intellectual characteristic of students directly affect HOTS, which supports the hypothesis. However, the family characteristic insignificantly affects HOTS. The result may be caused by the development of the hypothesis, which determined only direct effects. When this path included in the model had a variety complex variables, it did not support the hypothesis (Ali and Hamed, 2015). The psychological characteristic had effects on HOTS more than any other. This may be explained because  these variables can be continuously developed by various techniques of the learning process, the classroom environment, and parents’ support (Hoang, 2007; Dorman, 2009; Baeten et al., 2013; Umo, 2013).

The classroom environment and the family characteristic directly affect the intellectual characteristic, which supports the hypothesis. Morris and Maisto (2002) assert that the elements of social environment affect the intellectual characteristic. The classroom environment and the family condition are considered to be parts of the social environment. Additionally, this study found that the classroom environment has more than twice an effect of the family characteristic. This may result from the classroom environment and learning management that aim to encourage students to learn and develop their intellectual characteristic. Moreover, classroom management can design situations or experiments practice students thinking skills in various ways during the period of learning (School Drug Education and Road Aware, 2013).

The psychological characteristic of students is an important mediator variable for HOTS. The study indicates two indirect   effects   through    students’   psychological characteristic: the classroom environment and the family characteristic. The results may be caused by the attribute of SEM analysis that able to analyze various effects including direct effect, moderating effect, and reverse effect. These also allow to identify linear and additive relationships of recursive and non- recursive model, as well as indirect effect through the mediator variable (Schumacker and Lomax, 2010; Barbara, 2012). The result of this research shows that the classroom environment has more an indirect effect than the family characteristic. The results may be caused by the effective instructional management, which benefits in organizing classroom environment to support the feelings, attitudes, knowledge, and thinking skills of students (Nelson and Debacker, 2008; Chini et al., 2009; Pascarella et al., 2013).

The family characteristic had insignificant indirect effects on the intellectual characteristic. The results may because the intellectual characteristic had several effects on both the classroom environment and the family characteristic. When this path included the complex model, it did not support the hypothesis. However, the family characteristic is also an important factor that is indirectly affected  through the psychological characteristic and increases students’ HOTS.

The authors have not declared any conflicts of interest.