Sixth graders and non-routine problems : Which strategies are decisive for success ?

This study investigated the role of each strategy in explaining sixth graders’ (12-13 years old students’) non-routine problem solving success and discriminating between successful and unsuccessful students. Twelve non-routine problems were given to 123 pupils. Answers were scored between 0 and 10. Bottom and top segments of 27% were then determined based on total scores. All scripts of pupils in these segments were then re-scored with regard to strategy use. Multiple regression analysis showed that strategies explain 65% of the problem solving success. Order of importance of strategies are as follows: make a drawing, look for a pattern, guess and check, make a systematic list, simplify the problem, and work backward. According to discriminant analysis results, strategies which play a significant role in distinguishing top and bottom students are look for a pattern, make a drawing, simplify the problem, guess and check and work backward, respectively.


INTRODUCTION
Not only in the history of mathematics but also in mathematics teaching, problem-solving always plays an important role, since all creative mathematical work demands actions of problem-solving (Burchartz and Stein, 2002).Besides, mathematics education communities commonly agree that teaching problem solving means teaching non-routine problems as well as routine problems.Actually, a large body of literature about mathematical problem-solving shows that non-routine problems are the kind of problems which are most appropriate for developing mathematical problem-solving and reasoning skills, as well as development of the ability to apply these skills to real-life situations (Cai, 2003;London, 2007).Although routine problems can be solved using methods familiar to students by replicating previously learned methods in a step-by-step fashion, non-routine problems are problems for which there is no predictable, well-rehearsed approach or pathway explicitly suggested by the task, task instructions, or worked-out examples (Woodward et al., 2012).Importantly, nonroutine problems require reasoning and higher-order thinking skills and often go beyond procedural skills (Kolovou et al., 2009).
Non-routine problem solving strategies can be defined E-mail: yazgany@uludag.edu.tr.Tel: (+90) 538 6751775, (+90) 224 2942218 Authors agree that this article remain permanently open access under the terms of the Creative Commons Attribution License 4.0 International License as procedures used to explore, analyze, and probe aspects of non-routine problems in an attempt to formulate pathways to a solution (Nancarrow, 2004).These strategies play a very important role in the mathematical process experienced by students while solving non-routine problems.Results of recent studies have provided evidence for the use of non-routine problem solving strategies as a means to enhance problem solving (Elia et al., 2009).In the literature, the most outstanding non-routine problem solving strategies are as follows: Act it out, look for a pattern, make a systematic list, work backward, guess and check, make a drawing or diagram, write an equation or open sentence, simplify the problem, make a table, eliminate the possibilities, use logical reasoning, matrix logic, and estimation.(Herr and Johnson, 2002;Altun et al., 2007;Leng, 2008;Posamentier and Krulik, 2008;Posamentier and Krulik, 2009;Fang et al., 2009).
According to Tiong et al. (2005), non-routine problem solving strategies do not guarantee solution, and have non-rigorous suggestions about what should be done.Besides, these strategies are not applicable to a certain type of question in a specific topic area.On the contrary, non-routine problem solving strategies are generally applicable through types of question and topic areas.This raises another important characteristic of nonroutine problem solving strategies, that of transferability.The most important reason to learn non-routine problem solving strategies is that they can help individuals solve problems in unfamiliar topic areas and expand their point of view.
The terms "non-routine problem" and "problem" will be used interchangeably from this point.Additionally, the terms "strategies" and "problem solving success" will refer to "non-routine problem solving strategies" and "nonroutine problem solving success", respectively.

A general overview of studies on non-routine problem solving
Most research about non-routine problem solving is aimed at examining students' currently used skills and attitudes on this subject without any intervention (Hok-Wing and Bin, 2002;Muir and Beswick, 2005;Wong and Tiong, 2006;Wong, 2008;Muir et al., 2008;Elia et al., 2009;Salleh and Zakaria, 2009;Mabilangan et al., 2012).There are also some studies examining effects of an intervention on non-routine problem solving skills of students (Follmer, 2000;Ishida, 2002;Nancarrow, 2004;Johnson and Schmidt, 2006;Lee et al., 2014).Some research aims to reveal what problem solving abilities mathematically promising and/or low achieving students show in solving non-routine problems (Johnson and Schmidt, 2006;Budak, 2012).Gender related differences on solving non-routine problems have been investigated in other studies (Salleh and Zakaria, 2009;Abedalaziz, 2011).Another group of studies is focused on the place of non-routine problems and strategies in mathematics textbooks and syllabi (Lianghuo andYan, 2000, Kolovou et al., 2009;Marchis, 2012).
Grade levels of aforementioned studies vary from primary school to high school and their results can be summarized in five points.Firstly, many students consider that non-routine problems are more complicated and difficult than routine problems.Therefore these students may not initially believe that the non-routine problems are mathematical since they are not familiar with this kind of problem.Secondly, providing students with a framework for the use of strategies is beneficial and increased students' level of confidence.Third, mathematically promising students are more determined in solving nonroutine problems and look for alternate ways if the one they tried does not work.However, the direct teaching of strategies to low achieving students makes them more comprehensive and positive about solving non-routine problems; Fourth, there are no significant differences between males and females with regard to ability and attitude to solve non-routine problems.Lastly, only a very small proportion of the problems included in the textbooks is non-routine.In some textbooks series these problems are completely absent.

Related studies
Some of the studies carried out with sixth graders will be explained more in-depth here as they are related with the current study: In his study, Ishida (2002) also used nonroutine problems, and strategies examined by him were the same as this study.But distinctly from the present study, participants in Ishida (2002)'s study had been taught strategies explicitly since 2 nd grade.In Ishida's study (2002)'s, students solved only two non-routine problems, usually in at least two ways, and they were asked which of their solution strategies for each question was better.Students generally selected a strategy as being better because it was efficient, and easy to use or to understand.Mathematical values such as generality were rare.Muir et al (2008) and Wong (2008) also elaborated on strategies selected by sixth graders to solve non-routine problems.However, the former focused on characteristics of behaviors associated with "novice" and "expert" problem solvers.Muir et al. (2008) asserted that these categories were not fully adequate and instead "naïve", "routine" and "sophisticated" approaches to solving problems were identified.Wong (2008) made a comparison of strategies used between two parallel tests which were administered after a few months.The comparison showed that some pupils did not use similar strategies to solve parallel problems.In studies by Ishida (2002), Muir et al. (2008) and Wong (2008), pupils' written solutions were analyzed according to strategies used.Although it was not carried out at sixth grade level, another investigation which is most related to the present study in respect to aim and statistics was done by Altun and Sezgin-Memnun (2008).It was carried out with mathematics teacher trainees that were given problem solving instruction.Pre, post, and retention tests were also conducted.Statistical analysis revealed that instruction increased the trainees' problem solving success.The most successful strategies were ranked as follows: simplify the problem, look for a pattern, reasoning, make a drawing, make a systematic list, guess and check, and work backwards.According to discriminant analysis results, reasoning, work backwards, make a drawing, make a table and simplify the problem, respectively had a big impact in separation of successful and unsuccessful participants.The analysis also confirmed that 80% of problem solving success could be explained by the strategies used.

Research questions
Studies investigating the impact of each of above mentioned strategies on the success of students are really rare.Moreover, none of them are at sixth grade level.Therefore, this study investigates the role of each strategy in explaining success and in discriminating between successful and unsuccessful students at sixth grade level.Thus, research questions are as follows: i. What is the role of each non-routine problem solving strategy in explaining the non-routine problem solving success?ii.Which non-routine problem solving strategies are more effective in discriminating between successful and unsuccessful students at sixth grade level?

Participants
One hundred and twenty three students in sixth grade participated in the study from a secondary school governed by MoNE (Ministry of National Education) in Bursa/Turkey.Age range at six grade level may be different in some countries, but sixth graders in Turkey are 12-13 years old as it is in most countries.
Sixth grade level was chosen because sixth graders were supposed to have adequate literacy to enable understanding of the questions and to be able to record their answers in written form.Participants had not had any special training on non-routine problem solving in their school life before the current study.

Information about data collection instrument and procedure
To measure the problem-solving success of students, a paper and Yazgan 1809 pencil test comprising 12 non-routine open-ended problems (Problem Solving Test-PST) was constructed by the author.All problems in PST were chosen with regard to their appropriateness in terms of degree of challenge offered to the selected age group, and their potential to be answered using a variety of strategies both across the problems and for any given problem, like Muir et al. (2008) did in their studies.For example, problem numbered as six and eleven in the PST could be answered successfully through applying make a diagram, look for a pattern and simplify the problem strategies together.
The PST instrument was administered to participants by a PhD student who was enrolled in a program on mathematics education.PhD student was also a mathematics teacher and he conducted the PST to all his students.PST was presented to students in two sets since it was thought that 12 questions were too much for students to solve at once.There was one week between two structurally parallel sets.Students were given 90 minutes in total to complete the PST.But, if a student needed it, he/she could have extra time.Students were encouraged to write down all their thoughts while they were solving the problems.Problems in the PST were intended for pupils to make use of make a systematic list, look for a pattern, work backward, simplify the problem, make a drawing, and guess and check strategies (see Appendix).Previous studies, projects and books (Ishida, 2002;Altun et al., 2007;Posamentier and Krulik, 2008;Fang et al., 2009) dealing with strategies that can be learned and used at sixth grade level were taken into consideration during selection of strategies.
All students' answers to each problem in PST were scored between 0 and 10 by categorizing them under the titles of correct, little mistakes stemming from calculation errors or inattentiveness, insufficient answers despite understanding the problem and taking the right action, wrong answers and no answer.Every student got a success score between 0 and 120.The administrator of the PST had previously been trained in analyzing answers of students and non-routine problem solving.Answers were evaluated independently by the author and PhD student, and the given points were compared to each other.Nevertheless, when uncertainty arose, the author and the student discussed the solution and decided upon an agreed code.This process was similar to that used in Wong's (2008) study.The focus was on the correctness of the students' problem solving processes.Strategy usage was not taken into consideration at this point.
Based on scores for each question in the PST instrument, Cronbach's alpha was used to assess the reliability of the instrument.Generally, the lower limit for Cronbach's Alpha should be .70(Robinson et al., 1991).The alpha coefficient was computed as .75 for the PST indicating acceptable reliability for the instrument.
To assess validity of the PST, confirmatory factor analysis with Varimax rotation was performed based on the same scores as used for the Cronbach alpha reliability.Before proceeding with factor analysis, it was necessary to assess the sampling adequacy.This was done using the Kaiser-Meyer-Olkin (KMO) and Bartlett's test for Sphericity.The level of significance of KMO and the Bartlett's test for Sphericity should be significant at the .05confidence level (Hair et al., 1998).Table 1 shows that the results of the tests meet acceptable levels that permit proceeding with the factor analysis.
Factor analysis results suggested the existence of five factors.The eigenvalues showed that these factors explained 14. 69, 12.83, 11.54, 10.74 and 10.26% of the variance of PST scores, respectively.Collectively these factors explained 60.06% of the variance.As seen from the rotated component matrix (Table 2), factor loading of almost each item is greater than .45.Thus, it can be said that problem solving success was properly measured by items of PST.In other words, the administered PST test was  validated.
To answer the research questions (except the scoring system explained above), all student scripts were evaluated with regard to strategy use again.Instead of scoring each question separately, usages of each strategies dealt with in this study were coded as 2 (correct and effective usage of strategy), 1 (incomplete usage of strategy), and 0 (no usage of strategy).So, every student had a point about usage of each strategy.
To explain scoring processes of problem solving success and strategy use thoroughly, some samples from students' answers to questions in PST will be reprented here.As seen from Figure 1a, Ozan 1 did not prefer to use make a drawing strategy to answer the second question in PST (Students in a class are standing in a 1 Pseudonym name circle; they are evenly spaced and are numbered in order.The student with number 7 is standing directly across from the student with number 17. How many students are in the class?).Instead, he wrote: "Firstly I found that there are 10 students on the half of the circle (including 7, except 17).Then I thought that the number of students on the other half of the circle must be the same.Therefore I multiplied the 10 by 2".Since Ozan's reasoning and answer were right, his problem solving success score for this question was 10 (Figure 1a).But Ozan did not get any score about use of make a drawing strategy from this question.Another student's answer to the same question is represented in Figure 1b.Damla 1 directly used make a drawing strategy.Filling half of the circle, Damla recognized that there are nine students between 7 and 17 (except them).And she concluded that there must be nine students on the other half as well.Lastly, Damla added 7 and 17, and she reached the solution.Damla's success score was 10, and she exhibited correct and effective usage of make a drawing strategy.Sixth question in PST (Each of following shapes consists of small rectangles like the first one.How many rectangles do you need to make fifteenth shape?) is required using a variety of strategies together.One of students, Ceren 1 succesfully used look for a pattern and simplify the problem strategies to solve the problem (Figure 2).She first examined the numbers of rectangles in first three shape.Then Ceren realized that if she multiply the number of shape by itself, she can find the number of all rectangles in the shape.She got full success score from this question.Besides, her strategy scores on use of look for a pattern and the problem strategies were 2.
To account for inter-coder agreement, Cohen kappa coefficient for inter rater agreement was calculated based on codes about strategy use and found to be .79.Based on these codes again, means and standard deviations about each strategy were then computed.To reveal how functional the strategies were in the problem solving success, multiple regression analysis was carried out by using strategy scores (independent variables) and total problem solving success scores (dependent variable) for each student.
In order to see which strategies were more useful in discriminating between successful and unsuccessful students, first bottom and top segments of 27% were determined according to the total success scores which the students achieved in the PST.The group at the bottom (low achievers) consisting of 27% of the students was composed of those who had got a score of 16 or lower, and the group at the top (high achievers) consisting of another 27% of the students achieving higher than 27.There were 37 students in each of bottom and top groups.Discriminant analysis was carried later out based on the strategy and total success scores of students.
With regard to the first research question, multiple regression analysis was used to determine each strategy's contribution to problem solving success.The results of the regression analysis are given in Table 3.
According to the dual correlations between the problem solving success and strategies, it can be seen that the highest correlation coefficients belong to the strategies of look for a pattern (.55), simplify the problem (.52), guess and check (.41) and make a drawing (.41).Correlation coefficients which belong to work backward and make a systematic list strategies (.30 and .11)indicate that there is a positive but weak relationship between each of these strategies and problem solving success.Generally speaking, there is a significant relationship between strategies and problem solving success (R = .80,R 2 = .65,p = .00).Strategies used in this study as independent variables, explain almost 65% of problem solving success.According to the standardized regression According to the results of discriminant analysis (considering the low values of Wilks' Lambda and high value of F), strategies that play a significant role in distinguishing successful and unsuccessful students were look for a pattern, make a drawing, simplify the problem, guess and check, and work backward respectively (Table 4).The make a systematic list strategy did not have any significant contribution.With the help of the discriminant function, a classification with an accuracy rate of 86.5% was achieved.

DISCUSSION
Non-routine problems, which are usually not specific to any mathematical topic, have no fixed procedure for solving, and require the use of one or more strategies to solve.Moreover, non-routine problems are especially challenging for many students, since these kind of problems require an integration of several cognitive processes such as accounting for all possibilities, visualizing relationships (Lee et al., 2014).
Regarding the findings about the research questions, the R 2 value found through the multiple regression analysis demonstrates that in general, knowledge of strategies explains 65% of non-routine problem solving performance.Besides, 86.5% accuracy in classification according to discriminant analysis results suggests that strategies have a dominant and decisive role in determining novice and expert problem solvers.Altun and Sezgin-Memnun (2008) achieved the similar results in this sense.However, the most important contribution of the present study is the detailed information on the role of each strategy.Even though almost all strategies had statistically significant roles, two strategies were conspicuous: look for a pattern and make a drawing.These two strategies had strong effects not only in explaining success, but also in discriminating high and low achieving students.Simplify the problem, quess and check had relatively weak roles.Work backward was the weakest strategy in explaining success and in discriminating high and low achievers.Neither high nor low achieving students could use this strategy successfully.In spite of its significant impact on success, make a systematic list did not has function to differentiate low and high achieving students.When this inefficiency of make a systematic list is combined with the high mean of it, it can be said that students in both achievement group coped with the problems requiring application of this strategy.
Determining importance order and distinctiveness of each strategy in terms of problem solving success may bring some important outcomes which can be used to arrange textbooks and learning environments.First, since each of six strategies sought in the current study has meaningful contribution to success, it can be concluded that all these strategies are feasible to work at sixth grade level.High potential of look for a pattern and make a drawing strategies might indicate that these strategies can be highlighted.However, other strategies should not be neglected.On the contrary, impotent strategies such as work backward should be given more time so that they can also explain the success and can be used effectively by all students.
Despite the fact that this study provides meaningful conclusions, further research is necessary.Firstly, using more different types of non-routine problems would provide more robust evidence for the findings of this study.Strategies which were not investigated in this study such as logical reasoning or make a table can be included in future studies.Maybe routine problems can also be involved to provide more detailed information about the general problem solving success of students.Besides, thinking processes of students were not analyzed in detail in the present study.Because this study was based on students' written answers and many students have difficulties in writing down their thinking processes.Complementary qualitative methods such as clinical interviews, observations and videotaping of pupils solving problems could be used to collect data about students' cognitive processes.In addition, examining students of different grades, age and school year level, mathematical abilities, and educational systems would be more excellent analysis.
In summary, if a primary goal of instruction is to develop students' ability to think strategically so that they have a problem solving disposition including the confidence and willingness to take on new, unfamiliar and difficult tasks, non-routine problems should be developed from the first grades of elementary school.Teachers may benefit from non-routine problems that force students to apply what they have learned in a new way.Furthermore, teachers should increase the variety of strategies which they present to the students by considering grade level, and difficulty level of the strategy.

Figure 1 .
Figure 1.Ozan and Damla's answer to second question in PST.

Figure 2 .
Figure 2. A student's answer to sixth question in PST.

Table 3 .
Multiple regression analysis results.

Table 4 .
Wilks' Lambda and F values of bottom and top groups in terms of success.