Non-routine problem solving performances of mathematics teacher candidates

The aim of this study is to determine the non-routine problem solving performances of mathematics teacher candidates. A descriptive survey model was used for this work and it was conducted with 50 teacher candidates studying elementary mathematics teaching in a medium-sized university in Turkey. To determine the non-routine problem solving performance of the teacher candidates, open ended nonroutine problems were prepared according to an expert’s opinion. Firstly, the data obtained were examined with a holistic rubric to know the problem solving levels of the teacher candidates and secondly the levels of the teacher candidates for each problem solving step of Polya were evaluated with a two point scale. When the problem solving performances of the teacher candidates were examined in a holistic manner, it was concluded that their performance is low when the difficult level of problem is raised. From the results, it was determined that teacher candidates mostly tended to go through a solution by trying to use arithmetic operations to solve non-routine problems. As a result, it was suggested to examine teacher candidates’ problem solving knowledge and pedagogical content knowledge and to re-evaluate the results of the research with different question types.


INTRODUCTION
Solving mathematical problem has been one of the key points in research in mathematics education for several decades. It is an advanced thinking ability and consists of different thinking processes (Codina et al., 2015). It is also a decision-making process which involves goaloriented studies that require the recognition of the nature of the problem, the creation of a strategy and the implementation of the strategy (Hayes, 1989). The main focus point is being able to solve problems in a wide array of issues in science, technology, business, science, finance, medicine and daily life.
Problem solving is also a powerful and effective tool for learning, which is also mentioned in the publication of NCTM (2000) standards. When the principles and standards are examined, problem solving is seen not only as a goal of learning mathematics but also as a basic tool of teaching mathematics. Therefore, problem solving is an essential part of learning mathematics and should not be considered separately from mathematics programs. One of the important topics that should be considered E-mail: gozde@balikesir.edu.tr.
Author(s) agree that this article remain permanently open access under the terms of the Creative Commons Attribution License 4.0 International License here is the improvement of the problem solving skills of the students. As skill development is a process, problemsolving skills should be acquired in school and the level of the problem contents and their complexity should be increased as the students' grade level and age increase. Considering that the definition of the problem is a situation that can be solved by analyzing and synthesizing previously acquired information, problem solving requires higher level thinking skills. In this context, the problems can be categorized into different groups with respect to their level of complexity. Altun (1998) categorizes problems into two groups. First group includes the routine problems, which are mostly seen in textbooks and can be solved through the basic operations. Second group consists of non-routine problems which require different skills such as organizing and classifying data, discovering the relations, determining the rules and generalities. Similarly, problems were categorized by Jonassen (1997) into two types: (i) a well-structured problem where all the information needed for the solution is provided; and (ii) an unstructured problem with multiple unknown and multiple states. Well-structured problems are an example of the questions used in school lessons. Problems which are suitable for real world situations are regarded as second type problems. Real life math problems are important to help students, where their previous knowledge is required to find a solution. The problems that students may encounter at school vary because of the different mathematical structures they contain and the differences in their solution purposes. For example, when students move from primary to secondary education, the acquisition of the problem solving method with algebraic reasoning becomes one of the most important tasks (Schmidt and Bednarz, 1997). In secondary education, they are introduced with more advanced methods of algebraic thinking and problem solving. Equations and the symbols representing the unknown become an important part of the solution process in the problem solving approach. Problem solving is used both to reveal the algebraic characters of arithmetic activities, and to provide development on students' algebraic skills with arithmetic thinking used in problem solving process (Van Dooren et al., 2002). Until the twentieth century, the most important reason behind the problem solving, taking part in mathematics teaching programs, was that problem solving was seen as improving one's thinking. In this respect, mathematics education programs mostly included arithmetic operations or logic questions that required a certain solution method. Later, among mathematics educators, the idea that mathematics should include problems related to how to use mathematics in real life to increase the motivation of learning has gained importance (Bingölbali et al., 2016). In this context, non-routine problems and real-life problems were added to the curriculum and the meaning of the concept of problem and problem solving has changed. Polya (1957), who is the pioneer of the research in mathematical problem solving, explains a guideline for problem solving and provides some necessary hints to implement it in his book How to Solve It. Schoenfeld (1987) states that Polya has a great influence on both mathematical thinking and productive thinking and in mathematics education, "problem solving" means "problem solving a la Polya". Polya identified a four-step process which is mostly used for learning problem solving and helps to become a better problem solver and develop problem-solving skills. According to Polya's work, these four steps of problem solving are (i) understanding the problem, (ii) devising a plan, (iii) carrying out the plan, and iv) looking back.
It is seen that an understanding of the aim to develop problem creating and solving skills which are important building blocks in the development of mathematical skills is established in mathematics curriculum. In this direction, it is accepted that one of the main objectives of mathematics education is to improve students' problem solving skills. Problem solving is considered as a basic skill which is expected to be developed for each subject within the curriculum. In the program, it is emphasized that a process including problem-creating should be included in the studies aiming at improving the problem solving skills of the students and related learning outcomes were included at each grade level (Ministry of Turkish National Education (MoNE), 2017). Besides its importance of being a skill that needs to be developed, problem solving is also an important teaching tool. Many of the important mathematical concepts and operations can be taught in the best possible way by problem solving (Van de Walle, 2007). In this context, teachers are the people who are one of the most important dynamo stones expected to improve problem solving skills of students.
The task of teachers, who can establish a strong relationship between mathematics education and problem solving and shape problem-based learning environments, is of great importance in this context. When the effect of a teacher on the quality of education is examined, many components such as subject matter knowledge and beliefs and attitudes of them related with education and subject area become more of an issue. As a result of the research studies carried out in the field of teacher education, besides the general pedagogical features, the competence of the teacher in the subject area that he / she is teaching comes into prominence. It is accepted with the definition of pedagogical content knowledge developed by Shulman (1987) that the teacher's knowledge should not be considered independent of the content he/she teaches. In this context, "General Qualifications of Teaching Profession" and "Subject Matter Knowledge Qualifications" which include the knowledge, skills and attitudes that the teacher should have in order to determine his / her own area of development and to improve themselves in this field have been developed by the Ministry of Turkish National Education (MoNE, 2017), as in many other countries. When the subject matter knowledge is examined, it is seen that it covers the practices that aim to develop students' problem solving, reasoning, and communication skills. Within the scope of developing problem solving skills, teachers' competencies mentioned include understanding the contribution of problem solving ability to mathematics learning, organizing activities to provide development of problem solving skills and enabling students to question the problem-solving process and to verify the results (MoNE, 2018).
Examining the problem solving performances of the teacher candidates in the field of mathematics education and conducting researches for the solution processes they have applied in the process of problem solving may shed light on the knowledge level of the teacher candidates who will take part on this mission mentioned in the future. Although teacher educators generally acknowledge that teacher candidates require guidance in dealing with problems and to face problems, what is often overlooked is that their thought structures should be revealed. It is important not to evaluate the accuracy of the solution, but to examine the behavior of teacher candidates in the problem solving steps and to identify errors. Many research studies focus only on the performance and there is limited number of study which focus on the thinking process of teacher candidates. The goal of this study is to determine the non-routine problem solving performance of mathematics teacher candidates and to focus on their problem solving processes using Polya' problem solving steps. The research problems can be formulated as: (1) What is the problem solving performance of teacher candidates in two different non routine problems at different difficulty levels?
(2) What is the problem solving performance of teacher candidates with respect to Polya's problem solving steps?
(3) What are the strategies that teacher candidates prefer in problem solving?

Research model
In this study, a descriptive survey study was planned to meet the goal, which was to determine the problem solving performances of the mathematics teacher candidates in non-routine problems at different difficulty levels. The descriptive survey model was preferred because it can be used to summarize the characteristics (capabilities, preferences, behaviors, etc.) of the study group, as it is intended to describe a situation that exists in the past or still exists (Büyüköztürk et al., 2017).

Sample
The study was conducted with 50 teacher candidates, who study elementary mathematics teaching in a medium-sized university in 2017-2018. They are from the third graders of 85 students. They were selected by random sampling method. 32 (64%) of the teacher candidates were females and 18 (36%) were males. The teacher candidates are students who completed compulsory mathematics, general education and teaching mathematics courses. The teacher candidates have the knowledge of Polya's problem solving steps from teaching mathematics courses.

Data collection tools
Problem solving performance of teacher candidates was discussed in the context of non-routine problems. In order to determine the non-routine problem solving performance of the teacher candidates, two problems were used, which can be solved with different problem solving techniques, to make in-depth analysis of teacher candidates' problem solving processes. The problems were chosen at different difficulty levels to also examine whether the level of difiiculty of problem affects the results. The two problems used in this study are considered as non-routine problems, since there is not a ready algorithm to solve them at first glance. The problems were selected in line with an expert's opinion among the seven items that were prepared as open-ended problems. In order to determine the validity of the scale, the opinions of 2 mathematics educators and 3 primary mathematics teachers were consulted and their opinions were taken in terms of clarity of the questions, compliance with the Turkish spelling rules and their suitability for achievements. Each item was rated by experts as "The item measures the targeted structure", "The item associated with structure but unnecessary", "The item partially measures the targeted structure "and" The item does not measure targeted structure". Validity rates of each item were obtained according to expert opinions. The items with a validity ratio over 0.78 were included in the pilot scale (Veneziano and Hooper, 1997). It was applied to 14 teacher candidates who were not included in the study group, in order to evaluate the quality for the verbal and visual understandability and clarity of selected problems. The scale was finalized with the problems that did not have any problems in understanding, at different difficulty levels. In the study, teacher candidates were asked to answer the problems using Polya's problem solving steps. P1. Kobe, the famous basketball player who played in the Lakers, scored 63 points in his last match. This score was scored by 29 shots each with two or three points. According to this, how many two -point and how many three-point baskets would Kobe have scored? P2. Robi, Jane and Dan are playing a game with playing cards. At the end of each round, the losing player equally divides his money among the other two players. After three games are played, each player has lost once. At the end of these three games, Robi has $400, Jane has $1000 and Dan has no money. According to this who lost the first game and how much money did Robi, Jane and Dan start with at the beginning of the game?

Data analysis
In order to evaluate the problem solving performances of the teacher candidates, firstly, the data obtained were examined with the "Holistic Rubric" for determining the problem solving scores of teacher candidates. It was chosen due to its focus on the whole process, rather than segmenting the problem-solving process and evaluating each skill or criterion independently. For this purpose, a five-point holistic rubric in Table 1 which was developed by Umay (2007) was used. Thus, a numerical score is given to the whole solution process of each problem in the test. In this context, scores

Description
Scores Completely blank Only data were written down, no attempt for solution Wrong answer and indicators of an inappropriate reasoning were seen 0 Indicators of a correct strategy was written but no application Not reached the aim, some unclear mathematical work, but no put-forth result Correct answer but inappropriate reasoning 1 Correct strategy was found, but the student was not able to apply it or he/she has not tried hard enough Correct answer was found, but there was no indicator as to how it was achieved 2 Correct strategy was found and applied, but there was no correct answer due to some calculation errors and misconceptions Correct strategy was found and correct answer was present but some errors during the application were seen 3 Correct strategy was found and applied correctly, but because one or several of data were misevaluated, correct answer was not reached 4 Complete and appropriate solution and correct answer 5 over 4 were considered as high performance and below 4 were considered as low performance. It was scored as 5, if there is complete solution and as 4, if the solution method and the operation were made correctly, only if the numbers in the problem were taken incorrectly but solved correctly regarding that number. Participants scored as 3 were also considered as having low performance because there were misconceptions or errors at mathematical level or the strategy was not used in a complete correct way. The participants got score 2 if there was no mentioned reasoning with correct answer or the correct solution method with respect to the mentioned strategy. The teacher candiadtes were scored as 1 if there is no correct reasoning and unclear mathematical work and 0 if there is no answer or indicator of appropriate reasoning. Secondly, the solution processes of teacher candidates were analyzed with a two-point (0-1) scale by taking into consideration the expected indicators within the scope of Polya's problem solving steps. 0 point was given to the teacher candidates who cannot answer the problem, 1 point was given to the teacher candidates who correctly answered and used the related strategy. In addition, strategies that the teacher candidates prefer to use were determined, and examples from the analysis and mistakes in the problem solving steps are presented descriptively with the quantitative data.

FINDINGS
The data obtained from the holistic rubric related with the problem solving performance of the teacher candidates are given in Table 2 and the descriptive statistics about the problem solving performance scores are given in Table 3.
When the non-routine problem solving performances of the teacher candidates were examined in a holistic way, it can be said that they perform low especially as the difficulty level of the problem was raised. When the descriptive results of the problem solving performances of the teacher candidates for the problem types were examined, it was seen that the mean score obtained for the first problem ( = 4.02) was almost twice as much as the average score ( = 1.74) obtained for the second question. 31 teacher candidates' solution had no or inappropriate reasoning. When the solutions of the participants about the first problem were examined, it was seen that they used guess and check strategy predominantly and they answered the first problem correctly. In the second problem, the teacher candidates who preferred the strategy of using equation could not solve the problem; candidates who used the strategy of working backwards are successful in finding the answer to the problem. The difficulty level of the problem led to a decrease in the performances of teacher candidates. Although 36 teacher candidates answered the first problem completely, the fact that this number remains at 9 in the second problem supports this idea. Although 11 teacher candidates chose the right strategy in the first problem, a correct solution was not reached, due to the calculation error or wrong evaluation correct answer was not revealed. This situation was encountered in greater proportion in the second problem.
Teacher candidates were asked to explain their problem solving steps with respect to Polya, namely understanding the problem, devising a plan related to the solution, carrying out the plan and looking back. The solutions of the teacher candidates were evaluated separately for the solution steps and the steps were scored as 1 and 0 for each problem based on the indicators in the problem solving steps of Polya. This gives the potential to analyze why teacher candidates had difficulty in the second problem. Descriptive statistics of the data obtained in this direction are presented in Table 4.
When Table 4 is examined, it was determined that 94% of the teacher candidates had the first problem, 96% of them had the second problem answered by writing the expected sentences in their own words or stated that they made the correct association and explanation about the events and relations that were asked in the problems.
Only 3 teacher candidates had issues related with The strategy was chosen correctly but the strategy was not implemented. Some ambiguous mathematical studies have been conducted, but the result is not revealed. Correct answer but inappropriate reasoning 1 5 10 16 32 The strategy was chosen correctly, but the strategy could not be implemented or not enough. The correct answer was found, but there is no indication of how it was found.
2 4 8 7 14 The correct strategy has been selected and implemented, but there is no exact right answer due to a calculation error or misconception.
The right strategy has been selected and the correct answer is present but some errors have been seen during the application.

----
The correct strategy was selected and applied correctly, but the correct answer could not be reached because one or more data were incorrectly evaluated.  understanding the first problem and 2 teacher candidates had issues in understanding the second problem and they left the answers blank or just copied the problems in the same way. It was observed that teacher candidates chose the right strategy for the first problem with the rate of 92% and for the second problem with the rate of 74%. On the other hand, it was determined that 8% of teacher candidates for the first problem and 26% for the second problem chose inappropriate strategy or could not choose any strategy. It was determined that teacher candidates found a complete and correct solution by using the strategy they chose with the rate of 72% for the first problem and with the rate of 18% for the second problem. However, it was determined that 28% of the teacher candidates did not apply or could not find a correct solution for the first problem and 82% of the teacher candidates did not apply or could not find a correct solution for the second problem. In particular, a high failure is considered in the second problem. For the looking back step of the solutions to the problems, it was observed that the teacher candidates have controlled the correctness of the solution they made and created similar problems for the first problem with the rate of 38% and for the second problem with the rate of 4%. In addition, another finding is that there were no teacher candidates seeking to make logical verification. 38% of the teacher candidates have only created the wrong problems in the problem solving step for the first problem and 96% of the teacher candidates for the second problem or in the evaluation of the solution, they have not done any study.
In the problem solving process, teacher candidates were asked to name the strategy they used in "devising a plan" step. Table 5 presents the frequency percentage distribution of the teacher candidates about their preferred strategies of problem solving. When Table 5 is examined, it is seen that teacher candidates prefer to use guess and check strategy with 34%, making a table with 26%, using an equation with 20%, looking for a pattern with 10%, drawing a diagram and eliminating possibilities with 2% in the first problem. In the second problem it is seen that they prefer to use equation with 24%, working backwards with 20%, guess and check with 16%, making a systematic list with 6%, making a table and drawing a diagram with 4%. However, while 92% of the teacher candidates could choose the right strategy for the first problem, only 72% of them have ended up with the solution by using the strategy correctly. This ratio results in an even greater difference in problem 2; while the percentage of teacher candidates who chose the right strategy was 74%, the number of teacher candidates who achieved the result by applying the chosen strategy remained at 18%. This is due to some of the mistakes students make while implementing the strategy. The responses of T8 and T16 using the table making strategy and guess and check strategy for the first problem are given in Figure 1. As shown in the figure, T8 and T16 preferred to use guess and check and making a table strategy to reach the solution of the problem correctly. T20 preferred the strategy of making a table for the first problem. The solution that has been realized in this direction is presented in Figure 2.
When Figure 2 is examined, it is seen that teacher candidate understood 2 and 3-point shooting giving a total number of 63; however, it is ignored that this number was obtained with 29 shots. In addition, the solution he implemented with the strategy that he considered to be used is not compatible with each other. He did not understand the strategy of making tables and problem. In the second problem, 20% of the teacher candidates used the strategy of studying backwards. The response of T5 and T2 is given in Figure 3. When the responses of T5 and T2 are examined, with the reference of Robi has $400, Jane has $1000 and Dan has no money at the end of three games, they carried out the backward studying and reached the correct result.
Although 82% of teacher candidates chose the right strategy, 32% of teacher candidates did not apply correct study according to the strategy, 14% of the candidates chose the right strategy but did not apply the strategy and 6% of them chose the right strategy, and applied it correctly; however because of wrong evaluation of some data they could not get the right answer.
In addition, it is determined that some of the teachers did not know the names of the strategies they preferred to use correctly. For example, T7 realized the solution of Figure 4 for the first problem. As shown in Figure 4, T7 decided to use the "Pattern Search" strategy for the first problem, but by using the equation with two unknowns, he valued the variables and used the guess and check strategy. The responses of T23, T40 and T32 for the first problem are given in Figure 5. When Figure 5 is examined, it indicates that T23 preferred the making a table strategy but used the guess and check strategy, T32 preferred the making a table strategy, but   T8: Creating Table  T16: Guess and check strategy T5 T2  implemented using an equation strategy by placing the variables of two equation into the table and solving the equation, T40, firstly, solved the system of equations with two unknowns and placed the variables into the table and called this study as the strategy of making a table. The solutions of the teacher candidates T37, T41 and T28 for the first and second problems are given in Figure 6. When Figure 6 is examined, it is seen that T37 named the drawings he used to visualize the data of the problem as drawing a diagram strategy and could not solve the problem correctly. T41 used the expression "we can solve by establishing equation and connecting the variables with diagram". T41, who formed a structure with three unknowns, had drawn arrows in each game according to their lost situation and named them as drawing a diagram strategy and could not reach the correct answer. T28 used the following expression while solving the first problem: "First, we use a diagram drawing strategy, then we use equation using and guess and check strategies". He evaluated the different images he drew concerning the variables as a diagram strategy, solved the question by using two unknown system of equations and determined the operations he performed while checking the accuracy of the response as guess and check strategy. The responses of T48 and T9, who mentioned that they use guess and check strategy and making a systematic list strategy in the second problem, are presented in Figure 7. When Figure 7 is examined, it is seen that, although T48 preferred the strategy of guess and check, he used the strategy of using a variable and working backwards, T9 preferred to use the making a systematic list strategy, but used variable and could not reach the result.

DISCUSSION
In this study, which was conducted to determine nonroutine problem solving performances of mathematics teacher candidates, problem solving performances were discussed in the context of non-routine problems. Problem solving scale was prepared in order to determine non-routine problem solving performance of teacher candidates and firstly the responses were evaluated with the holistic rubric and a two-point scale for evaluating the problem solving steps. Although the results based on two problems with different solution strategies and difficulty levels, it provides valuable information about teacher candidates' problem solving skills. As a result, it is seen that 94% of the teacher candidates for the first problem, 96% of them for the T23 T32 T40 Figure 6. Responses of teacher candidates using drawing a diagram strategy for first and second problem. second problem expressed what was asked in the problem in their own words and made the correct relation and explanation about the events and relationships; 92% of teacher candidates chose the right strategy for the first problem and 74% teacher candidates chose the right strategy for the second problem. They made a complete and accurate solution using the strategy they choose for the first problem as 72% and for the second problem as 18%. In the first problem 38%, in the second problem 4% of the candidates controlled the accuracy of the solution they made completely and established similar problems.
In addition, 38% of the teacher candidates for the first problem, 96% of them for the second problem established only incorrect problems or have not done any study in evaluation of the solution step. Secondly, it was determined that teacher candidates preferred mostly guess and check strategy (34%) in the first problem and using an equation (24%) strategy in the second problem. However, 92% of the teacher candidates chose the right strategy for the first problem, while only 72% of them applied the chosen strategy correctly and reached the solution. This ratio resulted in even greater differences in problem 2; while the percentage of teacher candidates who chose the right strategy was 74%, the number of teacher candidates who achieved the result by applying the chosen strategy remained at 18%. This is due to some of the mistakes students make while implementing the strategy.
When the problem solving performances of the teacher candidates are examined in a holistic way, it can be said that they perform low. When the descriptive results of the problem solving performances of the teacher candidates for the problem types are examined, it is seen that the mean score obtained for the first problem ( = 4.02) is almost twice as much as the average score ( = 1.74) obtained for the second problem. The difficulty, which was created in problem quality, led to a decrease in the performances of the teacher candidates. Although 36 teacher candidates answered the first problem fully and completely, the fact that this number remains at 9 in the T37 T41 T28 T48 T9 second problem supports this idea. The main reason why the level of performance was low is that teacher candidates mostly prefer using equation and guess and check strategy. These are also very common problem solving strategies used in routine problem solving. Although the teacher candidates knew the theoretical knowledge about the strategies, it was observed that they had difficulty in applying the strategies other than using equations and guess and check. Some of the teacher candidates could name the correct strategy but could not reach the correct answer due to misapplication of the strategy or could reach the correct result but the name provided for strategy was not compatible with what they carried out. This is because they do not encounter problems that require different strategies. In their study, Kaya and Kablan (2018) mentioned that the success in using more than one solutions for non-routine questions is low. They concluded that difficulties mostly arise from using multiple strategies and everyday experiences when solving especially non-routine problems.
The results show that the problem-solving performance of teacher candidates decreased when the difficulty level of the problem increased; also, it is seen that the strategies they use for the solution did not correspond to the strategies they wrote. It was determined that the teacher candidates were quite inadequate in the evaluation of the solution. These results indicate that teacher candidates' problem solving skills are not sufficient and therefore they perform poorly. Sufficient knowledge of the problem solving skills of the teachers who are responsible for developing their students' problem solving skills is related to the knowledge of the field.
The primary objective of the curriculum is to improve the mathematical problem-solving skills, to develop reasoning skills and to develop the ability to use these skills in solving problems encountered in real life. It is also declared that problem solving facilitates mathematics learning and affects mathematical thinking (Verschaffel et al., 1999). However, when the results and different studies were examined, it was seen that teacher candidates showed low success in problems involving non-routine problems (Dündar, 2014;Akgün et al., 2012); they tend to use the arithmetic operation to solve nonroutine problems (Dündar, 2014;Lawrence, 1977). However, the program does not provide sufficient support for the development of professional skills and accomplishments. Therefore, there is a big gap between theoretical preparation and school experience (Orgoványi -Gajdos, 2016). Shulman (1987) defined teachers' knowledge on the pedagogical content knowledge model. In this model, he emphasized that the defined content knowledge is related to the teacher's own field. He also emphasized that content knowledge should not be considered separately from mathematics teaching knowledge. There are many mathematics educators who suggest that the inclusion of problem solving and problem Akyüz 223 formation in mathematics lessons can have a positive effect on students' mathematical thinking (Abu-Elwan, 1999;Kilpatrick, 1987).

Conclusion
In this case, it is recommended that a more effective teaching process and practices of different models should be included in teacher education in order to increase the performance of teacher candidates for problem solving. Practical work should be included in all the courses in the curriculum of mathematics teacher education programs. Both the content courses related to mathematics and the pedagogical content courses should be implemented in a problem solving environment. Teacher candidates should be encouraged to encounter different types of problems both in their courses and out-of-school activities through mathematics clubs and competetions arranged in the faculty or university. In this context, it is recommended to examine the problem solving knowledge and pedagogical content knowledge of the teacher candidates, and to reevaluate the results of the study with different question types.