Full Length Research Paper
ABSTRACT
Mobile technologies have started to be preferred as a new learning technology in teaching and learning. Considering the need for adequate and rapid feedback in solving mathematical problems that students have difficulty with reflects the need for a different educational practice and technology. In this study, based on designbased research (DBR) model, a mobile application was developed, which enables the extension of the traditional classroom to a virtual space, where students can practice mathematical problem solving and easily access solutions. The purpose of this study was to determine which mathematics questions that students have difficulty in solving are mainly distributed over which subjects and learning outcomes, and to reveal the cognitive levels of the questions sent. The findings revealed that 'numbers and algebra' was the learning domain, and ‘derivate’ was the subject, which students had the most difficulty with. Moreover, the questions were found to be mostly at the cognitive levels of understanding and applying in Bloom’s Taxonomy. Based on the results, it can be concluded that the mathematics mobile application can be used as an alternative tool examining the learning outcomes and cognitive levels of students.
Key words: Mobile app, design and development, mobile learning, mathematics education, learning outcomes.
INTRODUCTION
With the increase in the processing power of mobile devices, a wide range of mobile applications have entered and are entering the market, and the number of those that can be used for educational purposes is increasing. Rapid developments in mobile technologies have paved the way for different and new formations in education. Mobile technologies, which developed with the concept of networked society or mobile society, have affected learning processes and approaches (Gan and Balakrishnan, 2017), and have led to the start of a new era in education (Huang et al., 2010) by providing "learning onthego" and "justintime learning" (Zoraini et al., 2009).
The tendency of students to mobile technology, their adaptation in a short time and their enthusiasm for this technology has attracted the attention of educators. For this reason, the necessity of this technology, which has an important place in the social and cultural activities of students, was strongly felt in the field of education. Mobile devices provide a critical source of information and motivation for students' learning experiences, as students often develop a strong attachment to and never let go of their mobile devices. Students have started to use mobile technologies, especially smart phones (Davies et al., 2012), as a mobile learning platform that enables them to develop their individual and independent learning (Nedungadi and Raman, 2012) at their own learning pace and to access information quickly on the go (Gikas and Grant, 2013), regardless of time and place.
Mobile learning appears to have an impact on education (Alden, 2013) and be considered as engaging and useful in many different contexts, with different target groups (Baya’a and Daher, 2009; Kalloo and Mohan, 2012). For instance, it is suggested that the integration of mobile technologies into mathematics teaching and learning could provide more meaningful, practical, and engaging learning experience (Bray and Tangney, 2016). Learning mathematics can be a struggle as well as challenge for many students, at which point effective use of technology by both teachers and students can help to increase students' motivation, bring out the inspiration to learn, and foster understanding of math. In addition, the use of technology in learning mathematics can reduce math errors as it helps students focus more and use their time efficiently (Wadlington and Wadlington, 2008). Hegedus et al. (2015) reported that their hypothesis was supported in their study such that the implementation of technology in a high school mathematics course improved student engagement, teacherstudent interaction, and students’ test results. In recent years, a number of review studies have been conducted with regards to mobile technologies, in other words, mobile learning in mathematics. Crompton and Burke’s (2017) found out in their literature review study that there is a rising interest in mobile technologies supporting positive learning outcomes in mathematics. Moreover, Fabian et al. (2016) reviewed 31 studies, the majority of which reported mobile technologies improved students’ success. On the other hand, student success is affected by numerous factors (curriculum, students’ selfefficacy, parents’ socioeconomic status, etc.) (Hattie, 2009) but teacher support including providing feedback and guidance to difficult math problems play a critical role in improving student achievement in significant ways (CorneliusWhite, 2007; Roorda et al., 2011). Students perceive teacher support in a way that they receive their teacher’s care, respect, approval, and assistance (Klem and Connell, 2004; Ryan and Patrick, 2001).
Mathematics has been a problematic subject for students all over the world. It is not easy to develop skills, especially in solving mathematical problems and finding elegant and alternative solutions when faced with challenging conditions.
The quantitative multiplicity caused by the increase in the number of students in Turkiye every year, the placement of students in the educational institutions through examinations, and the supplydemand imbalance between the number of students and the number of schools make student selection exams mandatory (Gundogdu et al., 2010). The exam taken by students in the last year of secondary education to transfer to a higher education institution is a multiplechoice exam.
These exams in the education system lead students to solve more questions. In particular, high school students who wish to be placed in a university are expected to solve math problems regularly and systematically, and engage in meaningful analysis and comparison of those. The acquisition of mathematics knowledge and problem solving skills in high school is essential for students’ enrollment to a major at the university, and their career success in a progressively hightech economy (Geiser and Santelices, 2007; National Mathematics Advisory Panel, 2008; Wang, 2013). However, traditional mathematics instruction may not be sufficient for improving problem solving skills. Besides, required to be solving many math problems can be burdensome for students.
In general, primary school children can solve math problems with the help of their parents. On the other hand, the ability of parents to support children in the following years of education becomes very limited according to their education level. Families seek various solutions according to their economic situation; however, they may not have the resources to support their children, for instance, on private lessons. In case of a math question that cannot be solved, students ask someone else, usually their teacher to solve the question. However, there are some limitations at this point.
In order to present the question, it is necessary to find a suitable time for the teacher during school hours and consult the teacher for a solution. This situation limits problem solving to school time. When students leave their questions blank to be solved later and continue with other questions, similar questions or questions on other topics are also left blank as the question at firsthand left blank was not answered. This leads to stacked questions that need to be resolved.
Therefore, the construction of semantic knowledge on the math subject remained unfinished, which possibly lead to a failure in math.
Being successful or failed in mathematics, particularly in the case of solving higherorder math questions, is profoundly reliant on in what way the students are provided with the feedback (Hattie and Timperley, 2007). Feedback hence can promote and support students to develop their mindset and go further in their learning experience to achieve desired success goals (Guskey, 2010). It is important for students to have their teacher's solution to check for the "correct" solution(s) (Nicol and MacfarlaneDick, 2006). Although it is accepted that being involved in the higherorder thinking process by solving math questions and getting teacher support for this may be factors for students' math academic success, there are fewer studies on these factors in the context of mobile platform.
High school students, in particular, need to have a continuous, and fast feedback and support by the teachers in solving mathematics questions, which might help them become successful at the university entrance exam. This process is significant especially for the students coming from a low socioeconomic status. High school students try to find solutions and realize their shortcomings by asking their teachers about the problems they cannot answer. Being able to reach a solution by sharing the questions in a digital platform can provide an important convenience for the students. A digital platform where many students share only the questions they cannot solve among thousands of questions from different educational resources can provide important data in determining the learning outcomes that students cannot reach. Learning outcomes are decisive in teaching activities and the distribution of the questions shared by the students according to the learning outcomes is important. Since learning outcomes have an impact on students' learning processes, it is of great importance to determine learning outcomes correctly. Additionally, despite a growing body of research on mobile math learning, learning outcomes in high school using this mobile technology have rarely been studied.
Through such mobile learning platform, it can also be revealed that cognitive levels students have difficulty in reaching. A mobile learning platform to be developed for this purpose can contribute to the determination of the learning outcomes that can be reached and the cognitive levels of the students. In this study, a mobile application was developed in order to determine which mathematics questions that students have difficulty in solving are mainly distributed over which mathematics subjects and learning outcomes, and to reveal the cognitive levels of the questions sent. The present study both describes the design and development process of a mobile application and answers the following research questions:
1. In which mathematics subjects and learning domains did students have the most difficulty in problem solving and sought support for their solution?
2. At which cognitive levels are the learning domain questions that students have the most difficulty with?
3. What is the distribution of these cognitive levels among classes in high school?
4. In this learning domain, what math learning outcomes and associated cognitive level did students submit the most questions about?
3. What are the teachers' experiences and views on using the mobile learning application?
METHODS
In this study, designbased research (DBR) model involving a mixed method approach was conducted as it focuses on the design, production, implementation, and an impact of a learning initiative in practice in real educational contexts. The study adapted the processes of DBR by (Reeves, 2006:59): 1) Analysis of Practical Problems by Research and Practitioners in Collaboration as ‘Analysis Stage’, 2) Development of Solutions Informed by Existing Design Principles and Technological Innovations as ‘Design and Development of the Technological Innovation  Mobile App Stage’, 3) Iterative Cycles of Testing and Refinement of Solutions in Practice as ‘Testing and Refinement Stage’, and 4) Reflection to Produce “Design Principles” and Enhance Solutions Implementation as ‘Implementation of the Mobile App Stage’.
Designbased research has important features in designing and testing of educational interventions and solutions, such that it involves both quantitative and qualitative methods and supports design principles through participatory and collaborative work (Anderson and Shattuck, 2012; DesignBased Research Collective, 2003).
In the quantitative part of the present study, descriptive statistics obtained from the mobile learning platform was used to examine and determine the distribution of questions based on learning outcomes and cognitive categories of Bloom’s taxonomy. In the qualitative part, semistructured interviews were conducted with the purpose of obtaining teachers’ experiences in and opinions about using the mobile app.
Design and development of mobile learning application
Since the dialogue and communication between teachers and students is not always as desired, it was thought to create a digital tool, namely a mobile application, that would facilitate and be useful in solving mathematics problems. The mobile application called “Matematik Cepte (Mathematics in Pocket)” was an innovative application that provides versatile communication between target groups, exchanges mathematics questions and answers, creates and manages a pool of mathematics questions and answers based on specific mathematics topics, and ultimately creates a learning community. The impact of this learning community depends on mathematics teachers' involvement and commitment to their important role in providing feedback. In times of urgent need, the presence of feedback and assistance is essential to reinforce students' motivation and success. This mobile app, which was free of access upon registration and was available for users in partner schools, was expected to be useful in developing this community action and bridging the gap between students' need for help and their success in solving math problems. The following stages for the design and production of a mobile application based on DBR are explained.
Analysis
User analysis
Needs analysis was conducted for all beneficiaries based on user experience (UX).
System analysis
Technical requirements were determined.
Task and content analysis
The math question submission, solution and feedback process were determined.
Design and development of the technological innovation  Mobile app
Interface design
Designing and creating mathspecific mobile user interfaces to increase user friendliness according to user preferences. According to the user ergonomics of a mobile device, form factors such as screen size, minimizing keystrokes, offering image sharing and being taskoriented were taken into account.
Bidirectional system operation and pool system
This phase was about the design with regards to how the application would function, how its related elements would be arranged, and how users would fulfill their tasks. Designing the pool tool to enable registered users to reach and share all the questions and answers saved in the system.
Data labelling system
All questions (as in image files) sent by students were solved, labelled and submitted to the question pool by teachers according to specific mathematics topics. The categorization of questions based on labelling process helped learners to easily search the math topics in the pool and access the relevant questions and their solutions. The learning domains and sublearning domains in the high school mathematics curriculum (Milli Egitim Bakanligi, 2018) (Table 1) were taken into consideration to create a list of math topics.
In order to increase the effective use of mobile app, more math topics were included in the learning domains. For instance, Greatest Common Divisor (GCD), Least Common Multiple (LCM) and Absolute Value subjects which are in the sublearning domain of "Equities and Inequalities" were presented separately in the app. Thanks to this topic classification, it tried to create a structure that is more useful for students and teachers in which they can access questions about a particular topic more easily. Thus, there were a total of 61 mathematics topics in the application (Table 2). While creating the topics, it was aimed that the students and teachers understand the content of the question more easily.
Integration of system user analytics
Considering system security according to determined criteria, this phase is about designing a mobile platform as well as an associated 'web platform' (http://math.dijitaladam.com/login.html) to manage user registration and analytics that provide statistical information. Table 3 and Figure 1 give some examples for statistical information, or findings. However, the purpose of the present study is different and will not focus on the findings indicated in the tables.
Testing and refinement
The following three procedures were carried out for mobile application product verification with user groups, and the formative evaluation was not limited to this process, but was applied continuously (Table 4).
Ethical principles
One of the principal design features of this mobile app focuses on the security of data within a closed platform where registration is required with the information (participants’ names, email addresses, ID numbers) obtained officially from schools. Communication and interaction among students via mobile devices may raise some ethical issues (for example, bullying, racism, and stalking), so it was essential to design a control mechanism and a closed platform for a learning community purpose to prevent unacceptable practices by users.
Implementation of the mobile app
This stage was the usage of the end product mobile platform by the target audience. A web platform indicated above was also developed for the administrative purposes (Figure 2). A mobile application that works on all mobile platforms (Apple, Google Play) was developed, where teachers shared solutions to questions which students were unable to solve or understand. In addition to submitting questions and answers, this application also helps in collecting desired statistical data. After the teacher's feedback reached to the student, the application automatically sends a satisfaction survey and this survey is saved.
Students
The mobile application provides students with the opportunity to ask questions anytime, anywhere and to easily access the solutions of mathematics questions that they have difficulty in solving. Students also create a pool of questions by sending their unsolved math questions to this application (Figure 3).
In this question pool, students can access the questions categorized according to mathematics topics from any smart mobile phone or tablet with an Internet connection, and ask their questions without time constraints (Figure 4).
Students can access the answers to their own questions, as well as the questions and answers sent by other students (Figure 5).
Teachers
Teachers had a critical role in this mobile app, especially by providing support and feedback on students' math questions. It was believed that teacher support is an important predictor of students’ success. Students' perceived teacher communication and support was found to be the factor most closely related to their success in high school (Gregory and Weinstein, 2004). In order for teachers to provide a support and feedback to the students’ unsolved math questions, teachers, they were registered to this mobile application platform. Teachers were able to review all questions submitted by students in the system. They picked and took a math problem and submitted its solution as an image file, sometimes providing more than one solution, to the mobile platform. While sending the solution to the system, the teachers were required to match the related question with an appropriate mathematics topic from the given list and send it by labeling it.
Once a question was taken by a teacher to solve, other teachers could not see that question in the question pool unless it was reloaded by the teacher who did not want to continue working on it. However, the administrator had the right to warn the teacher to make a decision whether to work or not on a question in an appropriate timeframe. Not only could teachers have an access to their own solutions/answers in the system but also the pool including all questions and their answers. The following Figure 6 explains the procedure in visual.
Teachers generally provided students with one or more solutions of a math problem as much in detail as possible as shown in Figure 7.
With this application, students can become the managers of their own time without wasting time. In addition, it saves teachers and students from the trouble of stacked questions that are expected to be solved. In this way, teachers and students do not have to make a separate time plan for solving the questions. Since there is more than one teacher in the system, questions are solved and answered quickly, and a teacher's workload is lighter than normal and shared by other teachers in the mobile application.
Sampling / participants
The target audiences of the mobile app were high school students from 9th to 12th grade studying at nine high schools located in the Karesi district of Balikesir province in Turkiye, and mathematics teachers and preservice teachers. 1041 students and 131 teachers actively used the mobile application by sharing questions and answers.
Since the research questions of this study were directly related to the analysis of the math questions submitted by the students and the answers by the teachers, the statistical population of the quantitative part of this study included more than 116,988 questions in the mobile application platform. The "Numbers and Algebra" learning domain, where the most questions were asked, was discussed in order to determine which learning outcomes the questions were most related to and how their cognitive levels were distributed. Questions were selected from the question pool belonging to the 'Numbers and Algebra' learning domain of the mobile application. From the questions asked in this learning domain, 294 questions for sampling purpose were randomly selected by using the Rand Between function of MS Excel.
The sample group in the qualitative part consisted of six mathematics teachers. They were selected based on their volunteering and experience of using the developed mobile application.
Data collection and instruments
The mobile learning platform provides user analytics and statistics about the questions and solutions submitted by students and teachers. The platform stores all shared image files of questions and solutions. Therefore, research data were collected through the mobile application/platform. In order for 294 questions to be analysed, 14 forms (two forms for each expert in seven groups), which contained 42 questions submitted by the students and a list of learning outcomes covering the learning domain of Numbers and Algebra, was developed to obtain opinions of experts. Semistructured interviews were conducted to obtain teachers’ opinions and experiences in using the developed mobile app.
In the interview form, there were questions about getting general opinions on the use of the mobile application, its pedagogical and technological use, and learning experiences of students.
Data analysis
As mentioned previously, high school mathematics curriculum has three learning domains: i) Numbers and Algebra, ii) Geometry, and iii) Data, and Counting and Probability (Table 1). Each learning outcome in the high school mathematics program is numbered in a sequential order. For example, the learning outcome numbered as 9.3.2.1 refers to 9 as grade level, 9.3 as sublearning domain, 9.3.2 as mathematics subject, and 9.3.2.1 as learning outcome (Figure 8). The numbering of a learning outcome is depicted as follows.
Expert opinions were taken to determine which learning outcomes questions in the sample were associated with. 294 questions belonging to the 'Numbers and Algebra' learning domain were arranged so that seven groups would receive an equal number of questions from each math topic (42 questions per group). For example, each group had three questions on logic, two questions on numbers, and two questions on limits. 14 preservice mathematics teachers were assigned to a group in pairs and the learning outcomes with their explanations of the 'Numbers and Algebra' learning domain was given to them (Table 5). They were asked to determine which outcome the questions belonged to without interacting with each other.
Seven questions that were determined not to belong to the Numbers and Algebra learning domain were excluded from the study. The forms evaluated by the two preservice teachers in each group were compared with each other, and the questions stated to have the same learning outcome were coded as consistent, while the questions stated to have different outcomes were coded as inconsistent. 100% consistency was achieved in 180 questions.
After the learning outcomes measured by the questions were determined, the researcher and a Mathematics teacher came together to determine which cognitive level each outcome was for. The cognitive process dimension of Bloom's Taxonomy (Anderson and Krathwohl, 2001) was taken as a basis in determining the cognitive levels (that is, remembering, understanding, applying, analyzing, evaluating and creating) (Figure 9). In the process of determining cognitive levels, discrepancies were observed between expert opinions in 9 items. Experts came together, revised these items, resolved any difference and made a joint decision based on the structure of the content and the questions asked about the relevant outcome. Data about learning outcomes were analyzed descriptively using frequencies and percentages.
180 questions were revised again later to analyze the distribution of cognitive levels of Bloom’s Taxonomy based on high school grades. 18 questions addressing more than one grade level were eliminated and 162 questions were analyzed based on the agreement among the coders. Frequencies and percentages were calculated over the questions of the same grade level. In this analysis stage, questions in relation with the learning outcomes of the basic level mathematics curriculum was also included in the calculations. The basic level curriculum aims to enable students to actively benefit solving skills is one of the main goals of the from mathematics in their daily and business life after graduation. It is predicted that students who do not prefer a mathematicsbased major at a higher education level will more effectively overcome the problems they encounter in real life. Developing students' problem program (Milli Egitim Bakanligi, 2018). At the qualitative level, thematic analysis was used to analyze interview data about teachers' experiences and views on how they perceived learning mathematics using the mobile application.
FINDINGS
In this section, research findings are presented based on the research questions.
In which mathematics subjects and learning domains did students have the most difficulty in problem solving and sought support for their solution?
To answer this research question, the frequency and percentage values of the descriptive statistics obtained from the system were calculated. As a result of the evaluations made with the teachers, it was decided that 567 questions uploaded to the system contained errors and were not included in the analysis. Therefore, 116,988 questions registered in the system were examined. It was found that the total number of questions asked in the 'Numbers and Algebra' learning domain was 93876 (80.24%), 20553 (17.57%) in the 'Geometry' learning domain and 2559 (2.19%) in the 'Data, Counting and Probability' learning domain (Table 6). The 'Numbers and Algebra' learning domain was found to be the dominant with a large difference comparing to others. It is seen that students had the most difficulties in questions of Numbers and Algebra domain. Moreover, the first five mathematics subjects that students had the most difficulty in solving the questions and demanded solutions for them were determined as follows: 1. Derivative (n=28329, 24.22%), 2. Numbers (n=23367, 19.97%), 3. Problems (6639, 5.67%), 4. Trigonometry (n=6302, 5.39%), and 5. Quadratic equations and inequalities (n=4093, 3.50%) (Table 6).
At which cognitive levels were the learning domain questions that students had the most difficulty with?
It was found that the most questions sent by students on the mobile platform came from the 'Numbers and Algebra' learning domain. It was aimed to determine at which cognitive levels the questions of 9th, 10th, 11th and 12th grades related to this learning domain occur according to the cognitive process dimension of Bloom’s Taxonomy. The 180 questions were consistent between the groups related to the learning outcome analysis, and their level according to Bloom’s Taxonomy was tabulated according to frequency and percentage values (Table 7).
While there are more questions at the level of remembering, understanding and application (n=156 in total) that require lowlevel thinking; there are fewer questions that require highlevel thinking at the level of analysis, evaluation, and creation (n=24 in total).
According to this result, it can be concluded that students have problems in understanding or solving questions that require lowlevel thinking. It can be said that there are fewer questions that require highlevel thinking due to the fact that these questions may be scarce in the source books they used. Sample questions can be seen in Figure 11.
What is the distribution of these cognitive levels among classes in high school?
The question groups belonging to categories of the cognitive process dimension were interpreted by the researcher. Then, frequency and percentage tables were created by associating the questions with those according to the grade level. As indicated above in the data analysis section, 162 questions were analysed to answer this question.
According to Bloom's Taxonomy levels, it was determined that the most questions out of 162 questions were at the level of applying and understanding. For the 9th grade, 28 questions from the understanding level and 33 questions from the applying level were asked by the students, while for the 12th grade, eight questions from the understanding level and 35 questions from the applying level were asked (Figure 10). In line with these results, it was seen that the students in the 9th grade have deficiencies in understanding and applying the mathematical questions, while in the 12th grade they have more problems in applying rather than understanding the math questions.
According to the grade levels, it was determined that questions coming from the 9th and 12th grades were the most out of 162 questions. The reason for this may be that Numbers and Algebra domain takes up more space in the High School mathematics curriculum in the 9th and 12th grades compared to other learning domains.
In this learning domain, what math learning outcomes and associated cognitive level did students submit the most questions about?
Percentages were used to present the number and description of learning outcomes associated with cognitive levels belonging to each grade level (Appendixes 1,2,3,4,5). Learning outcomes were grouped according to sublearning domains (9.1, 9.2, and 9.3). Students of the 9th grade asked most questions in relation with the learning outcome – “9.2.2.1 Able to solve problems with the help of union, intersection, differenceand complementation processes in sets” (46.66%), which was at the Analyze cognitive level. Students of the 10th grade asked most questions in relation with the learning outcome – “10.3.2.1 Able to factorize a polynomial” (46.66%), which was at the Analyze cognitive level.
Students of the 11th grade asked most questions in relation with the learning outcome – “11.3.3.1 from the graph of a function, she/he can draw new function graphs with the help of transformations” (66.66%), which was at the Apply cognitive level. Students of the 12th grade asked most questions in relation with the learning outcome – “12.2.1.3 Able to perform operations using the properties of arithmetic and geometric sequences” (58.82%), which was at the Apply cognitive level. Students of the BS of 11th grade asked most questions in relation with the learning outcome – “BS.11.1.1.2 Able to solve problems related to natural numbers” (50.00%), which was at the Apply cognitive level.
What are the teachers' experiences and views on using the mobile learning application?
Thematic analysis was conducted under three core themes: general views on the use of the mobile application, its pedagogical and technological use, and students’ learning experiences.
General views of teachers on the use of the mobile application
Teachers stated that the mobile learning application is very useful, especially since it allowed them to solve many different math questions. In fact, it has been stated that the main purpose of using this application by teachers is to practice problem solving. It has been deduced that the teachers see the mobile application as a question bank containing alternative solutions from different perspectives. Example quotes with regards to this issue as follows:
“I benefit from solving questions and the solved questions in the question pool.” “It's a nice application to spend time in my spare time. When I'm tired, instead of picking up a pen and paper, pick up the phone and solve it over the phone.”
“We also see different perspectives from our own perspective.” “As a teacher, I see different questions and improve myself. A great resource as a question bank” “… at least I am useful in my spare time so I feel happy”
One teacher also indicated that she “began to see more clearly the subjects that students had the most difficulty with” by using this mobile learning app. Teachers stated that the most liked features of this application are that it resembles a private lesson, like a oneonone lesson, and being a part of a learning community. Example quotes with regards to this issue as follows:
“…kind of onetoone lessons... helps those who don't have the opportunity to ask questions in the classroom...”
“I think that the children who use the app and myself are also part of the learning community. I find the app very useful”
“It definitely gives an experience, it's a privilege to even be a part of it, it is like a social network”
Pedagogical and technological use
Teachers reported that they were impressed with the student feedback process provided by the mobile app. Some responses stated that the mobile app as a feedback and support system has the advantage of increasing student engagement in the learningteaching process by interacting with the platform and sharing their questions regardless of time and place. Example quotes with regards to this issue as follows:
“…it has positive effects. The student cannot find a teacher at home on issues that he or she is missing at home, thanks to this application, they can reach the solution of questions from home and learn the technique of problem solving…”
“…they are definitely getting positive feedback and they say that this application adds a lot to them”
“The fact that the application answers the questions quickly increases the motivation of the students”
“At least she knows that if she has a question, there is a place where she can get an answer at any time, any hour”
Teachers also reported that they developed their teaching skills and methods by using this mobile app.
“The more questions we solve, the more different our instruction becomes”
“I didn't use an app like this, I used online applications in the classroom, I can help the individual student directly here through this app”
“…I also use the questions uploaded to the app in the classroom… interesting and good questions come up, for example, about the problems…”
Students’ learning experiences
When asked what effect the mobile application had on students' learning, teachers generally stated that it had a positive effect.
“Of course, there is an effect. Having learned the solution of the problem, the student can now solve similar questions”
“A useful application for children who use it for its intended purpose and for those who want to learn, it always provides the opportunity to learn...”
“It has a great effect when they receive feedback; it has positive effects as it allows them to make up for their shortcomings at home”
“Kids these days live fast. They want to solve the question that comes to mind immediately. It helps them with this”
“She solves the questions she cannot solve and learns the shortcuts”
“…absolutely helpful... I don't understand the opinions of those who didn't use it”
In contrary, some teachers stated that the effect felt was not as great as it seems.
“Students do not understand anything from the solution, sometimes they come with very simple questions”
“…some students' phones are not suitable”
According to the answers given to the question of how using this application improves students, teachers saw the application as a very helpful and useful tool.
“They may stop solving questions they have difficulty with, or they may stop working on them. So, they can give up. However, they know that they are not alone and they continue to take part in the process as long as they get the solution to their questions.”
“…they learn question techniques and how to approach the question”
“Solving those questions increase the selfconfidence of the students”
“My students like the application; they are positively motivated when they see that it is useful”
CONCLUSION
High school students, especially students in their final year preparing for the university entrance exam, were solving a large number of mathematics questions. There is a need for teacher support regarding the solutions of those questions.
A mobile application was developed in this study with the purpose of providing this support at this point. According to previous research results, interactive mobile learning apps improve students' mathematical problemsolving skills (Amir et al., 2018; Amir et al., 2020).
Mobile devices along with an appropriate application can provide a learning context where students can create and share knowledge for better learning outcomes (Melhuish and Falloon, 2010; Haßler et al., 2016). Achieving learning outcomes has an impact on students' learning processes. For this reason, it is of great importance to determine the learning outcomes correctly. It is also important which cognitive skills and levels are included in mathematics teaching.
According to the results of the study, it was seen that the learning outcomes were mostly directed towards the lowlevel cognitive skills of Bloom’s Taxonomy. Although it was seen that the students were able to reach Evaluate level, it was observed that they mainly shared questions at the Apply and Analyze levels. This finding suggests that educational approaches or policies focusing on highlevel cognitive skills should be taken as a basis. The positive impact of using technology throughout the curriculum can help students develop higherorder thinking skills that can help them learn math beyond the classroom (Murphy, 2016).
Mobile learning has been included in research literature as a developing topic in the context of teacher education (Kearney and Maher, 2013). It has been observed that teachers who solved questions in the system improved themselves in terms of different question types and alternative solution methods. In addition, it was observed that the teachers prepared new notes for the questions frequently asked by the students, contacted the students again on these issues, and made additional lessons. According to the findings of the qualitative part, it was observed that the teachers' use of the mobile application was high and they were willing to apply mobile technology in mathematics education, especially in providing their support for solving difficult math questions.
They stated that the mobile app has the potential to become an effective solution for supporting high school students in developing selfconfidence and selfefficacy, which may lead to an academic success. This finding is in line with the research study by (Yu and Singh, 2018). The authors reported that teacher support through positive interaction between teacher and students has a strong and positive effect on students' selfefficacy and therefore indirectly affects students' mathematics achievement. Consequently, the teachers showed great enthusiasm and satisfaction in the use of this innovative application.
The educational capabilities of mobile devices offer learning activities for a better understanding, which were previously not possible (Cumming et al., 2014; Montrieux et al., 2016). This study is in line with the research by Jeng et al. (2010) showing that mobile apps provide a convenience in learning in the daily lives of student. Based on the results of the study, it can be concluded that the mathematics mobile learning application can be used as an alternative learning environment that can give an idea about the learning outcomes and cognitive levels of high school students.
LIMITATIONS AND FUTURE RESEARCH
Although the present study has important implications, there are some limitations as well. The sample of the study was limited to students studying at and teachers working in nine high schools in one district of Balikesir province. Therefore, future studies should be expanded to include more students and teachers from more high schools. This study did not investigate students' perception of the use of and engagement with the app. Moreover, the study did not take into account internet connection quality and students’ digital literacy and skills. It could be thought that not all students could afford the accessibility to mobile learning experience (Sabah, 2016). Moreover, the current study did not report any findings about whether or not all students benefitted from and enjoyed the developed mobile app. Therefore, future studies should conduct quantitative and qualitative methods to gain deeper understanding about the influences of the app on students’ mathematical problem solving and higherorder cognitive skills. Another limitation of the study was that it did not focus on whether the mobile app has an effect on students’ mathematics academic achievements. Factors related to the impact of mobile learning on student achievement can be identified and examined, so that the mobile application can be redesigned and developed accordingly. Further improvements of the application in terms of enhancing functionality and usability as well as ensuring longterm user engagement should be considered and tested.
The questions and answers in the system have created a question pool  a question bank. According to the statistical data available from this pool, it was determined which mathematics subjects were asked the most questions. In this direction, it can be revealed what kind of deficiencies there are on the basis of subject in the lessons in schools. Hence, a mathematical map can be drawn about which subjects are not understood on the basis of schools and which subjects the mathematics teacher should focus on. Therefore, it can be determined which region is deficient in which subjects on the basis of districts. This could be an internal control and studies can be planned by investigating why the most difficult math topic is difficult, whether enough time is allocated, whether the necessary support is provided or whether it is due to the complexity of the topic. After these analyzes are done, the learning activities that need to be developed can be shared and discussed with the mathematics teachers working in the schools. As a result, it may be possible to determine and improve mathematical literacy on the basis of individual student, schools and districts.
Many teachers and students are now more experienced and willing to use mobile technology, which is more affordable and accessible than it was a decade ago. It is clear that students and teachers are already using mobile technologies to create their own time and space to practice a more flexible learning (Traxler, 2009).
We are only at the beginning of exploring the use of these promising mobile technologies in education. Technology that supports mobile learning is changing very fast. This study can be an initial study for designers, experts and teachers in mobile technology. The findings in this study are among the many topics in this rapidly evolving field. There are many opportunities and potentials waiting for us to realize. Finally, teachers are expected to continue to use everevolving mobile technologies in new ways, not just in the classroom, but outside the classroom anytime, anywhere to help students prepare for their future careers and lives.
CONFLICT OF INTERESTS
The author has not declared any conflict of interests.
ACKNOWLEDGMENTS
This work was supported by the Matematik Cepte project funded by the Karesi Municipality. The author gratefully thanks to Dr. Ikikardes and Dr. Ates from the department of mathematics at Balikesir University for their support during the design and development of the mobile application and for their helpful discussions in mathematics learning.
REFERENCES
Alden J (2013). Accommodating Mobile Learning in College Programs. Journal of Asynchronous Learning Networks 17(1):109122. 

Amir MF, Hasanah FN, Musthofa H (2018). Interactive Multimedia Based Mathematics Problem Solving to Develop Students' Reasoning. International Journal of Engineering and Technology 7(2.14):272276. 

Amir MF, Ariyanti N, Anwar N, Valentino E, Afifah DSN (2020). Augmented Reality Mobile Learning System: Study to Improve PSTs' Understanding of Mathematical Development. International Journal of Interactive Mobile Technologies 14(9):239247. 

Anderson LW, Krathwohl DR (Eds.) (2001). A taxonomy for Learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Addison Wesley Longman. 

Anderson T, Shattuck J (2012). Designbased research: A decade of progress in education research? Educational Researcher 41(1):1625. 

Baya'a NF, Daher WM (2009). Learning mathematics in an authentic mobile environment: The perceptions of students. International Journal of Interactive Mobile Technologies 3(1):614. 

Bray A, Tangney B (2016). Enhancing student engagement through the affordances of mobile technology: a 21st century learning perspective on realistic mathematics education. Mathematics Education Research Journal 28(1):173197.? 

CorneliusWhite J (2007). Learnercentered teacherstudent relationships are effective: A metaanalysis. Review of Educational Research 77:113143. 

Crompton H, Burke D (2017). Research trends in the use of mobile learning in mathematics. Blended learning: Concepts, methodologies, tools, and applications 20902104. 

Cumming TM, Strnadova I, Singh S (2014). iPads as instructional tools to enhance learning opportunities for students with developmental disabilities: An action research project. Action Research 12(2):151176. 

Davies BS, Ra"que J, Vincent TR, Fairclough J, Packer MH, Vincent R, Haq I (2012). Mobile Medical Education (MoMEd)  how mobile information resources contribute to learning for undergraduate clinical students  a mixed methods study. BMC Medical Education 12(1). 

DesignBased Research Collective (2003). Designbased research: An emerging paradigm for educational inquiry. Educational Researcher 32(1):58. 

Fabian K, Topping KJ, Barron IG (2016). Mobile technology and mathematics: Effects on students' attitudes, engagement, and achievement. Journal of Computers in Education 3(1):77104. 

Gan CL, Balakrishnan V (2017). Enhancing classroom interaction via {IMMAP}  an interactive mobile messaging app. Telematics and Informatics 34(1):230243. 

Geiser S, Santelices MV (2007). Validity of highschool grades in predicting student success beyond the freshman year: Highschool record vs. standardized tests as indicators of fouryear college outcomes. Research & Occasional Paper Series. Centre for Studies in Higher Education, University of California. 

Gikas J, Grant MM (2013). Mobile computing devices in higher education: student perspectives on learning with cellphones, smartphones and social media. The Internet and Higher Education 19:1826. 

Guskey TR (2010). Formative Assessment: The Contributions of Benjamin S. Bloom. In Andrade HL, Cizek GJ (Eds.), Handbook of Formative Assessment. Taylor & Francis, New York, NY. Pp. 106124. 

Gundogdu K, Kiziltas E, Cimen N (2010). Seviye belirleme sinavina (sbs) iliskin ogrenci ve ogretmen gorusleri (Erzurum il ornegi), Elementary Education Online 9(1):316330. 

Haßler B, Major L, Hennessy S (2016). Tablet use in schools: a critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning 32(2):139156. 

Hattie J (2009). Visible learning: A synthesis of over 800 metaanalyses relating to achievement. London: Routledge. 

Hattie J, Timperley H (2007). The power of feedback. Review of Educational Research 77(1):81112. 

Hegedus SJ, Dalton S, Tapper JR (2015). The impact of technologyenhanced curriculum on learning advanced algebra in US high school classrooms. Educational Technology Research and Development 63:203228. 

Huang YM, Hwang WY, Chang KE (2010). Guest Editorial  Innovations in Designing Mobile Learning Applications. Educational Technology and Society 13(3):12. 

Jeng YL, Wu TT, Huang YM, Tan Q, Yang SJH (2010). The AddOn Impact of Mobile Applications in Learning Strategies: A Review Study. Educational Technology and Society 13(3):311. 

Kalloo V, Mohan P (2012). Correlating Questionnaire Data with Actual Usage Data in a Mobile Learning Study for High School Mathematics. Electronic Journal of eLearning 10(1):7689. 

Kearney M, Maher D (2013). Mobile learning in math teacher education: Using iPads to support preservice teachers' professional development. Australian Educational Computing 27(3):7684. 

Klem AM, Connell JP (2004). Relationships matter: Linking teacher support to student engagement and achievement. Journal of School Health 74:262273. 

Melhuish K, Falloon G (2010). Looking to the future: Mlearning with the iPad. Computers in New Zealand Schools: Learning, Leading, Technology 22(3):116. 

Milli Egitim Bakanligi (2018). Ortaogretim matematik dersi (9, 10, 11 ve 12. siniflar) ogretim programi. Ankara: Talim ve Terbiye Kurulu Baskanligi. 

Montrieux H, Schellens T, Van Landeghem J, Mouton T (2016). Introducing tablet devices during mathematics education: "is it really a magical learning tool?" PONTE: Multidisciplinary Journal of Science and Research 72(7):393410. 

Murphy D (2016). A literature review: The effect of implementing technology in a high school mathematics classroom. International Journal of Research in Education and Science (IJRES) 2(2):295299. 

National Mathematics Advisory Panel (2008). Foundations for success: The final report of the National Mathematics Advisory Panel. Washington, DC: U.S. Department of Education. 

Nedungadi P, Raman R (2012). A new approach to personalization: integrating elearning and mlearning. Educational Technology Research and Development 60:659678.? 

Nicol DJ, MacfarlaneDick D (2006). Formative assessment and self?regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education 31(2):199218. 

Reeves TC (2006). Design research from the technology perspective. In Akker JV, Gravemeijer K, McKenney S, Nieveen N (Eds.), Educational design research (pp. 86109). London: Routledge. 

Roorda DL, Koomen HMY, Spilt JL, Oort FJ (2011). The influence of affective teacherstudent relationships on students' school engagement and achievement: A metaanalytic approach. Review of Educational Research 81(4):493529. 

Ryan AM, Patrick H (2001). The classroom social environment and changes in adolescents' motivation and engagement during middle school. American Educational Research Journal 38(2):437460. 

Sabah NM (2016). Exploring students' awareness and perceptions: Influencing factors and individual differences driving mlearning adoption. Computers in Human Behavior 65:522533. 

Traxler J (2009). Learning in a Mobile Age. International Journal of Mobile and Blended Learning 1(1):112. 

Wadlington E, Wadlington PL (2008). Helping students with mathematical disabilities to succeed. Preventing School Failure: Alternative Education for Children and Youth 51(1):27. 

Wang X (2013). Why students choose STEM majors motivation, high school learning, and postsecondary context of support. American Educational Research Journal 50(5):10811121. 

Yu R, Singh K (2018). Teacher support, instructional practices, student motivation, and mathematics achievement in high school. The Journal of Educational Research 111(1):8194. 

Zoraini WA, Chng LP, Norziati M (2009). A Study on Learner Readiness For Mobile Learning At Open University Malaysia. Sanchez IA, Isaias P (Eds.), IADIS International Conference Mobile Learning (pp. 151157). New York: Curran Associates, incorporated. 
Copyright © 2023 Author(s) retain the copyright of this article.
This article is published under the terms of the Creative Commons Attribution License 4.0