Educational Research and Reviews

  • Abbreviation: Educ. Res. Rev.
  • Language: English
  • ISSN: 1990-3839
  • DOI: 10.5897/ERR
  • Start Year: 2006
  • Published Articles: 2008

Full Length Research Paper

Examination of the PISA 2009 reading skills and information and communication technology (ICT) use skills of Turkish students

Tulin ACAR
  • Tulin ACAR
  • Measurement and Evaluation, Turkey.
  • Google Scholar


  •  Received: 22 May 2015
  •  Accepted: 19 June 2015
  •  Published: 10 July 2015

 ABSTRACT

The aim of this paper is to determine relation between PISA Reading Skills and ICT use skills of Turkish students. In this study are four variables such as joy/like Reading, use of Libraries, Online Reading and Plausible value in reading which are dealt with as indications of reading skills. It constitutes six variables such as attitude towards computers, ICT internet/entertainment use, ICT for school related tasks, ICT availability at home, ICT availability at school, use of ICT at school. They are dealt with as indications of ICT use.  Canonical correlation analysis is used to determine the relation between reading skills and ICT use skills. A high level significant relationship is found between 2009 PISA Reading skills and ICT use skills of Turkish students.

Key words: PISA 2009 reading skills, communication technology use skills, canonical correlation.


 INTRODUCTION

Education is definitely a systematic process in which permanent patterns of behavior have been created in individuals. As for teaching, it is total sum of processes practiced for carrying out learning and development of behaviors which are desired in individual (Özçelik, 1998). Thereby, such terms as teaching, education, teacher, learning and student play a significant role in educational surveys. According to Morgan (2004), terms of learning is defined as continuing change which exists as a result of review or experience. As for term of teaching, it can be thought as learning activities of an individual.

Education system is not a static structure. Thereby, learning- teaching processes in education show various changes in accordance with times. In fact, many more terms such as e-learning, e-education, computer-assisted education, individualized tests, smart boards, education technologies, information communication technologies have come to enunciate frequently for last 20 years (A?kar, 1992; Güngör and A?kar, 2004; Ziya et al., 2010). Especially, an increase in computer use in daily life has also increased use of computer in education. However, it is observed that Turkey falls behind regards to literacy of the information communication technologies use in education compared to developed countries and countries which are members of OECD (Çetin, 2011). In his study, A?kar (1992) stated the purpose of information communication use in education as follows: 1-Keeping students informed on computer technologies and use it as an instrument 2- Supporting learning-teaching.

Using information and communication technologies has become an integral part of everyday life. It is possible to define information and communication technologies as audiovisual and written tools used for generation and transmission of knowledge. Tools of information and communication technologies are being used in several fields such as health, law, art, and education. Particularly in educational settings, information and communication technologies have become an indispensible tool in developing knowledge and skills of students, in other words, as part of teaching and learning. As tools of information and communication technologies have become a part of learning, the impact of such tools on student's performance is extremely important for pedagogues, because today, it is one of the objectives of education to develop children's thinking skills in reading, mathematics and science, role of information and communication systems in attaining these objectives is undeniable. Particularly distance education systems have increased the use of information and communication technologies.

Program for International Student Assessment (PISA) is a screening survey conducted for evaluation of information and skills that group of 15 age students have acquired by Organization for Economic Cooperation and Development (OECD) every three years (OECD, 2012).  PISA Project does not aim what degree group of 15 age students who continue formal education following compulsory education have taught subjects (mathematic, sciences and reading skills) within education program but aims at measuring information and skill using abilities in certain circumstances they encounter in today’s infor-mation society (OECD, 2012). Within this scope, student and school questionnaires for measuring the use of information communication technologies have been carried out in PISA. New questions have been added to the surveys with regard to how students around the age 15 use new technologies; and how they use new technologies for internet connection and entertainment.

Focal assessment framework of PISA implemented in 2009 was on reading skills of students. The assessment framework is changed to mathematics, science and reading in turns in every three years. The concept of literacy (reading skills) used in PISA in 2009 was defined as competence of students in using knowledge and skills, analyzing, making logical conclusions, and engaging in effective communication in interpreting and solving the problems they encounter in various circumstances in the domain of the basic topic (PISA, 2009; National Preliminary Report, 2010). Especially, researches carried out by Program for International Student Assessment-PISA at international level have also provided striking data in terms of many country. 


 LITERATURE REVIEW

Research shows that even though students and  teachers in Turkey do not much skilled in terms of using information and communication technologies, it is remarkable that they have positive attitudes and opinions towards using information and communication technologies (Cüre and Özdener, 2008; Yavuz and Ço?kun, 2008; Kayaduman et al., 2011; Ayvac? et al., 2014).

It is possible to see lots of research by which the effects of presence and usage frequency of information and communication technologies have on harmony with life, success, and attitudes are studied. For instance, it is seen that usage of information and communication technologies makes a significant difference in competence of students in science and mathematics (Bal?m et al., 2009). In addition, it is found by Acar (2012) on the data of Turkey that the variable of using computer technologies had a significant negative impact on 2009 PISA fields of reading, mathematics, and science.

In the research about results of PISA 2003, A?kar and Olkun (2005) have emphasized that an access to computer at schools is low in Turkey compared to countries of OECD and problem solving skills of students who have an access to computer at school and at home are higher than students who have not an access to computer. Gürsakal (2012) emphasized in her study that periods of computer use by students at home and in school, their strategies of study, and education levels of their parents are predictive variables in reading, science and mathematics performance of students. According to study conducted by Ziya et al. (2010) it is stated that students’ self confidence on processing which require good command of computer knowledge,  use of word processing and electronic spreadsheets programs and internet use have an negative effect on mathematic success level. Güzeller (2011) stated that self-sufficiency belief of Turkish students towards computer who have participated in PISA 2009 project shows no difference in accordance with sex, but there is a significant difference on attitudes relevant to computer according to sex and this difference is not significant in practice.

Delen and Bulut (2011) found that Turkish students' frequency of using information and communication technologies at home and school is a strong predictor of their performance in science and mathematics according to PISA 2009 data. Gümü? and Atalm?? (2001) emphasized that computer use for learning purposes has a negative effect while computer use for entertainment purposes has a positive effect on reading performance according to PISA 2006 data of Turkey. It is observed in PISA 2000 data that Canadian students who have a high performance of reading have a correlation between the use of internet and computer at home. However, a negative correlation is observed between the variable computer use in library and reading performance of Canadian students (PISA Canada). The study conducted by Biagi and Loi (2012) on the correlation between ICT variable  and   PISA   performance   in   the   data   of   23 countries participating in 2009 PISA offers information of universal value. That is the results of the estimates presented in this report point to a generalized negative correlation between the use of ICT (in terms of either intensity or deviations from the mean) and PISA test scores.

It is also possible to see the effects of several variables such as demographic, cultural, socioeconomic ones on science, mathematics and reading performance of students. It is remarkable that studies of Shelley and Y?ld?r?m (2013), in particular, reveal significant predictors of mathematics and science performance variables on reading performance of students. Therefore, it is considered that reading skills have important effects on the skills of science and mathematics, and the skills of ICT use have important effects on reading skills, because the use of the tools of information and communication technologies in communication among people in everyday life. Communication is essentially a part of a process that involves comprehension, expression or transmission of a piece of information, and solving problems. Communication is a skill and superior quality of this skill will certainly facilitate learning processes.

 

Aim of this study

Aim of this study is to determine relation between PISA Reading Skills and ICT use skills of Turkish students. Frequency of similar studies in literature makes a significant contribute to the education policies of countries in interpretation of PISA results.   


 METHOD

The population and the sample

The population of this study constitutes group of 15 age students receiving an education in Turkey.  As for the sample of this study, it constitutes 4996 students determined with random method by PISA international center from total sum of 170 schools by stratification of 56 cities and school types from 12 statistical region units (Ministry of Education [MEB], 2011). Of these students, 48.94% (n=2445) are female, 51.06% are male (n=2551), and 98% study in state schools. The majority, 67.9%, of students are in the 10th grade (n=3393).

 

Data collection

In this study, there is no data collection tool. Data of this study have been acquired from 2009 PISA student questionnaires, ICT for the student questionnaires and official web page of PISA. These are the tools for the study.

 

Data analysis

In date set of student questionnaire,  ten variables such as attitude towards computers, ICT internet/entertainment use, ICT  for  school related tasks, ICT availability at home, ICT availability at school, Use of ICT at school, Joy/Like Reading, Use of Libraries, Online Reading and Plausible value in reading have been discussed. Points of items for such variables were not summed up as a total point. On the contrary, predictor points related to variables were taken directly from the data file of PISA. When missing values in variables have been extracted from the data set, processing continues from 4540 students.

Canonical correlation analysis is used to determine the relation between reading skills and ICT use skills. Canonical correlation is a technique used in determining the relation between two variables set.  Canonic (Kalayc?, 2006:238). In other words, it examines the relation between linear combinations of variables belonging to one set and linear combinations of variables from other set (Tatl?dil, 1992).

The first set in this study are 4 variables such as joy/like Reading, use of Libraries, Online Reading and Plausible value in reading which are dealt with as indications of reading skills. As for the second set, it constitutes 6 variables such as attitude towards computers, ICT internet/entertainment use, ICT for school related tasks, ICT availability at home, ICT availability at school, Use of ICT at school which is dealt with as indications of ICT use.  

Set 1 ( Reading Skills=V): Joy/Like Reading

                        Use of Libraries

                       Online Reading

                       Plausible value in Reading

Set 2 (ICT use skills= U):   Attitude towards computers

      ICT internet/entertainment use

      ICT for school related tasks

      ICT availability at home

      ICT availability at school

      Use of ICT at school


 FINDINGS

The correlations between the variables of computer-related attitude, internet use, ICT use for tasks assigned at school, availability of ICT at home, availability of ICT at school, and correlations among the use of ICT at school, which are considered the indication of the skills of Turkish students of using ICT in PISA 2009, are shown in Table 1.

According to Table 2, there is a positive, medium level, significant relationship between skills of ICT internet/ entertainment use and use of ICT for school related tasks. (r=0,627 p<0.01). In other words, as the use of internet increases, level of ICT use in duties at school also increases or as the use of internet decreases, level of ICT use in duties at school also decreases. Regarding this conclusion, it can be stated that students’ perceiving information communication technologies as a joy results in that they perceive computer technologies at school in a similar way. It is notable that there is no relationship between students’ ICT availability at home and use of ICT at school. (r=0.024 p>0.05). Another result can be stated in a way that there is a relationship close to zero between variables of attitude towards computers and ICT availability at school and Use of ICT at school. In other words, ICT availability at school and level of use  can  not  reflect the students’ attitudes significantly. The correlations of variables taken for PISA 2009 Reading skills are handled and shown in Table 2.

 

 

 

According to Table 3, it can be said that the relationships between variables handled as the indicators of reading skills are generally low. The highest relationship between these variables is between the level of students’ Joy/Like reading and PISA 2009 reading skills performances. However this relationship is a positive, low level significant relationship (r=0,253 p<0.01). In other words, there is a linear relationship between liking reading and reading skills performances of Turkish students taking part in PISA 2009 project.

According to PISA 2009 data, a negative, considerably low level (almost close to zero) significant relationship is found between Online Reading skills and Joy/Like Reading performances of Turkish students (r = -0,068 p<0.01). It is also found that there is a low level, almost close to zero relationship between Online Reading performances of Turkish students and level of Library Use (r=0,054 p<,01). In other words, it can be said that there is no relationship among Online Reading skills and Joy/Like Reading and Library Use skills of Turkish students.

It is observed that there is a negative relationship between the frequency of using library and reading success of Turkish students taking part in PISA 2009 project (r= -0,210 p<0.01). In fact, it is surprising that reading   performances   of   students   who    use   library frequently are low. At this point, the frequency and the purpose of using library by students in schools should be reexamined in terms of student and teacher.  Eigenvalues and canonical correlations related to the canonical correlation between PISA 2009 reading skills and skills of ICT use are specified in Table 3.

According to Table 3, calculated eigenvalue related to canonical correlation pair is the indicator of total variable. As the eigenvalues increase, correlation coefficients also show increase. It is observed that in four canonical correlation pairs calculated, the highest eigenvalue takes place in the first function (1,06). In other canonical correlation pairs, eigenvalues decline steadily. When the significances of canonical correlations found in acquired canonical variable pairs are tested with Wilks Lamda statistics, it is found that four canonical correlation coefficients are significant at the error level 0.01. The highest relationship is observed in the first function (r=0,717 p<0.01). In other words, there is a high level significant relationship between 2009 PISA Reading skills and ICT use skills of students. Again, 51% of reading skills for Turkish students in 2009 PISA is characterized by the skills of ICT use. Raw canonical coefficients regarding first function in canonical variable pairs, standardized canonical coefficients and canonic loadings are shown in Table 4.

Standardized coefficients are those which show amount of variation occurred in canonical variable as standard deviation   in   response   to   increase   in   one standard 

 

 

deviation occurred in original variable. In other words, these coefficients are those showing the amount of original variation effect taking place in set in the formation of canonical variable (Keskin and Özsoy, 2004). According to this, equality regarding U1 canonical variable (reading skills) will be as follows:

U1= 0,191*y1 + 0,011*y2 – 0,948*y3 – 0,126*y4

In the formation of U1 canonical variable, the highest but negative contribution is made by Online Reading with 0.948 contribution amount. It is also observed that the lowest contribution to the U1 (reading skills) is made by Use of Libraries variable. V1 canonical function regarding ICT use skills is as follows:

V1= -0,131*x1 -0,573*x2 -0,176*x3 – 0,349*x4 – 0,040*x5 + 0,091*x6

In the formation of V1 (ICT use skills) canonical variable, the highest contribution is made by ICT internet/entertainment use variable with -0,573 contribution amount. However this contribution is negative. It is also observed that the lowest contribution to V1 variable is made by ICT availability at school.

As it is more appropriate to use correlations between original variables taking place in that set and canonical variable, these correlation coefficients are named as correlation loadings or weight. When canonical loadings of variables in reading skills data set and its own canonical variable (U1) are examined, it is seen that the highest loading value belongs to Online Reading with -0,980. For ICT use skills, the highest factor loading value is ICT internet/entertainment use variable with -0,930.


 DISCUSSION

A high level significant relationship is found between 2009 PISA Reading skills and ICT use skills of Turkish students. A similar conclusion was arrived at by Fuchs and Woessmann (2005) who used PISA 2000 data in their study and found a positive relationship between mathematics and reading performances and ICT use performances of students. However, Steffens (2014) determined PISA achievements do not increase with ICT use in his study. In addition, it is found by Acar (2012) on the data of Turkey that the variable of using computer technologies had a significant negative impact on 2009 PISA fields of reading, mathematics,  and science. Song and Kang (2012) determined in their study on elementary, secondary and higher education students that students with high ICT use skills for data collection and communication have also high academic success. However, Aypay (2010) did not find any significant relationship between ICT use skills and academic success of Turkish students taking part in PISA 2006 project.

In this study, it is observed that as the frequency of internet use of Turkish students taking part in PISA 2009 project increases, the frequency of ICT use for duties in school increases. It can be said that the tendency of students to use internet for e-mail, writing blog or chatting increases their ICT use skills for duties in school. It is also observed in this study that there is a negative contribution of ICT internet/entertainment use variable to students’ reading skills. Therefore, the frequency and purpose of Turkish students to use internet out of school should be examined carefully. Another striking conclusion is that in their study on PISA 2003 data, Sweet and Meates (2004) did not find a strong relationship between ICT use and having a computer in the house and success level in some of the countries. Nevertheless, according to the research by Spieza (2010), the effect of the use of computer at home rather than at school on Science points in PISA 2006 is greater and positive in many countries. In this study, on the other hand, availability of communication technologies at home rather than at school has a greater effect, albeit a negative one, on PISA 2009 reading performance of Turkish students.

It is fair to say that student is more passive in performance of reading printed sources, and more active in online reading. Needless to say, performance of reading printed sources would differ from that of reading online resources. It is notable that Online Reading variable has a negative contribution on reading skills of Turkish students taking part in PISA 2009 project. For this reason as Eraslan (2009) emphasized in his study, it should be our goal to train qualified teachers with ICT use skills. Therefore, teachers who have ICT skills would play a significant role in students' gaining of such skills. Teachers should provide ICT skills such as making online research, using online library to our students. Güngör and A?kar (2004) stated that being in e-learning process affects student’s self-sufficiency perceptions positively. However in this study, it is observed that the availability and the use of ICT did not make positive effect to the attitudes of students.

Another conclusion from the study is that the frequency and the purpose of students’ library use should be reevaluated. As stated by Leino (2014), it should be kept in mind that students with the best reading performance has an intermediate level of competence in computer use, however frequency of computer use is not correlated with strategies of summarizing, comprehending, and remembering in reading. It should also be kept in mind that the skills  of  using  information  and  communication technologies do not consist of computer skills only.

Biagi and Loi (2012) find that student’ PISA test scores in reading, mathematics and science increase with the intensity of computer use for Gaming activities while they decrease with the intensity of computer use for activities that are more related with school curricula.  Therefore, game activities should be included frequently, and classical curriculum design should be avoided, to improve the skills of ICT use in learning and teaching processes.

PISA 2009 Turkey report reveals that Turkish students' skills of using information and communication technologies are not at the expected level yet. It is well known that lack of competence among teachers in terms of ICT, lack of infrastructure in schools and houses, incompatibility of curriculum programs with the requirements of communication technologies, lack of knowledge on ICT, negative attitudes of teachers, students and parents towards ICT are among the reasons for ineffective use of such skills. Resolution of these adversities will affect academic performance of students positively.

The Movement of Enhancing Opportunities and Improving Technology, namely FAT?H, initiated by the Ministry of National Education in Turkey has been a positive initiative; however its outcomes have still not reached the expected level. The project's effects on students' performance have not been measured yet. It is thought that FAT?H project would have positive effects on academic performance of students due to its positive returns on ICT usage skills of Turkish students if the project is concluded as intended.

In conclusion, it is seen that there is a significant correlation between the use of information and communication technologies and reading competence levels of students. However, this result does not imply that there is a causal relationship between the use of information and communication technologies and reading competence levels of students. It is suggested that the causality between the use of information and communication technologies and reading competence levels be studied by different statistical methods.

 

Limitation

The Conclusions of this study cannot be generalized to other countries since the sample of this study only included 15-year-old Turkish students from PISA 2009 data.


 CONFLICT OF INTERESTS

The author has not declared any conflict of interests.



 REFERENCES

Acar T (2012). 2019 yılı uluslararası öÄŸrenci basarılarını deÄŸerlendirme programında Türk öÄŸrencilerin baÅŸarılarını etkileyen faktörler. EÄŸitimde ve Psikolojide Ölçme ve DeÄŸerlendirme Dergisi, 3(2):309- 314.

 

AÅŸkar P, Olkun S (2005). The use of ICT in schools based on PISA 2003 Data. Eurasian J. Educ. Res. 19:15-34.

 

AÅŸkar P (1992). Ä°lköÄŸretimde bilgisayar: kuram ve uygulamalar. Hacettepe Üniversitesi Dergisi, 8:209-216.

 

Aypay A (2010). Information and communication technology (ICT) usage and achivement of Turkish Students in PISA 2006. TOJET: Turk. Online J. Educ. Technol. 9 (2):116-124.

 

Ayvacı ÅžH, Bakırcı H, BaÅŸak MH (2014). Fatih projesinin uygulama sürecinde ortaya çıkan sorunların idareciler, öÄŸretmenler ve öÄŸrenciler tarafından deÄŸerlendirilmesi. Yüzüncü Yıl Üniversitesi EÄŸitim Fakültesi Dergisi, 11(1):20-46.

 

Balım AG, Evrekli E, Ä°nel D, DeniÅŸ H (2009). Türkiye'nin PISA 2006'daki durumu üzerine bir inceleme: Fen bilimleri yeterlilik düzeyinin bilgi ve iletiÅŸim teknolojilerinin kullanımına göre deÄŸerlendirilmesi. e-Journal of New World Sciences Acad. 4(3):1C0079.

 

Biagi F, Loi M (2012). ICT and Learning: results from PISA 2009 (JRC Scientific and Policy Reports) (Luxembourg, European Commission).

 

Çetin E (2011). Teknoloji kullanımında öÄŸretmenlerin görev ve sorumlulukları. Mutlu Çocuklar Dergisi Çocuk Dünyası, 2:17-18.

 

Cure F, Özdener N (2008). Teachers' Information and communication technologies(Ict) using achievements & attitudes towards Ict, Hacettepe University J. Educ. 34:41-53.

 

Delen E, Bulut O (2011). The relationship between students' exposure to technology and their achievement in science and math. TOJET: Turk. Online J. Educ. Technol. 10(3):311-317.

 

Eraslan A (2009). Reasons behind the success of Finland in PISA: Lessons for Turkey. Necatibey Faculty of Education Electronic J. Sci. Math. Educ. 3(2):238-248.

 

Fuchs T, Woessmann L (2005). Computers and student learning: Bivariate and multivariate evidence on the availability and use of computers at home and at school, CESifo Working Paper, No. 8. May Munich.

 

GümüÅŸ S, Atalmış EH (2011). Explorıng the relationship between purpose of computer usage and reading skills of Turkish students: Evidence from PISA 2006. Turk. Online J. Educ. Technol. 10 (3):129-140.

 

Güngör C, AÅŸkar P (2004). E-öÄŸrenmenin ve biliÅŸsel stilin baÅŸarı ve internet özyeterlik algısı üzerindeki etkisi. Hacettepe Üniversitesi EÄŸitim Fakültesi Dergisi, 27:116-125.

 

Gürsakal S (2012). An evaluatıon of PISA 2009 student achievement levels' affecting factors. Suleyman Demirel University, J. Faculty Econ. Adm. Sci. 17:441-452.

 

Güzeller CO (2011). PISA 2009 Türkiye örnekleminde öÄŸrencilerin bilgisayar öz-yeterlik inançları ve bilgisayar tutumları arasındaki iliÅŸkinin incelenmesi. Ahi Evran Üniversitesi KırÅŸehir EÄŸitim Fakültesi Dergisi, 12 (4):183-203.

 

Kalaycı Åž (2006). SPSS uygulamalı çok deÄŸiÅŸkenli istatistik teknikleri. Asil Yayın Ankara.

 

Kayaduman H, Sırakaya M, SeferoÄŸlu SS (2011). EÄŸitimde FATÄ°H Projesinin öÄŸretmenlerin yeterlik durumları açısından incelenmesi. Akademik BiliÅŸim, Ä°nönü Üniversitesi, Malatya.

 

Leino K (2014). The relationship between ICT use and reading literacy: focus on 15-year-old Finnish students in PISA studies. Unpublished doctoral dissertation.

 

Ministry of Education [MEB] (2011). PISA 2009 preliminary report national. 

 

Morgan CT (2004). Psikolojiye giriÅŸ. (Çev. Arıcı, H., Aydın, O. ve ark.), Ankara: Hacettepe Üniversitesi Psikoloji Bölümü Yayınları.

 

Organisation for Economic Co-operation and Development [OECD]. (2012). Database – PISA 2009 Retrieved 10 June 2011

view

 

Özçelik DA (1998). EÄŸitim programları ve öÄŸretim, Ankara: ÖSYM Yayınları.

 

PISA (2009). National Preliminary Report, 2010 is procured in the website

view

 

Shelley M, Yildirim A (2013). Transfer of Learning in Mathematics, Science, and Reading among Students in Turkey: A Study Using 2009 PISA Data. Int. J. Educ. Math. Sci. Technol. 1(2):83-95 ISSN: 2147-611X.

 

Song H, Kang T (2012). Evaluating the impacts of ICT Use: A Multi-Level Analysis with hierarchical linear modeling. TOJET: Turk. Online J. Educ. Technol. 11(4):132-140.

 

Spiezia V (2010). Does computer use increase educational achievements? Student-level evidence from PISA. OECD Journal: Econ. Stud. (1):127-148.

 

Steffens K (2014). ICT use and achievement in three European countries: what does PISA tell us? Eur. Educ. Res. J. 13:553–562.
Crossref

 

Sweet R, Meates A (2004). ICT and low achievers: what does PISA tell us? In A. Karpati (Ed.), Promoting Equity through ICT in Education: Project, Problems, Prospects. Budapest, Hungary: OECD and Hungarian Ministry of Education.

 

Tatlıdil H (1992). Çok deÄŸiÅŸkenli istatistiksel analiz. Engin Yayınları, Ankara.

 

Yavuz S, Coskun AE (2008). Sınıf öÄŸretmenliÄŸi öÄŸrencilerinin eÄŸitimde teknoloji kullanımına iliÅŸkin tutum ve düÅŸünceleri. Hacettepe Üniversitesi EÄŸitim Fakültesi Dergisi 34:276-286.

 

Ziya E, DoÄŸan N, KelecioÄŸlu H (2010). What is the predict level of which computer using skills measured in pisa for achievement in mathematics. TOJET: Turk. Online J. Educ. Technol. 9(4):185-191.

 




          */?>