In the past decade, rapid developments in information and communication technologies (ICT) have led to the developments in the field of education and increased the quality of education besides contributing to learning experiences (Yalman and Tunga, 2014). This rapid development of ICT has enriched teaching and learning experiences and led to a better-quality education (Tella, 2011; Maldonado et al., 2011). Due to the rapid growth of the Internet, e-learning has become an alternative that has facilitated students’ learning (Wang Tzu-Hua, 2008). When compared with the traditional learning, it is seen that e-learning provides students with more sources and allows them to gain more satisfactory learning experiences via instructional activities (Saulnier et al., 2008). Thanks to e-learning, students can carry out learning activities by determining the time and  their  pace of learning the subjects. In other words, with the help of e-learning materials, students have the chance to evaluate their own advances. In face-to-face higher education, evaluation methods could be central as a basic component of effective learning and can define such methods in the learning process as measurement of students’ achievements and their pace of learning (Gikandi et al., 2011). Mid-term exams, end-of-term exams and quizzes may be given as examples of evaluation. In general, examination is one of the main methods used to confirm the results of students’ learning. Quizzes can be conducted to produce better instructional materials and for faculty members to evaluate students’ learning experiences. In this way, the quality of students’ academic achievements and their learning experiences can be increased (Miguel et al., 2016; Jordan, 2012; Levy and Ramim, 2007).

It is known that different methods used to increase students’ academic achievements positively contribute to education levels. Probably, students’ attitudes constitute one of the most important processes that increase academic achievement (Gül et al., 2015). Attitude can be named as a mental process in which individuals determine their own behaviors in certain situations (Gagne, 1985). According to Smith (1968), attitude can be defined as a tendency that forms individuals’ thoughts, emotions and behaviors regarding a regular psychological object. An attitude and a learned fact shape individuals’ behaviors, can lead to a bias in the process of decision making. Attitudes occur as a result of the learning process and experiences (TavÅŸancıl, 2006). Online evaluations can be used in the evaluation of academic achievement. In addition, it contributes to meaningful learning and delivers the concepts or the learned information to the student as feedback. In traditional class applications, when teachers ask students a question, students are provided with a little chance of responding to the question. This makes it difficult for other students to see whether they have understood the subject or not. When questions are directed to students in distant education management system, the system can instantly provide online feedback and students thus, become more successful in learning when compared to the traditional system (Robles and Braathen, 2002). Online evaluation methods have a number of advantages mentioned above, yet there are limited areas for their application in higher education (Wen and Tsai, 2006). On the other hand, distance education at universities and private education institutions is still popular since it allows reaching a large population of students accommodating in different geographical regions.

The number of distant education software is gradually increasing, and more students can now access these software. Preference of online test to evaluate students’ performances in educational processes will allow conducting    both    test    applications     and     reporting procedures more rapidly. Attitudes are undoubtedly regarded as a good determiner of academic achievement. It is seen that technology has been in use to a great extent in educational studies in recent years. However, it is obvious that the number of studies conducted to reveal students’ attitudes towards online evaluations is quite limited (Dermo, 2009). In general, technology-aided studies were measured in relation to technology and computer, and their academic achievements were predicted by these variables. In addition, although, there are positive attitudes towards computers, students are likely to demonstrate negative attitudes towards online evaluation (Bindak and Çelik, 2006; Ergün, 2002).

Theoretical framework

MOODLE has been used as an education platform via the Internet by a number of public institutions and private corporations for years. Many researchers investigated general system features of these web-based learning systems, users’ attitudes towards these systems and their levels of satisfaction with these systems (Coates et al., 2005; Engelbrecht, 2005; Marikar and Jayarathne, 2016; Martínez-Torres et al., 2008; Njenga and Fourie, 2010; Seale and Cooper, 2010). There several reasons for this popularity of MOODLE: it is free of charge to access the system; it is an open-source system; and users can easily solve problems themselves (Gutiérrez et al., 2010; Kakasevski et al., 2008; Limongelli et al., 2011; Xu and Mahenthiran, 2016). This education system, whose courses or curriculum can be designed in line with users’ needs via the web, have such basic features as homework and source sharing, questionnaires and forums as well as a testing system that allows measuring students’ success at the end of the education given. The study aimed at determining the views and attitudes of students taking education via the distance education management system towards the exam system. In this way, it could be easier to see whether the whole system functions well or not.

Recent developments in information and communication technologies have caused a number of published papers to become out of date. In this respect, in web-aided instructional methods, information technologies are shaped in accordance with the renovations since these technologies are influenced by the related developments. In literature, when the papers in the field of web-aided education are examined, it is seen that issues mentioned by other studies were mostly examined via generalization (Brine et al., 2007; Georgouli et al., 2008; Zakaria and Daud, 2008). On the other hand, although, such systems are similar to each other in terms of function, methodology and form, their user interfaces, courses, questionnaires and  exam  systems  could  differ (Brine et al., 2007; Romero et al., 2009). The basic features used here such as presentation of courses, questionnaires and forms are for general use of the system, while the exam system is used to measure and evaluate the success of students. Correct and valid measurement of students’ gains at the end of an education process depends on the features found in the exam module. Students’ evaluation of the exam module to determine its negative or positive aspects is important to reveal the related deficiencies. This study focused just on determining the attitudes towards the exam module used in MOODLE system rather than determining the attitudes towards exam modules used in web-aided education systems.

Evaluation of the success of the web-based learning system of MOODLE, which has millions of users all over the world, will be possible via the related measurements and the results to be obtained from these measurements. When the related literature is examined, it is seen that there are studies which examined attitudes of users towards e-learning environments (Graf et al., 2009; Kakasevski et al., 2008; Sun et al., 2008)as well as those which investigate students’ views about the system (Kao and Tsai, 2009; Richardson, 2009; Sher, 2009; Yassine et al., 2016). Depending on the results of these studies, it could be stated that it will be better to overcome the problems existing or to exist in the system or to improve the popular applications in line with users’ feedbacks.

Measurement and evaluation constitute the basis of education. All the evaluations to determine students’ levels of knowledge about the subjects they have been taught, should include objective and healthy evaluations. In contrast with the exams given in traditional education, those conducted in web-aided education include different features and norms; Questions should be clear and comprehensible; time limitations should focus on solving the questions; and a simple language should be used in the instructions to solve the questions. Here, the purpose is to minimize the probable problems. Students’ evaluations regarding the web-aided exam system at the end of the exams will help overcome related future problems. 

The participants in the study constituted those taking their theology undergraduate education via distance education system using the platform of MOODLE. In the study, a scale was developed to evaluate the students’ attitudes towards the web-based exam system (MOODLE). The data collected in the scale development process were analyzed with “Exploratory Factor Analysis”, “Uni-Dimensional Data Analysis” and “Two-Dimensional Factor Analysis” to examine appropriateness of the values to the fit indices.

Sample

The participants in the study were 1300 students who registered in Theology Undergraduate Education Program; they received education with the e-learning management system in a distant education center of a state university. The scale designed for the study was applied via the system which was used to give education. A total of 1287 students responded to the scale conducted online. In this way, almost the whole research group was reached. In the study, as for the demographic backgrounds of the students filling out the scale in the study, 45.53% of them were female, and 54.47% of them were male. The data collected were examined in digital environment. Among the responses to the scale, those with all of its options marked same, those with most of its options marked same and those given to reverse items were examined, and the questionnaire forms which were not responded to correctly or completely were not included in the scope of the study to preserve the objectivity of the study. As a result, a total of 653 questionnaire forms were included in the study. Table 1 below presents the frequencies and percentages of the participants with respect to their gender. According to the demographic backgrounds of the participants, 51.61% of them were females, and 48.39% of them were males (Table 1).

Attitude scale for the web-based learning process

For the purpose of determining the success, level and quality of education given via the web, several scales have been developed by researchers. Examining such scales in the study (Dermo, 2009; Gül et al., 2015; Bahar, 2014) and considering the standards previously determined (Swedish National Agency for Higher Education, 2008), a trail item pool was formed depending on the related literature, and faculty members expert in the field were asked for their views. The scale was presented to three experts in the field and to two Turkish language experts. The scale made up of 31 items, 23 of which were positive and eight of which were negative, was piloted to see whether it was comprehensible or not. The results of the statistical analysis of the collected data revealed that five items were difficult to comprehend and 10 items considered by the experts to be statistically inappropriate were excluded. As a result, there were 16 items in the finalized scale. Following this, the updated version of the scale was applied again to collect the research data. The scale included four items in the dimension of “System and Usability”, three items in the dimension of “Comprehensibility”, six items in the dimension of “Examination and Features” and three items in the dimension of “Security and Reliability”. The scale was designed as Five-point Likert-type with the choices of 1- I completely disagree, 2- I disagree, 3- I partly agree, 4- I agree and 5- I completely agree. In attitude scales, a five-point rating method ranging from “I completely agree” to “I completely disagree” could be used (Dunn-Rankin, 2004; TavÅŸancıl, 2005). When the related literature is examined, it is seen that scale development phases are as follows (TavÅŸancıl, 2005; Dunn-Rankin, 2004; Devellis, 2003; Karasar, 1995).

1. Forming the item pool M

2. Asking for experts’ view

3. Conducting the pilot application

4. Applying the draft scale to the study group and factor analyses

5. Calculating the reliability of the scale

In order to determine the factor loads predicted for scale development, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were conducted. After the results of these two analyses that were obtained, confirmatory factor analysis was used for the model data fit. The fit indices used in the study were Chi-Square fit test, Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Root Mean Square (RMS) and Root Mean Square Error of Approximation (RMSEA). Table 2 presents the results obtained via the analysis of the data. The value of x²/sd obtained in the study was lower than 3, which demonstrated that the model had an acceptable fit (ÅžimÅŸek, 2007). The value in question was higher than three. As the value of x²/sd is sensitive to the size of the sample, it should be interpreted with other fit indices (Jöreskog and Sörbom, 1999). According to the model data fit, the value of 0.94 GFI was higher than 0.09; the value of 0.91 AGFI was higher than 0.09; the value of 0.064 RMSEA was lower than 0.08; and the value of 0.038 RMR was lower than 0.05.

Exploratory factor analysis

The scale developed was applied to 1288 individuals, yet the analysis was carried out with 653 participants due to the fact that some of the questionnaires were not filled completely or some of them included items all with the same options marked. The Kaiser-Mayer-Olkin (KMO) value of the scale was 0.877, and Barlett’s test was found significant (p < 0.01). The scale included a total of 16 items and four factors. In line with the expert view, no change was done in the scale. Table 3 presents the results of the exploratory factor analysis. 

Uni-dimensional confirmatory factor analysis (CFA)

Figure 1 presents the results of the confirmatory factor analysis conducted to determine the fit between the factors and items in the scale. According to the results of the confirmatory factor analysis, the Chi-Square  value  of x^2(98,N=685)=7663,63 calculated for the model-data fit was significant (p < 0.000). The fit statistics values calculated using the Lisrel GFI = 0.94, AGFI = 0.91, CFI = 0.97, NNFI = 0.96 software were as follows: RMSEA = 0.064, RMR = 0.038, and = 0.97 dir. Since these values were in appropriate ranges, there was no need for any modification in the scale.

IFI

Two-dimensional confirmatory factor analysis (CFA)

The attitude scale for the web-based learning process with its 16 items and four factors was tested with two-dimensional confirmatory factor analysis (CFA). According to EFA, the items for the fact of System and usability had standard solutions of 0.64, 0.66, 0.58 and 0.51; those for the fact of Examination and Features had standard solutions of 0.49, 0.49, 0.54, 0.56, 0.54 and 0.53; those for the factor of Comprehensibility had standard solutions of 0.75, 0.71 and 0.90; and those for the factor of Security and Reliability had standard solutions of 0.53, 0.54 and 0.49. As can be seen, all the standard solutions were found to be higher than 0.45. As a result of CFA, the fit statistics values calculated as x²/sd = 3.36 were as follows: RMSEA = 0.064, RMR = 0.038, GFI = 0.94, AGFI = 0.91, CFI = 0.97, NNFI = 0.96 and IFI = 0.97. All the fit indices obtained were found to demonstrate an acceptable fit according to Schermelleh-Engel, Moosbrugger and Müller (2003) (Figure 2). It was seen that the values obtained regarding the items and the whole scale were in acceptable ranges for the applicability of the scale.

Results of reliability analysis

Table 4 presents the reliability coefficients for each factor in the scale. The Cronbach Alpha value for the whole scale was calculated as 0.873. The Cronbach Alpha values were calculated as 0.834 for the sub-factor of “System and Availability”, as 0.793 for the factor “Comprehensibility”, as 0.761 for the factor of “Examination and Features” and as 0.740 for the factor of “Safety and Reliability”. The Cronbach’s alpha values obtained in relation to the sub-factors demonstrate that the scale developed was valid (Brownlow, 2004).

Depending on the related need in literature, the present study aimed at developing a valid and reliable attitude scale to evaluate the attitudes of distant education theology undergraduate students towards the Web-Based Examination System (Appendix). For this purpose, in line  with  the  scale  development  phases  reported  in   related literature (Devellis, 2003; Dunn-Rankin, 2004; Karasar, 1995; TavÅŸancıl, 2005), an item pool was formed; experts’ views were taken regarding the items; and the trial item pool was piloted. Following the pilot application, the draft scale was applied to the study group, and the data were examined in terms of the distribution of the scale scores obtained before conducting the Exploratory Factor Analysis. After the distribution was found to be at the desired level, to determine the factor structure of the scale, basic components analysis was preferred as the factorization method, and the maximum varimax technique, one of the vertical rotation method, was preferred considering and significance. In the study, Exploratory Factor Analysis was conducted sequentially and gradually on the data several times. For the statistical evaluation of the data collected during the application, five items lacking unity of meaning and ten items considered statistically to be inappropriate were excluded from the scope of the study.

After the scale items were updated, the scale was applied again to collect the research data. The final version of the scale, which was initially made up of 31 items, included 16 items. In addition, factor analysis applied again to the final version of the scale. According to the results of the exploratory factor analysis, there were four factors with an Eigen value higher than 1. It was found that the first factor (System and Availability) contributed to the common variance with a rate of 17,154%; the second factor (Comprehensibility) with a rate of 13,834%; the third factor (Examination and Features) with a rate of 16,496%; and the fourth factor (Safety and Reliability) contributed to the common variance with a rate of 12,868%. The contribution of the four factors to the total variance was calculated as 60,353%. Also, the Kaiser-Mayer-Olkin (KMO) value of the scale was calculated as 0.877, and Barlett’s test was found significant (p < 0.01). In order to evaluate the validity of the two-factor structure obtained as a result of the exploratory factor analysis, confirmatory factor analysis (CFA) was conducted. When the CFA fit values were examined, the Chi-Square value of x²(98,N=685) = 51.38 calculated for the model-data fit was found significant (p <0.000). The fit statistics values calculated as a result of the analysis carried out with Lisrel software were as follows: RMSEA = 0.064, RMR = 0.038, GFI = 0.94, AGFI = 0.91, CFI = 0.97, NNFI = 0.96 and IFI = 0.97. Since the values obtained were in appropriate ranges, no modification was done in the scale.

The web-based learning attitude scale made up of 16 items and four factors was also tested with the two-dimensional confirmatory factor analysis (CFA). The fit statistics values found to be x2/sd = 3.36 as a result of CFA were as follows: RMSEA = 0.064, RMR = 0.038, GFI = 0.94, AGFI = 0.91, CFI = 0.97, NNFI = 0.96 and IFI = 0.97. According to Schermelleh-Engel, Moosbrugger and Müller (2003), all the fit indices obtained were found to demonstrate acceptable fit. In addition, the Cronbach Alpha values were calculated as, 834; for the factor of “System and Availability”, as, 793; for the factor of “Comprehensibility”, as, 761; for the factor of “Examination and Features” and as, 740 for the factor of “Safety and Reliability”. The scale could be used as a valid and reliable data collection tool in studies to be conducted not only to determine the attitudes of students from the Department of Theology Undergraduate Education Program towards web-based distant education as well as towards related exam practices but also to examine the factors influential on their attitudes. 

Rapid changes and developments in computer and Internet technologies lead to the development and spread of e-learning management systems. In this process of changes, e-evaluation has gradually gained more importance and become a significant part of this transformation. Evaluation on paper-pen basis is quite an inefficient method of evaluating students’ success and making education decisions (Sırakaya et al., 2014). Although, the reliability and objectivity of exams conducted via e-learning platforms are examined by researchers and experts, these exams have a number of advantages when compared to those conducted with traditional methods. The most prominent advantages of e-evaluation include saving time and cost, gathering the responses to the questions in computer environment, providing appropriate and rapid feedback, allowing flexibility, increasing the reliability by minimizing the mistakes made by human (Struyven, Dochy and Janssens 2002; Angus and Watson 2009), decreasing faculty members’ involvement (Anderson et al., 2005) and obtaining the results rapidly (Kuhtman, 2004). In addition, exams conducted in computer environment, in contrast with paper-pen exams, allow enriching the presentation of information via integration of multimedia elements (Liu et al., 2001). Obviously, electronic evaluations are beneficial. Faculty members can obtain rapid results, thanks to e-evaluation, and use of computers that will make education better. This will allow transition from a traditional education environment to the one that makes students more active and contributes to their learning. In recent years, the traditional measurement and evaluation methods in the field of education have been replaced by e-evalaution. Thanks to this, the field of measurement and evaluation has gained a new dimension. As an alternative, faculty members try to spread the use of the Internet and computer in the field of measurement and evaluation (Özmen, 2006; Lawrenz et al., 2001).

The authors have not declared any conflict of interests.