Improving graduate students ’ learning through the use of Moodle

Moodle stands as an online tool that promotes enhanced learning in higher education. However, it often becomes a repository of contents instead of an interactive environment. In this paper we describe how this platform was used by university students and teachers in 104 courses and compare whether ICT-as core subject courses-use Moodle more effectively than non-ICT content related courses. A sample of 393 students answered a 20-item Likert-type questionnaire (OUS-Q) and three open questions. Descriptive statistical analyses, chi square comparisons and topical analysis were conducted. The results show that all courses include a large number of digital contents and activities. However, scores for ICT-courses were significantly higher in evaluation of assignments or video-learning. There were no noticeable differences in other factors. Qualitative data show 891 comments that were classified into five dimensions. In conclusion Moodle improved content management and interactivity but only ICT courses used it as a learning platform.


INTRODUCTION
The development of multimedia resources has allowed developing new ideas about teaching-learning concepts in the current educational landscape (Friedman, 2006).
In higher education it is additionally demanded that both teachers and students use online learning resources to support the knowledge acquired from the formal faceto-face context.That is why the concept of web-based instruction or mobile learning (m-learning) is gaining popularity in Teaching Education as a way of improving daily practices (Smith, 1999;Chen and Huang, 2010;Hwang and Tsai, 2011).The types of resources are considerably varied: from discussion lists, podcasts, database of libraries, or virtual learning environments, to virtual subjects.In a way, the three main reasons that protect the combination of e-learning and face-to-face teaching are: accessibility, flexibility, and interactivity (Rosenberg, 2001).
This extensive use of technology has resulted in many studies aiming to explore e-learning skills and the experiences that students and teachers reveal.Elearning is a new type of interactive learning in which the content to be learnt is available online and, therefore, provides an automatic feedback for students about these teaching activities (Toth et al., 2006).Consequently it creates a different way of understanding the manner in which both students and instructors interact (Bruce and Curson, 2001) which leads educational research to understand what the required characteristics are for this *Corresponding author.E-mail: juanjo_mena@usal.es.
Authors agree that this article remain permanently open access under the terms of the Creative Commons Attribution License 4.0 International License new generation of students (Oblinger, 2003;Hammond, 2013).

Moodle as a learning management system (LMS)
Schools are in permanent change, especially with the incorporation of ICT.Many teaching courses at universities offer a greater amount of online content, and some of them are a prescriptive requirement to promote virtual learning and encourage the interactions between teachers and students (Psycharis et al., 2013).
LMS refer to an integrated set of networked and computerized tools that support online learning (Kirner et al., 2008).They are complexly didactic systems that have a platform for web-learning including (1) traditional activities: presenting information, course materials, evaluating the students' work (Yueh and Hsu, 2008) and ( 2) additional features such as more communication with peers and instructors, social network site membership (Pereneder et al., 2012) access to learning material, submission of assignments (Melton, 2006), and active learner-learner discussion among participants (Swan, 2001).
One of the most used LMS is Moodle (Modular Object-Oriented Dynamic Learning Environment), an open source based on pedagogical principles (Goyal and Puhorit, 2010) that incorporates several multimedia resources to manage content lessons (Moodle, 2007).Moodle complements teachers' face-to-face teaching; it is available in more than 77 languages and present in 193 countries (Celik, 2010).The platform has become established as an online tool that allows the use of graphics, forums, chat, databases, quizzes, survey, wikis, web pages, video transmissions, and Java and Active X technologies to reinforce lessons.Besides, Moodle is expanding its use to cloud computing and mobile learning (Wang et al., 2014).Peat and Franklin (2002) value Moodle not only for its technical applications but for the promotion of new learning among students since it facilitates an organized display of the material.Dougiamas and Taylor (2003) emphasize that the fundamental value of this platform is that users can share learning objects: any digital resource that may be used to support learning (Wiley, 2000) and therefore "...must have an external structure of information to facilitate their identification, storage and retrieval: the metadata" (Rehak and Mason, 2003: 25) to accomplish that purpose.
Moodle, as e-(or b-) learning tool, extensively enables this type of learning because of these three characteristics: a. Interaction.It enhances student-student discussions (Swan et al., 2000, Picciano, 2002).Beaudoin (2001) found that students reported increased satisfaction for online courses.b.Usability.It has a variety of useful options for students such as easy installation (Katsamani et al., 2012), customization of the options (Sommerville, 2004), security and management (Chavan and Pavri, 2004), easiness of navigation; software attractiveness and users' satisfaction (Kirner and Saraiva, 2007).c.Social presence.Moodle promotes a sense of community in online courses (Sagun and Demirkan, 2009).Social presence is an essential aspect in any educational experience referring to participants' perception on the degree they see others as true speakers in mediated communication (Gunawardena and Zittle, 1997).It has been demonstrated to be a relevant predictor of students' perceived learning (Richardson and Swan, 2003).
And, as a teaching tool, Moodle allows for (Ross, 2008) (a) The management of subject contents (documents, graphics, web pages or videos); (b) Communication with students (i.e.forums or virtual tutorials) and (c) Students' assessment (i.e.grading or monitoring subject assignments) However, it also requires a better management of the classroom, the change of the format of tutorials and assignments as well as a continuous engagement in student progress (Antonenko et al., 2004).

Theoretical paradigms that support Moodle
Unlike other similar online platforms, Moodle does not emerge from the engineering context but, on the contrary, it has an educational background (Cole and Foster, 2007).For that reason, the development of this tool is based on different theories on learning.One of these theories is related to social constructivism (Duff and Jonassen, 1992).This epistemological foundation is based on collaborative discourse (Amundsen, 1993) and the development of meaning from sharing texts and a series of social devices (e.g.Graphics, diagrams, etc.).The basic precept is that students create their own learning and teachers guide the process of the construction of knowledge.This process of interaction also allows the development of learning communities (Lave and Wenger, 1991) where the discourse enables a process of searching for meaning.
Nonetheless, the development of a community is often difficult, especially when it is of virtual nature because limited (or no) face-to-face interaction take place (Dawson, 2006).Although ICT help to connect participants, the simple fact of using computer-mediated communication does not necessarily facilitate the emergence of a community (Brook and Oliver, 2003).
A second model that supports the use of Moodle is the theory of the types of knowledge (Belenky et al., 1986).It highlights the existence of two different learning styles: independent knowledge and connected knowledge.The independent students adopt a critical position when faced with contents, while the connected students tend to build ideas on the basis of others through collaboration.These two types of knowledge are independent and each of us uses them without distinction in different situations.To date the argumentative and critic learning has been encouraged as the main channel, rather than a collaborative one.Moodle allows the development of connected knowledge.
Finally, there would be a third theory that justifies the use of Moodle: the theory of emancipation (Habermas, 1984) that proves how critical and collaborative thought allows for transformation of perspectives by historical and political contingencies.The best way to achieve transformative learning is to help students to examine their own beliefs, feelings and actions and to explore the existing alternatives through negotiated reflections.Moodle can be an ideal environment to foster it.

Use of Moodle in higher education
To date, the success of this virtual platform among the university community has been mainly based on offering a permanent repository of contents, units, assignments, and essays that can be shared at any time (Medina et al., 2014).However, it is still unclear to what extent the use of Moodle allows students and teachers to build collaborative learning, in what is the ultimate promise of educational research.
Some studies confirm that both Moodle and online materials improve learning results (Escobar-Rodríguez and Monge-Lozano, 2012;Martín-Blas and Serrano-Fernández, 2009;Núñez et al., 2011).Soyibo and Hudson (2000) argue that teachers who use web pages designed for teaching or online virtual materials increase students' attention and participation and allow more significant learning experiences.Other authors, such as Steyaert (2005) show that both Moodle and Internet organize contents in thematic units and save time in the management of this tool for both teachers and students, whereas Peat and Franklin (2002) state that what facilitates learning is the fact that it provides students with a simple display of the syllabus.
However, it is crucial to point out that the samples of students are usually homogeneous and the methods to explore such virtual learning experiences are based on interviews and discussion groups.Sharpe et al. (2006) have proved how students with disabilities have different virtual learning experiences from those without disabilities in the same subjects or learning situations.This conclusion can be extended to all populations bearing in mind that individual differences mean different learning patterns.If it is true, we need to use Moodle or any educational tool in a way that reflect learners' and teachers' voice in the experimentation with these tools is increasingly advocated (Sharpe et al., 2005).Melton (2006) asserts that before implementing its use, the schools and educators must carefully evaluate it.Besides, it would be necessary to identify both teachers' and students' standpoints concerning the advantages and difficulties perceived in its use.Weitzman et al. (2006) provide a specific guide about the factors that must be taken into account before institutionalizing a tool like Moodle: (1) defining its purpose: The universities and high education institutions need to explicitly inform both teacher Educators and graduate students (i.e. in written form) about the Moodle platform: i.e., guidelines and protocols on how to use Moodle (i.e., criteria for uploading documents or designing quizzes or questionnaires), ( 2) collecting information about its users, (3) generating a list of suggestions based on the feedback obtained in steps 1 and 2: The universities have to conduct preliminary studies to know the potencial users`, opinions and suggestions (4) carrying out research that show its benefits (collecting empirical evidences), and (5) choosing and implementing the tool (according to collected research evidences).It seems that educational research still needs to consolidate steps 3 and 4.
Therefore, and bearing in mind the above, the present research emerged driven by the extensive use of Moodle in higher education institutions, especially in Universities.A descriptive-exploratory study was designed to obtain first-hand inputs of the real use in a particular context such as the Faculty of Education in the University of Salamanca (Spain).Our main goal is gathering research evidence on the potential of Moodle for teaching going beyond its use as just a repository of documents.
More specifically this study aims at (1) how teachers and students of the Faculty of Education (University of Salamanca, Spain) use Moodle (named as Studium: http://moodle.usal.es)taking as reference the students' perceptions; and (2) how the use of Moodle differs depending on the kind of subject (i.e., ICT content related subjects and Non-ICT content related subejcts).

METHODOLOGY
This study is part of a research project entitled: "Evaluation to optimize the use of Moodle (Studium) in the Department of Education at the University of Salamanca" reference number ID11/050.Twenty-five teachers of the University of Salamanca participated during the academic year 2012-2013.
The project followed four phases (1) Design of the questionnaires and methodological validity; (2) Application: computerization of the online questionnaire, which was uploaded into Moodle; 3) Statistical data analysis; and (4) Final report and improvement proposals.
The tasks carried out in the four phases were distributed among three teams that assumed different responsabilities: The Coordination Team (CT) organized the schedule and activities of the research project; the Area Team (AT) was responsible to construct and evaluate domain specific questions belonging to four main areas: research methods, educational technology, special education, and didactics; finally the Technical Team (TT), composed of IC technicians, was in charge of the computerization of the questionnaire.

Sample
The A non-probabilistic sample of the 393 graduate students was chosen (Arnal et al., 1992).The participants voluntarily responded to the questionnaire.

Variables and data collection instruments
The variables of the study were defined in the previous phase of the construction of the measuring instrument and, according to the subsequent data analysis, were divided into two groups: predicting variables (course, degree and type of subject) and criterion variables (quantity and quality of the use of Moodle).A 20-item questionnaire (Optimizing the Use of Studium Questionnaire -OUS-Q-) was constructed following the online survey process (Berends, 2006;Kerlinger and Lee, 2000) (phase 1).Then, it was applied online in all the subjects of the department (phase 2).The questionnaire was organized into seven units of contents: 1) Personal information (gender, degree, course, and subject); 2) Access to the virtual campus; 3) Training for the use of Stadium; 4) Contents; 5) Assessment; 6) Interaction, and 7) Learning.Please see Appendix A for an extract of the UOS-Q questionnaire Experts in educational research methodology, and experts in information and communication technologies revised and modified the questionnaire until the final version was completed.Cronbach's alpha α=0.92 was also calculated as a measure of the accuracy or stability of the answers.
The answers were arranged in a four-degree Likert scale associated to values of: (1) Completely disagree (not at all); (2) Disagree (not much); (3) Agree (quite a lot); (4) Completely agree (a lot).The neutral response was not used because we wanted the participants' position towards the attitudinal object.According to Schuman and Presser (1996) the middle alternative (i.e.labels such as "undecided," "uncertain," or "indifferent") can be associated to absence of opinion, or ambivalence about the attitude under scrutiny.Nunnally and Bernstein (1994) also indicate that there is an advantage to using a scale with no middle "undecided" position because a neutral response gives little information.

Analysis
The research design of this study is descriptive and correlational (ex-post facto) (Kerlinger and Lee, 2000).A mixed-method approach was chosen for the analysis.A first descriptive statistical analysis was conducted for the Likert questions including frequencies, percentages, measures of central tendendy (i.e., mean scores) and measures of scatter (i.e., standard deviation).A second inferential analysis was used to search for differences between the participants enrolled in the subjects related to ICT contents: that is to say, subjects where their sillaby included learning about ICT and technological competence acquisition (group 1) and those nonrelated to ICT contents: there is no intended learning on ICT tools, but they are used as storage medium (group 2).Three major aspects were compared (Ross, 2008): (a) contents, (b) evaluation and (c) interaction -which correspond to the sections 4, 5 and 6 of the QUS-Q-using SPSS 19.0.
Topical analysis was followed (Grounded Theory Analysis, Corbin and Strauss, 1990) to analyze the open-ended questions relative to three dimensions: (1) Advantages of using Moodle (Question 1: List and explain positive aspects in the use of Moodle); (2) Challenges and disadvantages of the platform and (Question 2: List and explain negative aspects in the use of Moodle) (3) Enhancements (Question 3: Point out some Moodle features that improve your experience with the Moodle interface).Student's statements (N=282) were divided into utterances (Crasborn et al., 2011) and were initially grouped into six macro-categories established by Kirner et al. (2008) for the assessment of Moodle: intuitiveness, operationability, efficiency of use, learnability, attractiveness, and user satisfaction.However, due to the level of saturation of frequencies in each of these categories, we carried out an inductive subcategorization in three additional levels.Reliability checks were done by two independent rates obtaining a coefficient of Cohen Kappa =0.79.

Quantitative study
The 80.7% of the sample were women while 18.8% were men.Regarding the subjects, 34.6% of the students took courses that corresponded to subjects content related to ICT whereas .65.4% took courses whose content was not specifically focused on technology in education

Contents and activities provided through the platform
First of all, students were asked to rate the degree of agreement or disagreement with nine items related to the organization of the units and activities in connection with the contents and activities provided through the platform Moodle.
As shown in Table 1, participants expressed a high degree of agreement, especially in the items concerning those aspects related to the contents (i.e.organization, adaptation, updating, promoting interest, understanding).The chi-square statistic test shows that there are only significant differences between the groups 1 and 2, about to the answers in two of the items.Item 6 "I like that the teacher provides us with class presentations" ( =10.296, p=0.016) and item 8 "The videos or the images selected allow us to learn in a more intuitive and dynamic way" ( =14.377, p=0.002).Group 1= Group of students enrolled in ICT-related subjects; and Group 2= Group of students enrolled in non ICT-related subjects.
On the other hand, and according to the students' perceptions, the results demonstrate that the teachers promoted "database", "choice", "survey", "forum", "lesson" and "assignments" as the most frequent activities in the Moodle platform.Generally speaking, a higher percentage of use is perceived in group 1.There are two activities that are only used by group 1, the "quiz" ( =9.455; p=0.002) and the "wiki" ( =33.886, p=0.000).
The least used activities are: "chat", "diary", "glossary", "Self-assessment exercises with Hot Potatoes Quiz", "videoconference", and "workshop".In that respect, it should be noted that the lack of use is higher in group 2, as the percentages show.The chi-square statistic calculated in each case shows that the differences discussed are significant (sig.0.05) in all the activities apart from "diary" ( =0.527; p=0.468), "glossary" ( =1.836; p=0.175), "Self-assessment exercises with Hot Potatoes Quiz" ( =0.539; p= 0.463), "videoconference" With regard to the activities done from the platform, it is significant that students of both groups have only used quite a lot or very much the activities "download files", "choice", "lesson", and "assignments".Furthermore, the use of the rest of the activities is not at all or not very frequent.As shown in Table 2, the chi-square statistic test shows that there are significant differences between both groups in the activities "download files", "survey", "forum" and "assignments" (sig <0.05).

Assessment through the platform
In this section, students showed the degree to which the teacher promoted the development of the assessment activities from Moodle, both in its approach: formative (through self-assessment) and/or summative assessment (exams), establishment of criteria, development of different activities and assessment resources, and in the communication of the results obtained by the students.
On the other hand, there are significant differences in the students' opinions (sig.<0.05) as shows the associated probability for the items 1 ( =13,846; p= 0.003), item 2 ( = 23.273;p=0.000), item 4 ( =30.026, p=0.000) and item 7 ( =12.324; p=0.006).The four of them related to self-assessment, teacher assessment and exams and influence of those proposed activities for the final mark.

Interaction through the platform
In the light of the data, there are no differences among the groups (1 and 2) in the interaction dimension.Students' answers in line with the platform allows a more fluid communication with the teacher ( =0.277, p=0.964), it encourages that tutorial with the teacher is more continuous and prolonged ( =2.470, p=0.481) and promotes the communication among students ( =1.912, p=0.591).In consequence, there are not significant differences between the groups as the results of chisquare indicate (sig < 0.05).

Qualitative study
In the qualitative study, a total of 282 comments were collected about the positive aspects of Moodle, 287 about the problems of the platform, and 322 about the aspects that should be improved.
Table 3 shows the analysis of the comments referred to the positive aspects of Moodle.In the positive aspects dimension five domains similar to Kirner et al. (2008) were found that were further broken down into a three-level category.Generally speaking, the operative level that contains the repository of contents, activities or assignments, and the organization of the subject include the largest number of student comments (57.05%) (dimension 2).On the other hand, 7.09% of the comments refer to Studium as a tool to facilitate learning (dimension 4) and 23.39% to promote interactivity (dimension 3).Finally, with regard to the technical management of the tool, the aspects that refer to intuition (dimension1) and attractiveness (dimension 5) represent 8.5 and 3.89%, respectively.The highest frequencies appear in the categories of repository of contents (25.88%), better organization of the subject (12.05%), and easiness of communication between teachers and students (7.80%), which draw together the highest number of comments.
Subsequently, we present some of the conclusions that can be deduced from the abovementioned analysis.We also suggest improvement proposals.

DISCUSSION AND CONCLUSION
There are many advantages and possibilities that have been suggested about Moodle as a provider of three essential resources: (1) online contents and activities; (2) interactive and transparent assessment and (3) teacherstudents fluid interaction (Ross, 2008).However, few studies demonstrate that the daily use of it can contribute to an improvement in these three dimensions.
According to the first section (online contents and activities), Connolly et al. (2007) found that online students consistently perform better than the part-time face-to-face students, obtaining the conclusion that online contents are strong resources for student learning.Mentzer et al. (2007), contrary to Connolly's et al. study, found that learning did not differ significantly between on-line and face-to-face environments, but satisfaction turned out to be lower in on-line contexts.In our study, although these two ways of teaching are not compared since Moodle is established as a complement to the face-to-face teaching, people surveyed thought that the available online contents facilitate a better update of the subject, and they were logically organized.However, they never consider this tool more valid than face-to-face teaching or as a replacement of it.
Regarding the assessment, this study reveals that the platform serves to publicize assessment criteria and to mark the assignments.However, it does not usually incorporate elements of feedback.Furthermore, it is even less frequently that self-assessment activities and exams are done through Moodle; as a consequence, the assessment resources of the platform are not totally profitable, and an unbalance is made evident between teaching resources (document uploads, forums of doubts, assignment uploads, comments, etc.) and specific assessment resources (surveys: multiple-choice, true or false, short answer, etc.).This result is in accordance with Gibbs (2006) when he states that it is necessary "to avoid the disjunction between teaching and assessment modes […] to enable formative feedback to students (a pedagogic issue)" (Gibbs, 2006: 173).Conversely, Coulby et al. (2010) proved that undergraduate students improved their level of feedback and technology skills through a competency-based assessment.
Regarding the degree of interaction, the results show how the platform promotes a fluid communication with the teacher and with other students, and a most continuous exercise of the tutorial action with the teacher.This contradicts other studies that indicate how online communication can be lower and more intermittent (not contingent).For instance, Jin (2005) noted a general lack of student participation in online discussions and the need for solutions to ameliorate this problem.Other studies go further and highlight how the essential function of Moodle is to be the facilitator of student interactive learning (Bruce and Curson, 2001;Zenha-Rela and Carvalho, 2006;Mateo and Sangrá, 2007).More recently Fatih and Demirkan (2015) have demonstrated that collaborative digital environments promote undergraduate students' creative processes such as idea generation and self-motivation.In all these cases, interactivity is not mere communication, but it turns into online learning opportunities.Although this study reveals that this tool facilitates learning, the predominant use of it is linked to the filing of information and interaction between users.
On the other hand, if we consider the distribution of users into the two established groups, generally speaking, the high scores showed that Moodle improves content management, course assessment and interactivity in education.Nonetheless, ICT Educational courses tended to use it as a dynamic platform (resource of learning) whereas non-ICT related courses used Moodle more as a static repository (support platform).That leads us to think that the content of the course is decisive to exploit all the resources offered on the platform from a didactic point of view and, thus, to achieve a change to transform the tool into a collaborative learning facilitator.However, when assessing the comments about positive aspects, in general, a high percentage of them refer to the use of the platform as a repository.
We can conclude assuming that Moodle is complementary to the teaching-learning process and that it is not a replacement of it (Coskun and Arslan, 2014).Considering its current support to the classroom lessons, Moodle is not being used at its full potential.Furthermore, it has been recently demostrated that the combination of Moodle with m-learning favours collaboration online assessment and knowledge dissemination, which in turn, enable ubiquitous learning (Alier et al., 2010;Bogdanov et al., 2014) present study is based on the population of graduate students enrolled in the subjects taught by the Department of Education of the University of Salamanca in 2012/2013: ICT in Education, Didactics and Education, Attention to Diversity, Counselling, Educational Intervention in Communication and Language, Educational Research Methodology, Methodological grounding for Educational Research, Special Education, and Learning Disorders.One hundred and four subjects integrate the whole syllabus of the Department of Education, which includes two different areas of knowledge: Area of Didactics and School Organization (70 subjects), and Area of Research Methods in Education (34 subjects).

Table 1 .
Students' opinion about the contents and activities uploaded in Studium (Moodle platform).

Table 2 .
Activities promoted in Studium by the Faculty Teachers according to the students' opinion.

Table 3 .
Positive aspects of the use of Studium(Moodle)according to the students' perceptions.