Analyzing Institutional Evaluation Reports prepared from 2013-2015 by European University Association

European University Association is an institution which guides not only European but all universities in their efforts to improve their quality within the context of Bologna Process. In this study Institutional Evaluation Reports prepared by EUA for all higher education institutions that applied to be evaluated between 2013 and 2015 were examined. These evaluations were grouped under six categories namely administration, finance, research, education, service to society and internationalization. It is understood that administration is the most evaluated subject, while finance is the least evaluated in the reports. As regards administration, the most emphasized points are the need of a more solid strategic planning and to improve quality assurance processes. In finance the universities are primarily advised to focus on raising money from different resources. For research, the most emphasized subject is the lack of research strategies in universities. Two important suggestions in education category are to use student centered teaching and to update curricula so as to focus on application. While for service to society category, the reports mainly suggest to involve external stakeholders more systematically in the universities and for the category of internationalization, it was pointed out that most of the universities are in need of an institutional strategy.


INTRODUCTION
When Bologna Process was first launched in 1999, the most important goal that was anticipated was to increase the competitiveness of European universities. One of the steps that had to be taken was to strengthen the quality assurance systems of universities. Quality and quality assessment started to rise as a major issue in 1980"s, well before Bologna Process, because of two main reasons: massification in higher education and the private sector becoming the main employer of university graduates (Amaral and Rosa, 2008: 22). Among the reasons for approaching quality as an issue in universities are; high increase in student numbers, emerging of different types of education programs in order to meet the demand, and extension of autonomy in state (Yükseköğretim Kurulu [YÖK], 2007: 22). Korkut (2001) lists the reasons why quality has become important in higher education as; the pressures imposed on state subsidy by the increase in demand to higher education, increase in the need for qualified human resources, the trend towards internationalization in higher education, development of a joint labor market as a result of globalization, and the increase in the demand to give more autonomy to higher education institutions.
Quality assurance can be defined as continuous assessment of a higher education institution, system or program according to predefined standards determined by various means like quality management, quality improvement, quality control, and quality evaluation (UNESCO-CEPES, 2007: 74). In other words, quality assurance is all the systematic and planned operations a higher institution does in order to guarantee that it carries out its activities and services like education, training, and research in accordance with certain standards (Edinsel et al., 2008).
Quality assurance is one of the most emphasized and supported areas of Bologna Process. Reports showed that higher education institutions place more emphasis on this subject day by day and develop new strategies to improve their service quality (Eurydice, 2015). One of these strategies is external evaluation.
External evaluation is a process of assessment of a certain unit or main activity in an institution by an expert organization through collecting data, information and evidence, and making a statement about its quality (UNESCO-CEPES, 2007: 56). Arslan (2009) mentions that external evaluation systems are important in evaluating activities of institutions from outside and using a different perspective, that they are crucial in determination of problematic areas in higher education institutions and therefore their numbers should be increased.
European University Association (EUA) is an important evaluation institution that works in the field of quality assurance. Founded in 2001 with the merging of two organizations, European Association of Universities (CRE) and The Confederation European Union Rectors" Conferences, the mission of this new foundation was stated thus: to promote the development of a coherent system of European higher education and research, through active support and guidance to its members as autonomous institutions in their development of the quality of teaching, learning and research, and in enhancing their contributions to society (EUA, 2002: 4).
EUA offers an external evaluation service called Institutional Evaluation Program (IEP) in order to promote quality assurance and evaluation studies in universities. Sursock and Amaral (2008: 37) point out that IEP aims at offering an alternative to the "more bureaucratic and intrusive methodologies" of governments. Long term objectives of the program are to strengthen autonomy of universities and to support institutional change. In order to realize this long term objective the program gives i) It depends on the voluntary participation of the members.
ii) It is independent of governments and national quality assurance institutions. iii) It evaluates higher education institutions within the context of their internal and external environments. iv) It is based on a detailed self-evaluation process. v) It focuses on improvement. vi) It does not impose any sanctions for the implementation of recommendations. vii) It is a peer review carried out by teams that reflect the diversity of European higher education.
These characteristics emphasize that IEP does not depend on a standardized, and predefined set of criteria. It is rather an evaluation in the form of "plan-do-checkact" cycle in which the institutions are required to present "what they are doing, how they are doing it, how they know it works and how they change in order to improve". Therefore each institution is evaluated within its own strategic goals and by a team consisting of people from across Europe and with institutional leadership experience (EUA, 2013). Rosa et al. (2011: 371) defines the methodology of IEP as "institutional audit" which, instead of using a set of criteria, focuses on the quality assurance processes of institutions.
Institutional Evaluation Program is a five stage process. After the application of a higher education institution to EUA, in the first stage, the association forms an evaluation team. Sursock and Amaral (2008: 40) define the characteristics of these team members as follows: IEP"s visiting panels include small European teams of university leaders, experienced in understanding the specific challenges faced by institutions in their national and international contexts. They are knowledgeable about European and international higher educational trends and are sensitive in applying this expertise to specific national and institutional situations.
In the second stage, higher education institution prepares a detailed Institutional Self-Evaluation Report within the frame work defined by EUA. As the program is selfimprovement oriented self-evaluation report is at the center of the process (Sursock and Amaral, 2008). Following the presentation of the self-evaluation report, in the third stage the team makes a preliminary visit to get to know the institution. After the first visit the team may ask for additional information and documents if necessary. The fourth stage is the second visit that the team makes in order to get more detailed information about the institution, and at the end of this visit the team makes an oral presentation of their evaluation report. In the final stage, Institutional Evaluation Report is officially presented to the institution to be announced on its website (EUA, 2014).
Institutional Evaluation Report, which is the outcome of the evaluation process, consists of good practices of the institution and recommendations for the aspects that need improvement. The report depends on the selfevaluation reports prepared by institutions and the observations of the team members. The reports generally consist of eight parts. First part is the introduction in which the institutions are briefly introduced. Then the teams reflect their observation results in six general headings namely "governance and institutional decision making", "teaching and learning", "research", "service to society", "quality culture" and "internationalization". Last part of the reports is the conclusion and recommendations part in which the team members guide the institution to improve its "weak aspects".
As the Institutional Evaluation Program drew the attention of various higher education institutions around the world since it first started, many evaluation reports containing many findings and recommendations about problematic areas were prepared. For this reason analyzing these reports that were prepared for universities will help to reveal both similar and different problems of higher education institutions.
Purpose of this research is to analyze Institutional Evaluation Reports of higher education institutions that have joined Institutional Evaluation Program between 2013 and 2015, to determine the most frequently evaluated fields in the universities, and to present the areas that most need to be improved and the improvement recommendations made in the reports. The article aims to guide higher education institutions in their efforts to improve their quality by summarizing previous observations and recommendations of Institutional Evaluation Teams.

METHODS
This is a qualitative research in a descriptive survey model. Within the scope of the research, Institutional Evaluation Reports of all institutions that joined Institutional Evaluation Program between 2013 and 2015 has been analyzed; the distribution of problematic fields according to countries and three most mentioned suggestions are evaluated. These reports were obtained from the official website of European University Association (http://www.eua.be/activities-services/institutional-evaluation programme/who-has-participated/iep-evaluation-reports.aspx).
Sample of the research consists of 85 higher education institutions that applied to Institutional Evaluation Program of European University Association between 2013 and 2015, and received an evaluation report as a result. 54 of these institutions are public and the other 31 are private. Table 1 shows the distribution of evaluated higher education institutions according to countries. When this distribution is examined, it is observed that countries mostly from Emerging Europe are interested in the program and there is no participation from developed European countries like United Kingdom, France, Belgium and Germany.

FINDINGS
Findings of reports and recommendations are evaluated under six topics namely, administration, research, education, internationalization, relation to society, and finance. Table 2 shows the frequency and percentage distribution of these topics observed in Institutional Evaluation Reports of universities.
It is seen that the most frequently evaluated topic in the reports is administration. All of the institutions have received a commentary on administration. Research, education and internationalization topics were evaluated in the same frequency in the reports. Relations to society follow them with a little lower frequency and the least evaluated subject in the reports is finance.

Findings and recommendations in Administration
The three most criticized sub-topics of administration in the universities are quality assurance, strategic planning and organizational structure. Table 3 shows the frequency and percentages of most criticized subjects in administration.
Regarding administration, improvement recommendations were made mostly in quality assurance and least in infrastructure. In institutions participating from Romania it is observed that strategic planning is a more prioritized subject than quality assurance. No criticism or recommendations for improvement were made for institutions participating from Montenegro, Slovenia, Lithuania, Hungary, Portugal, Ireland and Latvia about autonomy. In terms of infrastructure, institutions from Turkey, Ireland and Latvia received no criticism from the evaluation teams.
For quality assurance, at least one problematic situation was observed in 78 institutions and 36 of these institutions (46.1%) were recommended that "Improvements should be made according to evaluation and survey results. Evaluation results and improvements should be shared with the university community to ensure that everybody embraces quality assurance". 31 of these universities (39.7%) were recommended to "Define a strategy and a responsible department for quality" and 23 universities (29%) were recommended to "Include all employees and students in the process to enhance quality culture in the organization".
Of the 72 institutions that received an evaluation in strategic planning, 47 (65.2%) were recommended to "Support strategic plans with action plans and concretize and prioritize them with timetables and budgeting". 30 of the institutions (41.7%) were recommended to "Review their strategic plan by updating their vision and mission statements". 20 of the institutions (27.8%) were recommended to "Ensure the cooperation between different departments of the universities towards university strategy, harmonize department plans with university strategic plan, and include all stakeholders in the process." Organizational structure and decision making were evaluated in 65 institutions. 24 of these were institutions (36.9%) were suggested to "Simplify the organizational structure of the university (including the Senate) by decreasing the number of committees". 17 institutions (26.2%) were suggested to "Establish a balance of power and delegation of duties between different administrative boards (like the Senate and the Executive Board) in order to solve the problems arising from a multiple governance structuring". 15 institutions (23.1%) were suggested to "Unite different academic departments offering similar curricula".
When suggestions on human resources were analyzed, it was observed that 31 institutions (64.6%) out of 48 which received an evaluation on this subject were recommended to "Establish a center for the continuous professional development of academic staff, especially in terms of pedagogical skills". 7 of the institutions (14.6%) were recommended to "Review the existing HR systems like promotion, assessment and salaries with objective criteria to provide fair competition for all", and 7 institutions (14.6%) were recommended to "Update course contents and teaching methods and establish a balance between research and education by reducing course load of academic staff".
As a result of the evaluations in the field of student participation in 40 institutions, 33 (82.5%) were recommended to "Increase student participation in administration, decision making and quality processes". 6 of the institutions (15%) were recommended to "Give training to students on subjects like leadership skills to support student participation", and 4 institutions (10%) were recommended to "Review the student representative election process".
In terms of autonomy 23 institutions were evaluated. 13 of these (56.5%) were recommended to "Increase university autonomy especially financially", whereas 11 institutions (47.8%) were recommended to "Ensure some degree of decentralization in decision making". 20 institutions were criticized for their infrastructure and this is the least criticized subject in the reports. 14 of the institutions (70%) were recommended to "Invest in infrastructure to improve facilities like library, cafeteria, laboratories and student lodgings", while 8 institutions (40%) were recommended to "Improve infrastructure (including informatics)".

Findings and recommendations in Finance
It is observed that in finance, recommendations are gathered under two subjects, which are funding and budgeting. Table 4 shows the two most criticized topics in finance and their percentages according to countries.
On the subject of finance, most recommendations were made in funding. It is observed that Turkish universities did not receive any suggestions in funding and the reason might be that the visiting teams evaluated funding efforts of universities as sufficient. Similarly, in FYROM, Ireland and Latvia no recommendations were made on funding but on budgeting.
30 institutions out of 32 (93.7%), which received a commentary on funding, were recommended to "Bring in extra income to the university through internationalization, medical schools, graduates, external stakeholders and alternative resources like lifelong learning and consultancy services". 6 of the institutions (18.7%) were recommended to "Make efforts to remove the legal boundaries against funding". , which received a commentary on budgeting, were recommended to "Make a more effective financial planning", while 3 of the institutions (1.6%) were recommended to "Have a more transparent and accountable budget sharing in the institution".

Findings and recommendations in Research
Lack of a research strategy, insufficient incentives, inadequate resources and inadequate academic staff are the observations put forward in reports concerning research. Distribution of these topics according to countries can be seen in Table 5. In research, research strategy is the most criticized and commented issue. It is seen that all the institutions except for the one from Latvia received a commentary on this subject. The second most criticized subject is insufficient incentives. It can also be seen that inadequate resources are mentioned only in institutions from Romania, Montenegro, Slovenia and Lithuania.
In research strategy, which is pointed out as a field that needs most improvement, 52 institutions of 67 (77.6%) were recommended to "Develop a research strategy for the university and define priorities". 26 of the institutions (38.8%) were recommended to "Adopt an interdisciplinary approach in research", 10 institutions (14.9%) were recommended to "Strengthen research centers and their activities". 35 institutions were criticized for having insufficient incentives in research and 20 of these institutions (57.1%) were recommended to "Establish an office to support academic staff"s research activities". 17 institutions (48.6%) were recommended to "Increase the number of incentives to overcome obstacles like too much workload and insufficient resources", and 3 institutions (8.6%) were recommended to "Use ISI publications and patent as criteria for academic promotion and salary raise".
Inadequate resource was mentioned in 29 institutions and 18 of these institutions (62.1%) were recommended to "Create resources for research through national, international, and industrial cooperation". 9 institutions (31%) were recommended to "Use existing university resources for research more effectively".
Inadequacy of academic staff is evaluated both qualitatively and quantitatively in the reports and 18 institutions receive a recommendation on this subject. 8 of these institutions (44.4%) are recommended to "Train and improve academic staff on projects". 7 institutions (38.8%) were recommended to "Improve hiring and promotion processes of academic staff to increase the number of academics who can mentor doctorate students" and 3 institutions (16.6%) were recommended to "Employ doctorate students in researches to overcome the inadequacy of academic staff".

Findings and recommendations in Education
Two prominent issues in education are teaching methods and programs. Distribution of these issues according to countries is displayed in Table 6. The table displays the distribution of these issues according to countries.
When the total results are analyzed, it is seen that both teaching methods and problems related to programs have received high percentages of recommendations which means the teams believed these two subjects needed improvement in most of the institutions. It can be said that in the institution from Ireland teaching methods don"t need improvement but recommendations are made on programs. In institutions from Hungary and Portugal, all recommendations are on teaching methods and none on programs. In universities from Turkey it is observed that programs need to be improved more when compared to teaching methods. When recommendations on teaching methods are analyzed, it is seen that out of 67 institutions that received recommendations, 53 institutions (79.1%) were recommended to "Define learning outcomes, give more importance to practice, and promote self-learning in order to implement student centered learning more effectively". In 31 institutional reports (46.2%) the teams recommended to "Follow and implement new developments in educational technologies and pedagogic methods" and 3 institutions (4.48%) were recommended to "Share course evaluation results with students and academic staff".
For problems related to programs, evaluation teams made recommendations in 62 institutions. 27 of these (43.5%) were recommended to "Establish a balance between theory and practice in the programs and include more practice oriented courses". 25 institutions (40.3%) were recommended to "Review, assess, and update the programs continuously to ensure that the students acquire generic skills" and 21 institutions (33.9%) were recommended to "Take into consideration employer and alumni views while updating programs".

Findings and recommendations in Service to Society
Service to society has become an important topic in external evaluations especially in the last decade. Comments made on this topic can be gathered around two subtitles which are relations with external stakeholders and service to society. Percentages of these subtitles according to countries can be seen in Table 7.
Under service to society, it is seen that relations with external stakeholders need more improvement than service to society. However in the institution from Latvia it is seen that no recommendations were made on relations with external stakeholders. Similarly in universities participating from Turkey service to society arose as an area that needs more improvement than relations to external stakeholders.
In relations with external stakeholders, 62 institutions were mentioned that they needed improvement and 49 of these institutions (79%) were recommended to "Include external stakeholders in university operations in a more institutional and systematic way". 27 of the institutions (43.5%) were recommended to "Strengthen relations with graduates through Alumni Associations" and 16 institutions (25.8%) were recommended to "Advance cooperation with employers and other institutions". 48 universities received recommendations concerning service to society. 21 of these (43.7%) were recommended to "Use every opportunity to make service to society more institutional". 18 institutions (37.5%) were recommended to "Focus on lifelong learning and continuous education services for the society", while 11 institutions (22.9%) were recommended to "Increase the effect of university on society".

Findings and recommendations for Internationalization
Internationalization has become an important issue for universities especially after Bologna Process. In the evaluation reports, the most criticized points were strategies, mobility and foreign language practices. Percentages of these issues according to countries can be seen in Table 8.
It is seen in the table that institutional strategies and mobility have very close percentages which means they need more improvement compared to foreign language practices. In most of the reports situation of foreign language practices are considered to be adequate. However in universities from Turkey, foreign language practices need improvement more than institutional strategies and mobility. In Latvia no recommendations were made in any of these three subjects. Evaluation teams in 62 institutions made suggestions on institutional strategies. 54 institutions (87.1%) were recommended to "Develop an institutional strategy on internationalization". 12 institutions (19.4%) were recommended to "Establish an office for internationalization activities and employ experts in this office" and 7 institutions (11.3%) were recommended to "Promote internationalization in research and publications".
The teams made recommendations on mobility to 61 institutions. 37 institutions (60.7%) were recommended to "Implement incentives/Take measures (financial support, recognizing credits, internship opportunities etc.) to increase mobility". 35 institutions (57.4%) were recommended to "Attach more importance to and strengthen international partnerships" and 10 institutions (16.4%) were recommended to "Promote university more effectively to attract international students and staff more".
In order to improve foreign language practices, 45 institutions received recommendations from their evaluation teams. 33 institutions (73.3%) were recommended to "Increase the number of courses/programs offered in a commonly spoken foreign language, especially English" and 23 institutions (51.1%) were recommended to "Improve foreign language skills (especially English) of students and academic staff".

DISCUSSION
Institutional Evaluation Program is an important guide for universities. However when participation lists are considered it is seen that most participative countries are Romania, Portugal, Turkey, Slovakia and Spain (EUA, 2014). Schwarz and Westerheijden (2004) mention that less than 50% of European countries have already started national quality assessment activities in the beginnings of 1990"s (as cited in Amaral and Rosa, 2008, p. 23). When participation dates of institutions from developed countries like Germany, The Netherlands and UK are analyzed it is understood that these institutions benefited from this program in the late 90"s or early 2000"s. It may be that universities from these countries are already subject to certain quality assessment procedures at the national level and do not need guidance at an international level. Similarly, institutions from emerging economies might need more guidance to adopt a more European view in higher education.
It is observed that the most evaluated and commented subject on the reports is administration, whereas the least commented subject is finance. A research by Çalık and Bumin (2013) showed that academic staffs working in universities that are located in Ankara, Turkey mostly agree with recommendations made under administration subject in the evaluation reports prepared by EUA. When these two findings are considered, it can be said that academicians and evaluation team members both believe that administration is an area that needs to be improved on in the universities.
Quality assurance is a tool to increase competitiveness in higher education. In order to reach its aim, it is essential for quality assurance to be implemented in every process of higher education institutions, from education to research, administrative services, student affairs and physical resources (United Nations Educational, Scientific and Cultural Organization [UNESCO], 1998). For quality assurance to be implemented in all these processes, all the academic and administrative personnel involved in these processes should participate in quality assurance activities. Participation of all staff in quality assurance activities is one of the necessary conditions for these activities to be successful (Tükenmez, 1996, p. 134). However the employees will participate in these activities only if they internalize the concept of quality assurance. According to the evaluation reports, taking action in line with evaluation results will both ensure that quality assurance is implemented systematically in universities and make it easier for the staff to adopt quality.
It is understood that in finance creating alternative financial resources for the university is the most proposed recommendation in the reports. However it is observed that no recommendations were made to universities participating from Turkey. When ratio of higher education expenses of countries to their Gross National Product is compared, it is seen that Turkey has the highest rate of 1.54% and it can be argued that this is an important factor in this outcome (Eurydice, 2015). In recent years in Turkey, the share of higher education in national education budget remains 30% (Ministry of National Education, 2016), and this leads higher education institutions to search for alternative financial resources as already mentioned in the evaluation reports. As a result, between 1995 and 2005 the share of revolving funds in higher education incomes has risen from 27 to 38% (Buyrukoğlu, 2010, p. 67). Akça (2012) summarizes finance resources in higher education as budget resources or in other words public resources, private sector endorsement procured from universityindustry cooperation, student fees and grants and proposes income based credit facility in order to turn demand into income in Turkish higher education system. Altbach (2012) states that research universities are important for advancing in technological, social and human sciences, that they are both "national institutions" that contribute to their community in various ways, and "international institutions" that make a contribution to global information. Under the title of research which has such an importance in local and global position of universities, the most primary subject that needs improvement appears to be the definition or prioritization of research strategy. Şimşek and Aytemiz (1998, p. 166) mention the problems that universities face in research activities in one of their researches and argue that reasons for not allocating enough time on research are deficiencies in reward system and the heavy workload of academicians. Among the important features of a research university are, promoting research by helping academicians to make enough time for it and selfgenerating research income (Altbach, 2012). For this reason, recommendations on the reports that propose giving training to the staff in research, decreasing their workload, and increasing the number of partnerships to generate income could be important directives for institutions that aim to become affective research universities.
One of the important quality indicators in education is teaching methods. According to all the reports that are analyzed, implementing student centered methods both in the classroom and in planning activities like defining learning outcomes is of vital importance for universities. However Bologna Implementation Reports show that learning activities are seldom used in student evaluations (Eurydice, 2015).
With the development of modern university beginning from the second half of 19 th century, especially in the United States society and market began to take active role in administration as a third stakeholder of the university (Gürüz, 2010). But when the reports are considered, it is understood that this system, which has been implemented effectively in USA for many years, needs to be improved in European universities because according to the reports establishing institutional relations with external stakeholders needs the most improvement under the topic service to society. In addition, the reports show that service to society becomes more important today as a third mission of universities because they emphasize the importance of cooperating with and concentrating on lifelong learning activities for the society. Korkut (2001: 76), advocates that relationship of a university with its society needs to be set on a legal basis for the institution to fulfill this duty. Including external stakeholders into institutional processes is an important component of modern administration models. According to Poister and Streib (1999), an important characteristic of strategic planning, which is vital for the effective management of organizations, is taking into consideration interests of external stakeholders (as cited in Coşkun, 2011: 45).
Lack of institutional strategies and problems in mobility were evaluated almost equally as areas that needed improvement in internationalization. Inadequate internationalization efforts of universities are reflected in Bologna Reports as limited number of countries with a national strategy and as mobility rate that is below 5% in many countries (Eurydice, 2015). Mobility is emphasized as an important subject in the reports because increasing the number of international students will not only increase the income generated from these students and the number of qualified students, but also these students will act as cultural exchange agents (British Council, 2015). Third area that needs improvement in internationalization was foreign language practices and the most recommended activity was to increase the number of programs offered in foreign language, especially English. This recommendation has been argued over in the recent years in Turkey and higher education institutions show a tendency in this direction. However to give education in a foreign language some preconditions have to be met so that the education can be successful and of high quality. Hence a report prepared by British Council in 2015 states that these programs need to be limited until adequate number of students and academicians with a sufficient level of English are obtained (British Council, 2015).

RECOMMENDATIONS
Considering the immense increase in the number of universities around the world, for the quality improvement of these universities their participation in international external evaluation programs like IEP. For this purpose, universities can be supported by their governments especially financially to join these programs.
Main purpose of external evaluation is to maintain continuous improvement in universities and continuous improvement can be monitored through follow-up evaluations. For this reason it will be beneficial for universities that participated in IEP, to prepare action plans according to recommendations in their evaluation reports and apply for a followup evaluation.
It was previously mentioned that participation of universities from developed European countries to the program has been very scarce especially in the recent years. The reasons for this low rate of participation can be a separate research subject. EUA has been guiding universities as a quality guide not only in Turkey but also in many countries all around the world since the 2000"s. Evaluation reports prepared by the Association can be evaluated in a country to see the progress in quality assurance and other areas over the years. Also, involvement of management and employees are essential for the success of these activities. A research can be done to evaluate the attitude of administrators and employees in the universities towards external evaluation activities.