Measuring the institutional efficiency using data envelopment analysis and analytic hierarchy process : The case of a Mexican University

1 División de Investigación y Posgrado, Facultad de Ingeniería, Universidad Autónoma de Querétaro, Centro Universitario, Cerro de las Campanas s/n, C.P. 76010, Querétaro, Qro, México. 2 Universidad Politécnica de Querétaro, Carretera Estatal 420 S/N, El Rosario, C.P. 76240, El Marqués, Querétaro, México. 3 Facultad de Contaduría y Administración, Universidad Autónoma de Querétaro, Centro Universitario, Cerro de las Campanas s/n, C.P. 76010, Querétaro, Qro, México. 4 Hidrus S.A de C.V. Privada San Simón 51, Residencial San Mateo C.P. 76912, Corregidora, Querétaro, México. 5 Escuela de Ingenierías y Arquitectura, Facultad de Ingeniería Civil, Universidad Pontificia Bolivariana-Seccional Montería, Km. 8 vía a Cereté, Montería, Córdoba, Colombia.


INTRODUCTION
The public education policies in several countries are changing the traditional arguments which prefer the equity toward achieving goals of educational efficiency (Alam, 2009).The situation here at UAQ is not the exception.To this end, it is necessary to use techniques that enable an objective evaluation of the educational performance.UAQ like so many other institutions is compromised with issues of academic excellence and for the improvement of the current educational systems (Rubio, 2006).Consequently, it is very important to *Corresponding author.E-mail: jaaltami@gmail.com.evaluate the performance of faculties, in areas of research, teaching and administration through indicators and performance models that are of similar complexity with today's educational demands (Alam et al., 2010).
Assessment of higher education is a common practice in several countries, as can be seen in strategies for improving the quality of higher education in Europe (Martin, 2006).In fact, there are already performance indicators in place in certain areas and their results have impacted the decisions of students and employers (Colbert et al., 2000).
In UAQ, which was established in the 17th century, there are currently 133 educational programs (EP) being taught, from the level of higher university technician to doctorate degrees in 13 faculties or Dependencias de Educación Superior (DES).
Carrying out an evaluation of performance of any organization requires an understanding of the goals and objectives of the same (Johnes, 1992).In the case of the UAQ each of the 13 DES has its own peculiarities; therefore, the complexity of making a proper and just evaluation which satisfies all parties and reflects the actual behavior of each.
The objectives of this research were to: (1) investigate if a methodology is possible using AHP and DEA to determine a performance index for institutional evaluation and (2) to propose a tool for planning processes.This information is crucial in order to improve the decision making in higher education.

LITERATURE REVIEW
The AHP elaborated by Saaty (1977Saaty ( , 1980Saaty ( , 1982) ) were designed to solve complex problems concerned with multicriteria.Many researchers have applied AHP and DEA to solve multi-criteria decision making (MCDM) problems in a number of different scenarios such as: finding the optimum site for a railway station in the city of Mashad, northeast Iran (Mohajeri and Amin, 2010); developing a supplier selection system integrating fuzzy AHP and fuzzy DEA (Kuo et al., 2010); constructing a weight index for organizational innovation in Taiwanese high-tech enterprises (Liu et al., 2010); and developing an integrated model of operation effectiveness in small and medium sized manufacturing enterprises (Norita and Robin, 2009).When evaluating higher education institutions, authorities face an overwhelming amount of criteria without the proper tools to make their decisions (Hashim et al., 2010).This study combines AHP and DEA to establish a framework to measure institutional efficiency.
In Vaidya and Kumar's (2006) review, 150 articles of AHP applications were categorized by common criteria and different application areas.A small number of papers related to education were detected and even less concerning DEA.
The AHP requires that the decision makers supply assessments regarding the relative importance of every opinion which specify a preference for each alternative in the decision making process.The AHP's output is a classification sorted by priorities of the alternatives of decisions made, based on global preferences expressed by the decision makers (Tzeng and Huang, 2011).On the other hand, there have been several papers about efficiency in higher education using DEA.The most important ones are those of Rhodes and Southwick (1986) who studied the efficiency of the public and private universities of the USA, McMillan and Datta (1998) who used DEA to assess the relative efficiency of 45 Canadian Universities, and Ng and Li (2000) who examined the effectiveness of the reform implemented in the mid 80's in China.Abbot and Doucouliagos (2003) used DEA to evaluate research and teaching performance in public universities of Australia while Bougnol and Dulá (2006) applied DEA to assess performance in higher education.Figueiredo de Franca et al. (2009) presented a DEA methodology to assess the impact of the asymmetry of information on efficiency with an application to the higher education systems in Brazil.
However, Sav, 2012 compared private for-profit colleges to publicly owned colleges in terms of their operating efficiency and productivity using DEA.
Concerning higher education in Mexico few studies have been conducted using DEA.Two of such researches were held by Siegler ( 2004) who used DEA to evaluate the relative efficiency of the public institutions on economics research in Mexico City, and Güemes-Castorena ( 2008) who developed a model for higher education funding based on the efficiency achieved by each Mexican university.The model describes the performance efficiencies of the universities and proposes future funding based on those measurements.

METHODOLOGY Analytic hierarchy process model for DES of the UAQ
For the development of this research the indicators extracted from Programa Integral de FortalecimientoInstitucional (PIFI 2008(PIFI -2009) ) 1 -Integrated Program of Institutional Strengthening were used.The aforementioned results were used to evaluate the performance of the 13 DES of the UAQ on the basis of multiple criteria.
The results obtained from the integral evaluation of the PIFI 2008-2009 as shown in Figure 1 were selected using the following indicators: Capacity (1.1, 1.2, 1.3, 1.4, 1.5) as related to the full time professors and the "academics bodies" which are composed of full time professors working in the same research field; competitiveness (1.7, 1.8, 1.9) as related to educational programs; institutional selfevaluation (3.1, 3.2, 3.3, 3.4, 3.5, 3.6); updating of the planning in institutional scope (4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 5.1, 5.2, 5.3) and enrollment.In all, there are 26 Criteria and 13 alternatives (DES), which constitute the inputs for the AHP with the global goal of obtaining the best DES for UAQ.
Each of the criteria was compared in pairs so as to determine its relative importance.Then, the DES were compared with each other in pairs in regard to each of the 26 criteria.The methodology used for this model is as follows: 1. Selection of the global goal, that is "selection of the best DES", which by its academic results (capacity and competitiveness, institutional self-evaluation and updating of the planning in the institutional scope) would be in the best position.2. Selection of criteria, which in this case correspond to the 26 criteria; 25 PIFI indicators (1.1,1.2,1.3,1.4,1.5,1.7,1.8,1.9,3.1,3.2,3.3,3.4,3.5,3.6,4.1,4.2,4.3,4.4,4.5,4.6,4.7,4.8,5.1,5.2,1The PIFI is a project encouraged by the SEP (Secretaría de Educación Pública -Ministry of Public Education) to integrate planning, evaluation and financing in order to improve the capacity and academic competitiveness, fundamentally understood as the consolidation of academic bodies and the accreditation of educational programs respectively, as well as, to improve the management and mechanisms of accountability.

DEA model for DES of the UAQ
The methodology of data envelopment analysis (DEA) was first characterized by Charnes et al. (1978) as a way of comparing the efficiency of decision making units (DMU's) that have multiple inputs and outputs.A DMU can be a company offering a service, manufacturing or, as in this case, an institution of higher education.
DEA has been widely used to evaluate the relative performance of a set of DMU's based on multiple criteria.And since this requires very few assumptions, DEA has opened possibilities for institutional evaluations which can generally be very difficult to carry out, due to the complex nature of the relations between multiple inputs and outputs.
DEA, unlike other methods, use financial and non-financial elements.This method is also particularly appropriate to assess the efficiency of public universities because they operate outside the market.Criteria such as profitability and income are "not satisfactory".This is because public universities are not geared in order to make a profit.Furthermore, in these institutions the primary source of financing does not come from the sales of goods and services.
In this research it is assumed that if a DES, named DES1 is able to produce or generate Y1 output's units with X1 input's units, then other DES must also be able to do the same if they are operated efficiently.Similarly, if DES2 is able to produce Y2 output's units with X2 input's units, then the other DES must also be able to do the same.DES1 and DES2 can be combined to generate a DES (virtual) composed of inputs and outputs of them.This virtual DES is used like a standard of performance for the DES.

Inputs and outputs
In a model of DEA, undesirable inputs and outputs may be present.It is possible to have undesirable outputs as the number of defective products.Therefore, its desirable outputs reduce their number to improve performance (Zhu, 2009).
Problems arise in conventional models of DEA because it is assumed that the outputs should be increased and the inputs decreased in order to improve performance or reach the border of best practices.There are situations in educational practice where certain inputs need to be increased or some outputs decreased to improve institutional performance, these are then called "undesirable".In this case the undesirable inputs are: PTC Doctorado and PTC SNI.Seiford and Zhu (2002) developed an approach to deal with these input/output undesirables in envelopment models with variable returns to scale.DEA classification invariances were used in order to ascertain the efficiencies and inefficiencies which are invariant to the data transformation.
The inputs and outputs for this model are as shown in Table 1.

Model of DEA
The model considered herein is that of variable returns to scale with undesirable inputs.In order to increase the institutional efficiency two inputs were increased (PTC Doctorado and PTC SNI) and these were not to be reduced.

Variable returns to scale oriented to input model
Denoting with xij I as the inputs to be increased and xij D as the inputs to be decreased, in order to improve performance of a DMU.xij I was multiplied by -1 and then an appropriate ui was found to obtain The following model is based on the previous one, using the aforementioned transformation.A postgraduate program which pertains to PNPC means that it is recognized by academic community and society in regards to its quality; this recognition is the result of evaluation and monitoring processes conducted by a committee of researchers nominated by CONACYT.The aim of PNPC is to guarantee the quality of higher education institutions in Mexico.CA: CuerpoAcadémico (CA) -Academic Body is a set of professors/researchers who share one or more common lines of study, whose objectives are intended for the generation and/or application of new knowledge.In addition, because of the high degree of specialization that is reached in participating in the research and teaching, they provide a high quality of education.The academic bodies support academic institutional functions and integrate part of the system of higher education within the country.PE Acreditados: ProgramasEducativosAcreditados (PE) -Accredited Educational Programs: These are educational programs where academic bodies composed of professors within the institutions of higher education throughout the country, evaluate and certify the functions and the academic programs that are taught.They then delivered to the managers of these institutions, recommendations regarding improvement, which are contained in evaluation reports.and 4. The 13 th DES had the best results and the 9 th DES had the worst.

Computational results of DEA
The thirteen DES that exist in UAQ with their respective values of inputs and outputs are presented in Figure 7.For this research, DEA Frontier TM software was used, which is a complement to Excel as developed by Joe Zhu (Figure 5).

Analysis of the models
As presented in Figure 6, when comparing the two methods, the first three DES and the last three, have similar rankings which are understandable because within the first three, its academic positioning is highly consolidated.There are 10 graduate programs featured at UAQ which are part of PNPC.Of these, 6 are offered in DES 13, 2 belong to DES3 and 2 to DES2.The final three DES have a limited number of PTC with Ph.D. and any program within the PNPC.
The difference in rankings given, for example in DES10, 5º in DEA and 10º in AHP, is that DES 10 has a better position, using DEA, because it is the fifth DES in relation to PTC with Ph.D. and it is the fourth as related to PTC SNI.However, when the analysis is done with all the indicators utilizing AHP, its academic advantage is diminished.
In the case of DEA, the number of full time professors with doctorate degrees and SNI, make substantial contributions to increasing the academic competitiveness.These factors could be relevant in determining the best DES using DEA instead of utilizing AHP which requires more time to be implemented.

Conclusions
In this research an approach to measure institutional efficiency combining AHP and DEA has been established.The majority of results obtained using AHP correlated with those of DEA and, also reflected a widespread perception about how the performance of a university might be evaluated.The modeling of AHP and DEA combined, offers decision makers an opportunity to learn more about the educational systems in order to define policies that permit academic authorities to make better decisions in the short and long term.
When planning, and developing programs within the universities it is necessary to generate and analyze the indicators of academic performance, in order to improve academic competitiveness, stimulate educational innovation and strengthen academic ability.The combination of AHP and DEA can be used to facilitate this process.
According to Güemes-Castorena (2008), the allocation of resources in the higher education system in Mexico, correlates to the enrollment, the professors and administrators.However, they are poorly correlated to the SNI and academic efficiency.
If authorities want budget allocations from certain government programs to be developed into successful and equitable action, it would be necessary to promote efficiency and higher levels of academic performance in the institutions.Using a combination of AHP and DEA can facilitate the task of laying the foundations and criteria for the allocation of financial resources.
This approach can contribute to analyze the institutional efficiency and the planning processes of higher education.

Figure 2 .
Figure 2. Hierarchy of the Model of the DES-UAQ.
) software was used to record the calculations.The results obtained are shown in Figures3

PTC Doctorado:
Full time professors with a Doctoral Degree.PTC SNI: Full time professors that belong to the SistemaNacional de Investigadores (SNI 2 ) -National System of Researchers.PNPC: ProgramaNacional de Posgrados de Calidad (PNPC) -National Program of Graduate Studies of Quality, which is jointly administered by the Secretaría de EducaciónPública through the Subsecretaría de Educación Superior and ConsejoNacional de Ciencia y Tecnología (CONACYT) -National Council of Science and Technology.The program has established that its mission is "to promote the continuous improvement and the quality assurance of the national graduate studies, which offers support to increase (in) scientific capacity, technological, social, humanities, and innovation of the country".

Figure 3 .
Figure 3. Comparative results obtained from each of the DES with regard to the 26 criteria, in addition to its relative position with regard to the overall goal of selecting the best DES.

Figure 4 .
Figure 4. Relative position of the DES of the UAQ with respect to the overall goal of selecting the best DES.

Table 1 .
Inputs and outputs for DEA.