Performance auditing: Development of an audit model to evaluate efficiency, effectiveness and economy of the performance of a business

Performance auditing as an audit discipline has developed seriously since the late 1970s. Owing to its developing nature, however, it has been noted that it lacks formal audit methodologies. One of such audit methodology is in the specific design of an audit programme; this is important as the audit programme forms the basis of the audit. This paper presents a methodology for creating an audit programme specifically for performance auditing and further presents this in a model format. To develop this methodology, grounded theory was applied. The audit methodology and model were evaluated against the International Organization of Supreme Audit Institutions (INTOSAI) performance audit standards and were also tested in two private-sector companies. The proposed audit methodology is based on the relationship between process, risk, control and the audit programme; it uses risk-based auditing, and the specific performance audit objectives are inverted to form the primary performance risk; in addition, benchmarks and criteria are included; the model caters for both highlevel and detailed information. This methodology and model benefits both performance auditors and business managers.


INTRODUCTION
Assessing risk is the core function of an audit (Hayes et al., 2005: 23) and the audit programme comprises the auditor's plan of action (Dickinson, 1982: 84). Accordingly, meticulous preparation of the audit programme is important in order to define the audit questions, the information needed and the audit design (International Organization of Supreme Audit Institutions (INTOSAI) and International Standards of the Supreme Audit Institutions (ISSAI), 3000: s.2.1), and also to specify the analytical procedures and tests of detail to be performed in response to identified risks (Patrick and Van Esch, 2007: 233). In a study conducted by the US Securities *Corresponding author. E-mail: nirupa.padia@wits.ac.za. Tel: 083 660 5805, 011 717 8022. Fax: 0865535068. and Exchange Commission (SEC) of audit engagements that received SEC enforcement actions from 1987 to 1997, 44% of cases reported deficiencies in audit planning (Beasley et al., 2001: 1).
Performance auditing differs from financial auditing. According to Chapter 1 section 1 of the Auditing Profession Act (Act 26 of 2005) (APA), an audit is defined as: the examination of, in accordance with prescribed or applicable auditing standards; (a) financial statements with the objective of expressing an opinion as to their fairness or compliance with an identified financial reporting framework and any applicable statutory requirements; or (b) financial and other information, prepared in accordance with suitable criteria, with the objective of expressing an opinion on the financial and other information.
INTOSAI ISSAI 3000 presents three important differences between performance auditing and financial auditing: 1. While financial auditing tends to apply relatively fixed standards, performance auditing is more flexible in its choice of subjects, audit objects, methods and opinions. 2. Performance auditing is not a regular audit with formalised opinions, and it does not have its roots in private auditing. 3. The overall aim of performance auditing is to promote economy, efficiency and effectiveness.
A performance audit is generally understood as pertaining to the public sector (Summa, 2002: 18), but it is also practised in the private sector under descriptive titles such as operative audits, management audits, quality audits and environmental audits (Summa, 2002: 19). Summa (2002: 19) explains the primary difference between their use in the public and private sector as follows: performance audits are a part of the external control system operating in public organisations, whereas private-sector control mechanisms (that is, operative audits, management audits, quality audits and environmental audit) are internalised forms of corporate control. In South Africa, performance auditing is applied in the public sector (Prinsloo and Roos, 2006: 4), although attempts have also been made to introduce it into the private sector (Loots, 1989: 406). Performance auditing is defined as the audit of economy, efficiency and effectiveness, and it embraces: A. An audit of the economy of administrative activities in accordance with sound administrative principles and practices, and management policies.
B. An audit of the efficiency of the utilisation of human, financial and other resources, including the examination of information systems, performance measures and monitoring arrangements, and the procedures followed by audited entities for remedying the identified deficiencies.
C. An audit of the effectiveness of performance in relation to achieving the objectiveness of the audited entity, and an audit of the actual impact of activities compared with the intended impact (ISSAI 3000: Standards and guidelines for performance auditing).
Performance audit is a nascent audit discipline that has been developed seriously since the late 1970s (Loots, 1989: 1;Pollitt and Summa, 2002: 1;Daujotaite and Macerinskiene, 2008: 178). Largely due to its nascent nature it lacks formal audit methodologies (Lonsdale, 2002: 146); hence, an improvement in this regard would contribute to the effective practice of performance auditing.

LITERATURE REVIEW
In 1986, the International Congress of Supreme Audit Institutions selected the term 'performance audit' as the common future term to be applied to audits examining the economy, efficiency and effectiveness of operations (Hatherly and Parker, 1988: 21). There are various synonyms for performance auditing, such as broad scope auditing, cost-effectiveness auditing, efficiency auditing, operational auditing, project auditing, and value for money (VFM) auditing (Funnell, 1998: 446). Prinsloo and Roos (2006: 3) also offer the synonyms value-for-money audit and operational audit. Barzelay (1996: 22) lists five types of performance audit: efficiency audits, programme effectiveness audits, performance management capacity audits, performance information audits, and best practice reviews.
In historical terms, performance auditing is a relatively new procedure (Pollitt and Summa, 2002: 1;Loots, 1989: 1). The main thrust of development started in the mid 1970s (Loots, 1989: 1; Daujotaite and Macerinskiene, 2008: 178) and it is generally accepted that performance audit is still a developing field, which is perceived to be without a prescribed list of performance audit methods (Lonsdale, 2002: 146). Sloan (1996: 146) warns that, in practice, many performance audits are less than clear on matters of methodology, evaluative criteria and the development of recommendations. This is iterated by Shand and Anand (1996: 60), who warn that performance auditing itself covers a range of approaches in terms of scope, methodology and form of reporting.

The purpose of an audit programme
According to Mautz (1964: 170), the audit programme fulfils two purposes: it serves as a plan of attack for the verification problem at hand and as a record for the audit work performed.

Risk-based auditing
The risk-based approach requires an understanding of the risks within an organization/audit that are, in turn, addressed by systems of internal control; the converse approach would be to identify a set of controls and then determine if these are applied throughout the organization (Spencer, 2003: 12).
According to Hayes et al. (2005: 23), business risk results from significant conditions, events, circumstances, actions and inactions that may impede the company's ability to execute its strategies; furthermore, auditors are required to include business risks in the planning process. These authors conclude that assessing risk is the core of the audit.

Effective performance auditing
Mayne (2010: 1) asks whether performance auditing is doing the right thing the right way. He acknowledges the significant changes public management has undergone in the last 30 years, noting the increased popularity of performance audit. Leeuw (2011) posits that there is limited evidence to show that performance audits are effective; although there are indicators that show a difference but he warns that because the causal relationship between audits and 'change' is not evident, one should be cautious to conclude that audits are producing effects. In an earlier paper Leeuw (2009: 3) recognised that evaluation, monitoring, inspection and performance auditing had become a booming business, but questions what 'added value' means.

Limitations
The sample size was limited to two large private-sector retail companies. Some companies were reluctant to participate in the field test citing confidentiality concerns and a lack of available staff to participate in the exercise.

RESEARCH METHODOLOGY
Grounded theory was developed by sociologists Barney Glaser and Anselm Strauss in 1967. The procedures of grounded theory are intended to develop an integrated set of concepts that provide a thorough theoretical explanation for the social phenomenon under study (Corbin and Strauss, 1990: 5).
Grounded theory is not restricted to the social sciences but has also been applied in accounting and auditing (Gurd, 2008: 122;Parker and Roffey, 1997: 212), as well as in interpretive management accounting research (IMAR), (Elharidy et al., 2008: 139).
Such methodology and procedures are now among the most influential and widely used modes for carrying out qualitative research when the researcher's principal aim is to generate theory (Strauss and Corbin, 1997: vii). Grounded theory was applied to develop a performance audit methodology.
Two research questions were formulated for this study: 1. Can a performance audit programme methodology and model be compiled using grounded theory? 2. Can the proposed methodology and model be applied in practice?
Open coding was used to divide the data into segments and then scrutinise them for commonalities reflecting categories or themes. The data were evaluated and a general list of segments was compiled using a process of note-making. Subsequently, the commonalities, categories and sub-categories found in the segments were explored. In this respect, the primary segments consisted of planning, fieldwork and reporting. In the primary segment of planning, 37 sub-categories were identified.
Using axial coding, interconnections were made between categories and subcategories. From the 37 sub-categories identified, interconnections were evaluated and six general categories were created. These comprised objective, risk, control, process, audit and audit results.
Using selective coding, the interrelationships between categories and sub-categories were combined to create a sense of order and meaning. The following interrelationships were considered.
A. Risk and control: From a risk-based approach, controls are seen to mitigate risk. This agrees with Dittenhofer's (2001: 471) view that the antithesis of risk is control, and the control therefore addresses the risk. B. Types of control: According to ISSAI 3100 (Performance audit guidelines-key principles), controls can be of a qualitative (criteria) or quantitative (benchmark) nature. This is congruent with Edgett and Snow (1996: 11), who argue that benchmarking needs to be applied to the measure of success, and that benchmarks can come from internal and external sources. C. Actual controls: In the methodology, the actual controls are divided into preventative controls, detective controls, manual controls and system controls. D. Audit programme, control and risk: Is there an order to these three concepts that is universally acceptable? The audit programme tests controls, controls exist to address risk; the logical order seems to be to first identify risks, then identify the controls that address these risks, followed by the audit programme that tests the existence, effectiveness and sufficiency of the controls. E. Audit programme and audit results: The audit results are derived from the requirements of the audit programme. F. Process and risk: A risk approach is assumed that is contingent on the process. The risk-driven approach is both effective and efficient (Colbert and Alderman, 1995: 43); effective because it focuses the auditor's energies on areas of high risk, and efficient because the only work that is performed is that which addresses a specific risk. G. Risk is determined by the type of transaction and the process under review: Actual risks refer to the actual process; the approach taken in this paper is to include inherent risks since actual controls also address inherent risks, for example disaster recovery procedures. H. Risk and objectives: The risk is the inverse of the objective, for example incompleteness (risk)-completeness (objective) ( Table 1). I. Business process and company objectives: The business process is contingent on the objectives of the company.
In the model development stage relationships between concepts were identified and collated, and a theory was developed. Table 2 shows the resultant sub-categories and categories using grounded theory.
The proposed audit methodology and model was tested in two private sector retail companies with multimillion and annual turnovers. The first company retailed electronic goods, the second retailed food and household goods.

RESULTS AND DISCUSSION
The results are as follows: 1. Through the application of grounded theory a methodology to create an audit program specifically for performance audit was developed. A model was also created, 2. The methodology and model was tested successfully on two retailers.

The audit methodology and model
The proposed methodology was developed using grounded theory, and a model was created. The proposed methodology is as follows: Through the initial application of risk-based auditing, and by assuming there is a relationship between process, risk and control, the audit program can be compiled as a means to test the presented controls. As this is performance auditing with three specific objectives (efficiency, effectiveness and economy), the performance risk reflects the inverse of the objective (inefficiency, ineffectiveness, non-economy). Also included in the controls are the criteria and the benchmarks. From the methodology a spreadsheet model was compiled, which allows for recording highlevel and detailed information.

Explanation of the audit model
The major columns in Table 3 are, from left to right, process, risk, control, audit programme. Table 3 presents the spread layout. Table 4 explains how the model works. In applying the model, the auditor undertakes the following steps: 1. The auditor first fills in the process column, reflecting the entity's business processes that are being audited, preferably in point form. 2. The second column is risk; here the auditor records the actual or inherent risk for each of the risk types for that particular part of the listed process. The materiality of the particular risk (rating) is also recorded. 3. For each identified risk, a matching and existing control is suggested. If there is no control for an actual, material risk, then this may become a finding. 4. The benchmark compares actual quantifiable output to a norm. If the actual output is below the benchmark, then a risk situation may exist. Thus, the auditor may wish to divide the benchmark into lower, middle and upper tiers to determine the extent of the risk into which the actual output falls. 5. Criteria are the pre-set standards against which actual performance is measured. A difference between a criterion (what should be in place) and the control (what is in place) is cause for concern and may result in a finding. 6. The control risk is based on the auditor's judgement when considering whether a risk will be prevented, or detected and corrected. This is read in conjunction with the risk rating of the main risk categories, which gives the auditor an indication of the extent of the testing required. 7. The audit programme tests the existence, effectiveness and sufficiency of the controls. 8. When concerned with fieldwork, the identified audit findings are placed alongside the audit programme.

Evaluation of the audit model
The evaluation of the proposed model was done by applying two International Standards of the Supreme Audit Institutions (ISSAI) statements: 1. ISSAI 3000 standards and guidelines for performance auditing based on INTOSAI's auditing standards and practical experience, 2. ISSAI 3100 performance audit guidelines: key principles.
ISSAI 3000: Standards and guidelines for performance auditing based on INTOSAI's auditing standards and practical experience. The ISSAI 3000 presents two approaches to the performance audit: the results-oriented approach and the problem-oriented approach. In terms of the results-oriented approach, the auditor studies performance and relates observations to the given norms (for example, goals, objectives, regulations) or the audit criteria. Using this approach, shortcomings are likely to be defined as deviations from norms or criteria. Consequently, the recommendations identified are aimed at eliminating such deviations. The problem-oriented approach, on the other hand, deals with problem verification. In terms of this approach, the following questions were dealt with: "Do the stated problems really exist? And, if so, how can they be understood? And what are the causes"?
The proposed audit model caters for a results-oriented approach in that it allows for the recording of any process (subdivided into input, process, output and impact), as well as the criteria and the benchmark values. The proposed audit model also caters for the problemoriented approach in that the design of the proposed methodology is risk based, recording problems and shortcomings as risks to normal processing. Furthermore, the section for actual controls is divided into four subsections (systems controls, manual controls, Part 3 of ISSAI 3000, 'field standards and guidance: initiating and planning the performance audit' makes the statement that the most important steps in drawing up an audit proposal are: 1. Defining the specific issue to be studied and the audit objectives, 2. Developing the scope and the design of the audit, 3. Determining the quality assurance, the timetable and the resources.
The proposed audit model facilitates these requirements as follows: Firstly, the audit objectives for a specific performance audit may be determined by management or by the auditor. The proposed methodology requires that risks be ranked, which means that, for a collection of subprocesses, key risks are identified from the ranking, which gives focus to the issue to be studied. Secondly, the scope and the design of the audit can be deduced from the proposed model, in that it contains key information about risks, controls, criteria, risk ranking and control risk. Thirdly, the proposed model is in spreadsheet form, allowing the auditor to add additional columns alongside audit tests, identifying allocated time and resources, as well as actual time.
In the ISSAI 3000 preamble, the point is made that this standard is not a normative or a technical document, or a handbook, but contains a number of guidelines and other information with practical implications that take into consideration the special premises and features of performance auditing. The proposed audit model has taken the proposed guidelines and applied them in a particular manner. The standard does not offer such detailed application.
ISSAI 3000 warns against streamlining, that is, making a process efficient by stripping off nonessentials, warning that advanced performance auditing is complex investigatory work requiring flexibility, imagination and high levels of analytical skills, as well as hampering creativity and professionalism (ISSAI 3000 s.1.8, 1.9; s.3.3.1). In response to this warning, the proposed audit model allows for high-level interpretation (process, risk, control), as well as detail: the 6 general categories identified in the axial coding phase are subdivided into 37 sub-categories and the model's line-by-line approach requires attention to detail.
ISSAI 3000 emphasises the term impact in various forms. For example; 1. Comparing the actual impact of activities with the intended impact. 2. The impact of the audit findings on an organisation.

How the audited entity monitors impact.
The proposed audit model contains columns for process, risk and controls. In the process column, the user can divide the process into input, process, output and impact stages. If a process has a notable impact, this can be described in the risk column. The criteria column will reflect the expectation of the impact, and the benchmark column will reflect the quantifiable measure of the expected impact. ISSAI 3100: Performance audit guidelines -key principles. In line with guidelines for ISSAI 3100 s.2.4.1.13: Performance audits should have suitable audit criteria that focus on the audit and provide a basis for developing audit findings. The audit criteria, which can be of a qualitative or quantitative nature, should be reliable, objective, useful, and complete. It should be possible to identify the source of the audit criteria used.
The proposed performance auditing model satisfies this requirement in that it includes a column for both qualitative and quantitative criteria, that is, "criteria" and "benchmark".
According to ISSAI 3100 s.2.1: "Performance auditing is an independent and objective examination of government undertakings, systems, programmes or organisations, with regard to one or more of the three aspects of economy, efficiency and effectiveness, aiming to lead to improvements." It is the opinion of the authors of this paper that performance audit can apply to Table 3. The proposed performance audit model.

Ref. Description Explanation
1 Process Process refers to the business process in the audited entity. The auditor may further wish to divide the process into input, processing, output and impact stages.

Risk
Risk refers to the business risk and comprises the three primary performance audit risks, that is, ineffectiveness, inefficiency, non-economy (aka uneconomical). These risks are derived from the performance audit objectives of effectiveness, efficiency and economy. Within each risk type a distinction is made between inherent risk and actual risk. Inherent risk is the potential risk within the business and the business environment. It is appropriate to include inherent risk as actual controls exist to address it, for example disaster recovery procedures. An assessment of actual risk is obtained from fraud reports, audit reports, interviews, etc. The rating applies to the presented risk, e.g. if the risk is high the rating field can reflect a number from a risk ranking, for example, 1-10, or it can be a colour-coded index. 2a

Inefficient risks
The converse of the efficiency performance objective 2ai Inherent risks applying to inefficient risks As described 2aii Actual risks applying to inefficient risks As described 2aiii Risk rating applied to inefficient risks As determined by the auditor, for example, if the risk is high the rating field can reflect a number from a risk ranking, for example, 1-10, or it can be a colour-coded index. It should be read in conjunction with the control risk. 2b Ineffective risks The converse of the effectiveness performance objective 2bi Inherent risks applying to ineffective risks As described 2bii Actual risks applying to ineffective risks As described 2biii Risk rating applied to ineffective risks As determined by the auditor, for example, if the risk is high the rating field can reflect a number from a risk ranking, for example, 1-10, or it can be a colour-coded index. It should be read in conjunction with the control risk. 2c Uneconomical risks The converse of the economy performance objective 2ci Inherent risks applying to uneconomical risks As described 2cii Actual risks applying to uneconomical risks As described 2ciii Risk rating applied to uneconomical risks As determined by the auditor, for example, if the risk is high the rating field can reflect a number from a risk ranking, for example, 1-10, or it can be a colour-coded index. It should be read in conjunction with the control risk.

Control
Control refers to the existing controls, further distinguished by preventative and detective controls, each separated by manual or systems controls. The distinction between the different types of control is made since each type addresses the particular risk in a different way, for example, preventative system controls for a particular process and risk are dissimilar to detective manual controls. This distinction also facilitates finding the root cause of a problem resulting from a weak control.
3a Preventative control Controls that prevent a risk situation from arising.
3ai Manual preventative control Manual controls that prevent a risk transaction/situation from arising.
3aii Systems preventative control Electronic controls, such as sign-in rights, edit checks, data integrity checks, etc.
3b Detective control A control to detect a risk transaction that has occurred.
3bi Manual detective control Manual controls to detect a risk transaction that has occurred, for example, a paper trail of signatures, delivery notes, invoices, documented testimonial evidence, etc.
3bii Systems detective control Electronic controls, such as an audit trail to identify who performed the transaction, the date and time of the transaction, details of the transaction, etc.
3c Criteria This is the pre-set standard against which actual controls are measured.
3d Benchmark The standard of performance against which actual performance is measured. This is also a synonym for quantifiable criteria.

Control risk
Control risk, which is the risk that a material misstatement will not be prevented or detected and corrected. The control risk is determined by the auditor, for example, if the risk is high the rating field can reflect a number from a risk ranking, for example, 1-10, or it can be a colour-coded index.

Audit programme
The audit programme is compiled from the risk and control information. 6 Audit results Contain the results of the audit tests.
both the private and the public sectors. This was also the conclusion of Loots (1989: 406).

Benefits of the audit model
The proposed model collates key planning information for the audit in a single document. Moreover, it can be applied by the internal and external audit function, both in the public and the private sector, to compile an effective performance audit programme, which reduces the probability that an auditor will not be aware of key risks affecting the audit entity. The proposed model facilitates an independent reviewer in determining how the audit programme was designed, what the key risks were and how these were addressed; and can easily verify the integrity of the relationship between process, risk, control and the audit programme. The risk-ranking approach gives assurance to company management that the auditor has identified and evaluated key risks pertinent to the business process/project/strategy. If the same methodology is applied to multiple audit teams it has the potential to reduce audit plan design time and supervisor review time, and to align risks shared between multiple audits.

Field-test results
Results from the field-test indicate that the proposed methodology and model can be applied in practice. Two field tests were performed, both in the private sector. The first retailer had an annual turnover exceeding R10 million, the second retailer had an annual turnover exceeding R100 million. Interviews were held with key staff and the performance audit model was populated with data. The field tests were applied to established, high annual-turnover retailers as this assumes a competent management team exists, that the interpretation of the field test results by senior management would be competent, and that the two key questions would be answered accurately.
In the first field test four main focus areas of the business operations were identified: stock, sales, staff, cash and liquidity. In the second field test, seven main focus areas of the business operations were identified: purchases, stock, administration, staff, sales, customer and environmental.
On completion of the audit program, two key questions were put to senior management: 1. Have all material performance processes and risks pertinent to the business been identified by the proposed model? 2. Does the audit program addresses key performance risks?

Padia and Vuuren 10425
Management answered in the affirmative to both questions in both field tests. The first field test identified four findings, which management accepted. The second field test identified two findings, which management accepted.

CONCLUSION AND RECOMMENDATIONS
A performance audit is distinct from regularity or financial audits; some of its key distinguishing features are the three specific performance objectives, criteria, bench marks and performance measurement. This paper identified a need for an audit programme methodology specific to performance auditing, and applied grounded theory to produce such a methodology and model. The result was evaluated against INTOSAI standards ISSAI 3000 and ISSAI 3001, as well as field-tested. The field tests indicate that the proposed methodology and model identified material performance processes and risks, and that the audit program addressed key performance risks. The field tests were restricted to the private sector.
Further research is required to determine the universal applicability of the audit model in additional business sectors, in both the private and public sectors. Further research in this area would promote the understanding and usefulness of the performance audit methodology and model to be used by both auditors and business managers.