How Evaluation and Audit Is Implemented in Educational Organizations? A Systematic Review

© 2015 The Authors. This is an Open Access article distributed under the terms of the Creative Commons Attribution (CC BY), which permits unrestricted use, distribution, and reproduction in any medium, as long as the original authors and source are cited. No permission is required from the authors or the publishers. Article History: Received: 16 Feb 2015 Accepted: 1 June 2015 ePublished: 4 July 2015


Introduction
In the 21 st century, an increase in social demand for higher education and economic need for more educated society have led to an increase in the number of higher education students. 1n addition, a growing climate of increased accountability has placed the issue of quality management on the agendas of higher education institutions in many countries.These forces say that quality assurance processes must be rigorous and transparent, so quality enhancement initiatives must be placed in any quality management program. 2,3ducational institutes publicize their performances to assure students and non-students alike that the missions and quality criteria are met.An effective management system of any higher education organization evaluates the results at the program, departmental and institutional level.Thus, a variety of evaluation models and approaches in the higher education context have been established all over the world, with countries adopting or developing these models for evaluating their performance. 4uring the 1990s, a new concern, academic quality, was added to previous policy issues of access and cost.The first efforts to assess academic quality were in France in the early 1980s and were elaborated upon in the UK in the late 1980s.By that time new forms of evaluation in higher education institutions-often called "academic quality assurance"-had been spread all over the world; almost all of the countries in the European Union as well as many countries in Africa, Asia and South America began experiencing new forms of academic quality setting. 5he definition for quality assurance is: "the system that plans necessary actions to make sure and give adequate confidence that a product or a service will meet the specific requirements for quality." This process involves product/ service evaluation as well as evaluation of the system that supplies the product/service against pre-determined standards. 1,3here should be a system for educational quality assurance that measures all aspects of quality issues to ensure improvement of educational quality and that standards are met at all levels.This system should involve both internal and external quality assurance, and its methods and criteria for quality evaluation should be as definitive as ministerial regulations. 6valuation is an important element of an educational organization's survival.Evaluation can be defined as "a process which is designed and implemented to help stakeholders to assess an object's merit and worth" or "a study which assess the merit, worth and value of processes, systems, outputs and outcomes of an organization carefully, a process which is intended to play a role in future". 7he evaluation and audit of higher education institutions is a priority.This means that there is a need for designing a method to evaluate and identify the evaluator or audit team and develop performance evaluation framework. 8oday, educational evaluation activity is not only used to assess and test students in the classroom, but it has been expanded to include the entire educational system.Evaluation is used on all levels of education, including individuals, classrooms, programs, organizations and fields and at national as well as international levels. 9ducational evaluation is a formal process that uses data collection and analysis to evaluate the quality and value of an educational organization, program, curriculum and process.An educational evaluation's effects, values and results can be best seen when it can provide needed information to individuals which are directly related to, as well as those who might benefit from, the results.Educational evaluations can study and assess educational units utilizing standards, predetermined aims or educational quality.After such assessments, it is possible to address shortages in the educational system and arrange for a more efficient system. 10here are many different models and approaches used to evaluate higher education all over the world, and countries often adopt or develop these models to evaluate educational performance.We therefore reviewed published articles based on three questions: 1) What are the methods of evaluation and audit in educational organizations?; 2) What are the strengths and weaknesses of identified methods?; and 3) What are the important factors in developing and using an evaluation or audit model in educational settings?

Materials and Methods
Between March and May 2014, we used two search engines, Google Scholar and PubMed, and three databases, Scopus, Science Direct and Emerald Insight databases for articles.We also searched the Scientific Information Database and Magiran databases for Iranian articles.The selection criteria were as follows: • Articles that described determined methods, models or frameworks for evaluating in education • Articles in Persian and English languages • Articles which were published between 1993 and 2014 The exclusion criteria were: • Papers that did not mention an educational evaluation or audit model • Papers with insufficient information about methods of evaluation • Papers that explained educational evaluation and audit in general • Papers that were about evaluation of teaching quality • University audit handbooks and evaluation instructions We used a list of key words as follows: evaluation, audit, educational audit, educational evaluation, academic evaluation, academic audit, model, method, education, college, school, department, university, higher education institution and mixed them with "AND" and "OR" as a search strategy.The search strategy which we used was as follows: "Evaluation" OR" Audit" OR" Educational audit" OR "Educational evaluation" OR "Academic evaluation" OR "Academic audit "OR" Audit Model" OR " Evaluation Model" AND "Education" OR "College" OR "School "OR "Department "OR "University" OR" Higher education institution" We selected articles by reading the titles, abstracts and, if necessary, the full text.Articles that were relevant to the research questions and met the inclusion and quality criteria were selected.

Ethical considerations
This research was part of an MSc thesis which was approved by the Ethics Committee of the Tabriz University of Medical Sciences.Ethical aspects were considered in all steps of study and texts belonging to other authors that have been used in any part of this study have been fully referenced.

Results
The study selection process is outlined in Diagram 1.The search identified 1,336 papers, and we excluded 1,280 articles after title and abstract screening because: • Some of them were duplicated • The paper did not relate to educational evaluation • The paper did not not mention a model or method of evaluation in an educational setting From 56 articles that were ready for full text reviewing, 3 old articles were excluded.After adding 4 articles by hand searching and reference by reference, 57 articles were reviewed.After a full text review, 25 articles were excluded because they: • Had insufficient information about methods of evaluation.

•
Explained educational evaluation and audit in general.

•
Were about evaluation of teaching quality.Details of selected articles were appraised and extracted by two reviewers using standardized abstraction forms.Thirty-two papers completely related to the research questions were finally studied.Each of the included studies was categorized based on the following characteristics: country of origin, year of publication, type of evaluation and evaluation level.After that, data from each of the included articles were extracted and entered into a matrix (Table 1).We also identified strengths and weaknesses of these models and the important factors that should be considered in developing or implementing a model of evaluation.
According to literature, we can describe forms of evaluation as below: 1.Typical Evaluation, which measures the quality of a subject in all study programs in which the subject is taught, the quality of the study program itself, the quality of an institution in both educational and administrative aspects and the quality of a specific theme within higher education.2. Accreditation, a procedure in which the quality of an institution or a study program is evaluated by a private or a state-independent actor to certify that it meets specific and pre-determined standards.It may include a self-study and external evaluation by experts.Its main goal is to maintain and improve quality in a higher education institution, study program or course.Accreditation results can lead to achieving an award for the current status or a license to continue operating for a definite period of time 3. Audit, a process that assesses the effectiveness of quality assurance systems within educational organizations.Its focus is on accountability and examining whether or not determined objectives are achieved.The audit is done because units are responsible for the quality of monitoring procedures and implementing improvement activities.Audits also help organizations in the decision-making process. 8,11esult showed that educational evaluation can be done at these levels: university, school/college or institution, department or group, educational program or curriculum, course evaluation and practice placement or training evaluation.For each level there are some researcher developed models or pre-determined models that the authors applied.Results of studies in Iran show that evaluation is most often implemented at the department level.We presented Iranian articles in Table 2. Applying educational evaluation in the Iranian educational system began with the implementation of a pilot Educational Evaluation study in six educational groups in 1996.Results showed that Educational Evaluation in the Iranian culture would lead to improvement.Farzianpour 12 and Bazargan 13 revealed that Educational Evaluation is the best measurement index for evaluating university hospitals.In 2004, 15 basic science and clinical educational groups of Tehran University of Medical Sciences reported their educational evaluation results as desirable. 12

No
Author/ Year/ country

Evaluation level Main Findings
Strengths Weaknesses 14 External quality audit by external agency college evaluation -The external quality audit of private higher education was done by AUQA against four factors:" 1. Governance and management 2. Learning and teaching 3. Administration and support services 4. quality management and continuous improvement " 14 -18 months before the AUQA audit the college commenced its preparation and formulated a self-review.
External audit has had positive result with the use of self-assessment, external peer review, improvements and follow-up.
-friendly and open atmosphere -transparency of the process -professionality of the staff involved -the self-evaluation caused more preparation and increased accountability of key staff The need for more briefings and information sessions with staff

Course review
Two elements of subject review were:" The self-assessment document and The subject review team visit" 15 The self-assessment documents (SAD) are submitted at least six month before the visit.The format and sections were designed by the QAA.
After that the review team started visit and the main events of it included: meetings related to the "quality of education " section of SAD , student & employer consultation, inspections of facilities and resources, teaching observation and the final feedback section. 15gular visits to monitor the quality of courses were made by subject-specialist colleagues from other institutions.

3
Grebennikov L , Shah M / 2008/Australia (University of western Sydney) 16 External audit by external agency

University evaluation
The steps taken by the university to effective preparation of the external quality audit which were prescribed by the Australian Universities Quality Agency involved three site visits.
-First the university conducted a quality self-assessment of all academic and administrative units.The trial audit was held after that.
The University found the trial audit very useful.It complimented the UWS own self-assessment and various reviews undertaken earlier.

Researcher developed model for Internal evaluation
Course evaluation -This course review process is a framework that adopts quality assurance principles to reach the core processes in pharrmacy education and student learning.The process was able to identify areas for improvement within individual courses.
-The process was initiated by the curriculum committee and included 5 parts: 1. Self-reflection by the course director 2. review of the course by a course review team 3.review of team`s findings and recommendations 4. review of the team`s recommendation by course directors 5. Retrospective analysis.the course review process was efficient in which 30 courses were reviewed in a 6-month period 17 this process was satisfying among course directors -an efficient, effective, and economical approach to reviewing courses -Course reviews provide a foundation for individuals to share accountability with respect to improved teaching -a time bonus process -Faculty resistance to change

Table 1. Extraction table of reviewed articles result
Sconce C, Howard J/ 1994/England (Lancashire) 18 A conceptual frame work

Course evaluation
From the Stufflebeam ` s CIPP model the areas of context and input are integral, thus a three-stage model of evaluation emerged whose components are: professional input & analysis, process and product evaluation.The objective is collecting the professionals who are stake-holders in the education of nurses.Their views and beliefs were included in the decision making process and professional judgment of planning, delivering and developing the course. 18 comprehensive and easy to use framework -the professional nature of those involved to make the framework scientific -In order to propose a quality assessment metric the author summarizes the stakeholders' main attributes and elaborated all players of the education system with their role and possible metrics and criteria.19 -Consideration all stakeholders , their roles, responsibilities and interests in the model

Education programs
The project process was: analyzing o other countries experience and seminars with foreign experts having implemented similar projects in other countries.Designing questionnaire for study program ` evaluation (15 experts from different fields and also some Latvian students took active part in designing questionnaire and the methodology) -Evaluation is realized , basing on four main areas: Quality, Resources, Sustainability, Cooperation and Overlapping.Each area contains some criteria The study programs were divided in 28 branches and program evaluation is performed in frames of branches by expert team for each branch. 10e result of the project was useful and effective The management systems of all services and their results are compared with the criteria of the EFQM model.The final outcome was a self-report including: strengths, weaknesses, and improvement plans.20 -involvement of all staff -Providing knowledge of quality-related issues.
-Creating improvement approaches for the whole service.
-Increasing employees' awareness on the importance of quality -Giving an overview of the service processes. -

A conceptual framework
Institutional performance -This framework uses quality and strategy maps and classifies the targets according to the quality map using the strategic planning.The management of organization must consider the strategic plans and design strategic objectives.In the next step developing the internal structures to achieve the objectives is done.
-So the indicators which measure the achievement of objectives must be established by the management.Researcher developed model for self-assessment school -A survey to seven EU countries held, in which the opinions of important quality items and their related importance are surveyed among the relevant stakeholder groups by a questionnaire.
-After defining quality items, this casual model used to form a structured tree for the DEXi , Model to be used in the weighting and assessment tool.
-Finally a self-assessment tool that uses the DEXi structure developed for the use of the schools in their quality improvement.
-This model could handle multiple-criteria decision-making on qualitative data. 23et individual schools to make the adjustments of the weights (it was easy to use) -the opinions of important factors and their related importance are surveyed -could handle multiple-criteria decision-making on qualitative data -costly 13 Mohabuth AQ/ 2013/ USA 24 A researcher developed model for external audit

Work-based Learning and practice placement
The sample of 25 students (5 students from the 5 faculties), 20 mentors and 10 academics were interviewed and the required information necessary for development of the audit tool .the tool has been designed based on the facts gathered.. -there are five core aspectsof the WBL practice placement that makes provision for evidencing processes and some criteria and a checklist were designed for each area.Lead auditor from the university conducted the audit and the outcome of the audit was then classified under three main categories. 24e tool was tried and tested at various local placement settings and it was reliable.
-it led to student satisfaction -difficulty in collecting the organization's commitment -participants felt it was time consuming -The model is built on the evidence gathered from a pharmacy education context, students ` expectations(using focus groups) and concepts from related literature.This model focuses on student education.
-The themes were discussed and the model was expanded.Emergent constructs were identified from the data in the framework of input-process-outcome for each of them criteria were assigned.
-The model can be used for guiding college administrators.General evaluation of formal mentoring program and any academic environment could be guided through applying of this framework. 25onsidering students 'perceptions -used or adapted by other health professions according to the specific needs of their field -requires further investigation -not considering mentor and administrator experiences 15  Rous E et all/ 1994/ UK 26

An audit tool based on audit cycle
Training in public health medicine ( practice placement) The audit described in this paper was based on guidelines set by the Faculty of Public Health Medicine.
-The audit cycle included: 1. setting or reviewing of standards 2. moves on to the observation of practice 3. recommendations for improvement 4. implementation of change As a result of the survey findings, recommendations were made for improving training in public health medicine. 26ed to improvement -use follow-up and feed back 16  Fritz K/ 1997/ UK 27 multidisciplinary audit tool based on QualCube model

Practice placement in nursing education
The framework named QualCube was adopted by the college as a basis to 'set standards, monitor activity and create quality improvements that are consistent with the organization's values'.The three dimensions of the cube, audit characteristics, business elements and audit customers have five subdivisions.Working group explained five key quality statements for setting the overall standard for each area.Feedbacks were solicited from college staff through interviews with those who have used the tool. 27t can increase focus and precision when used in the context of audit.
-standards are nietherspecific enough nor measurable -time bonus

Internal evaluation School
-Through this study the quality of education provided by 12 departments were assessed in 13 fields.
-Each field was evaluated through these steps: 1) Establishment of standards 2) Determining criteria and data collection source for each factor 3) data collection 4)determining the importance of components 5) analayzing the collected data -Data were collected using interview, inspection, checklist, questionnaire Self-assessment based on EFQM and Iran excellence model University -13 universities were selected and the intervention had following steps: 1) discussion with relevant officials for approval and financing 2) A 3-day national workshop was held for managers of 13 universities.
-152 indicators of self-assessment appropriated with Iran health system were designed within the framework of nine EFQM model criteria and 32 sub-criteria and associated guidance point with 1000 points.
3) A 3-day workshop on each of 13 universities -on the second day and third day led of the researchers , trained managers performed self-assessment separately in each of the four groups of 5-7 people and 152 indicators were scored between 0-100.4  Bazargan A et al/2005/ Tehran 13 Internal evaluation model Department The regulation, circular and educational instructions were scrutinized and by faculty guidance, aims and objectives of educational department were identified and best performance requirements were described.Based on these requirements factors and criteria were determined and classified into systematic model od input, process and output.For data collecting different questionnaires were designed and finally collected data were analyzed for judging about department educational quality.

Higher education institution
First an assessment framework based on fuzzy approach and verbal expressions was designed.In this framework total quality management elements were divided to enablers and result.With use of this framework and literate review, the enablers' part of TQM was conceptualized and provided in a questionnaire format.This model was implemented in 8 steps.

Course evaluation
In this course evaluation model internal quality was assessed through nine elements which were determined by literature reviewing and selecting a model for this assessment.For each element questions were designed.in the external quality dimension it was assessed how this course was responsive and meet customer`s need. in this section units of this course were evaluated.

Internal evaluation model Department
In this model after determining aims and objectives of educational group(department) in three levels: education, research and professional service, national and international models were scrutinized and opinions of evaluation committee were considered.After that evaluation factors and criteria were determined and data collection instruments were designed.
12 Yar Mohammadian M, Kalbasi A/ 2004/ Isfahan 39 Internal evaluation model Department This model of evaluation was based on 10 steps of medical universities evaluation and evaluation standards of ministry of health.Data were collected through checklists and questionnaires which designed by faculties and group managers.The information were collected from two sources: 1) information and opinions of university`s experts about best factors and criteria of evaluation (based on CIPP model) and suggested factors.This information was collected by questionnaire which was about evaluation standards and criteria.
2) Other information which needed was about existing conditions of each standard and factor.From result of the previous questionnaire, another questionnaire was designed for collecting data about existing conditions and judgment about it.

Discussion
As the results show, the majority of studies are at the college and school level, but in Iran the department level has the most importance.There are many different aspects explored in scientific literature concerning evaluation in higher education institutions.In the educational organizations, improving the quality of teaching and learning must be considered an important priority.An evidence-based evaluation and audit are needed to achieve this objective.In the audit and evaluation, it is necessary to establish suitable indicators to evaluate specific strategic targets. 41he studies we examined show that multi-stage evaluation is reliable and acceptable in most countries, so it is implemented as the main quality assurance instrument for evaluation of higher education institutions.This procedure begins with an internal self-assessment that an academic program or institute conducts to analyze its own strengths and weaknesses.In the next step, peer reviewers conduct a site visit of the units under evaluation and prepare an external evaluation report.Implementation of the reviewers' recommendations occurs in the follow-up stage. 10ethods based on internal criteria are those that can interpret the scientific and educational authenticity of different educational groups.This is greatly welcomed by the academic community and is widespread throughout universities around the world.This is because this method provides a scientific, appropriate, precise, timely and valid basis regarding the interpretation of the quality of decisionmaking systems. 12overnments in many countries consider higher education policies related to external quality audits, and external agencies have been funded to conduct audits.The approach used to audit educational organizations differs between countries, but it is clear that the target of the audit is to ensure that higher education providers have effective systems for managing and enhancing quality.In addition, the outcomes of audits enable governments to appraise the quality and standard of higher education institutions and identify areas needing improvement. 3cademic audits, which first developed in the UK and then developed in New Zealand, Sweden, Hong Kong and Australia, do not attempt to comprehensively review an institution's resources and activities (unlike accreditation), but focus on those procedures by which the educational organization attempts to assure its academic standards.
Here the focus is on "quality work." The basic processes of academic quality assurance are evaluated through an audit, which reviews how an institution measures itself and sets chosen standards.Educational audits also offer some measure of public accountability. 5t is obvious that evaluating performance and implementing improvements are essential elements of organizational success.Audits and evaluations are seen as tools to help effective decision making and also as sources of information for managers.The purpose of these activities is to determine whether the organization is doing what it says it is doing effectively and suitably achieving objectives. 42eatures that the articles pinpointed as important factors in developing and using an evaluation or audit model in educational settings are: improvement of quality culture, increase in staff accountability, increase staff knowledge/ innovation, consideration of students' opinions, consideration of staff opinions, consideration of experts' opinions, improved connections, being effective using a structured method for data gathering and analyzing, identifying shortages found after the pilot study/feedback and including a follow-up stage.For implementing internal evaluation, it is better to start at the smallest level.The most important objective in course and educational program evaluation is to help in identifying strengths and weaknesses of educational objectives and decision making.Departments are the basic units at which we can start evaluation, but administrative services are not yet considered.College-level evaluation can be seen as regular strategic overview of the entirety of a college's learning and teaching activity and administrative services.It may be a time-consuming process.

Conclusion
In conclusion, higher education organizations can be defined as a complex set of human and material resources that work together to offer the services and meet the needs for education, knowledge, professional training and human formation.
Because of the importance of quality in higher education, educational organizations are responsible for establishing and monitoring quality in their settings.
Efforts to ensure quality are needed because higher education organizations operate in a competitive marketplace.Educational products are characterized by quality and competitiveness, and competition takes place both in price and in the quality of service. 43t is suggested that the procedures for measuring quality, quality indicators and criteria should be student-centered and express the organization's mission and values.
The criteria for quality evaluation in higher education institutions should consider the following requirements: (1) Appraise the intended curriculum; (2) Review teaching/learning processes; (3) Clarify the expected outcomes for students; (4) Examine the degree to which outcomes are achieved; (5) Evaluate the appropriateness of support provided for students; (6) Assess the research implemented by the academic unit; (7) Evaluate the collaboration of the unit or program based on the other aspects of the institutional values. 1 An organization can adopt different models of evaluation that exist and use the benefits of them or develop a model considering the features listed above.

Limitations
This systematic review had some limitations.In this study, some databases were not considered and universities audit handbooks were excluded.In addition, only studies in the English language were considered, which may have led to internal evaluation in Iranian university of medical science were implemented and evaluation standards were applied to determine criteria and evaluate the department.
evaluation was based on 10 steps of medical universities evaluation and evaluation standards of ministry of health.10FathiK, Shafiyi N/2007/Tehran37

Table 1 .
4xtraction table of reviewed articles result The development of model for the internal quality assurance system in the college ( this was based on analyzing documents, concepts and theories.Interviewing with experts to gain information.Synthesizing the results, expert-based seminar for developing the model and indicators and its guidelines) 2.The tryout for the implementation of this model 3. the assessment and improvement of the evaluation model -The process of this developed model was consisted of three steps: 1. preparation for evaluation 2. Evaluation which consists of collecting, analyzing, interpreting and summarizing the data and results.3.Overall report of plans and projects4

Table 1 .
Extraction table of reviewed articles result

Table 1 .
Extraction table of reviewed articles result

Table 1 .
28traction table of reviewed articles resultThe learning opportunities available, concentrating on the physical environment, the learning climate/environment, and the students' Perceptions of the learning environment were examined within the practice setting, across hospital and community sectors.28

Table 1 .
Extraction table of reviewed articles result

Table 2 .
extraction table of evaluation models in Iranian Universities Akhlaghi F et al/ 2011/