Evaluating the educational program of bachelor of science in healthcare management using CIPP model: A case study at Yazd University of Medical Sciences

Background: It is necessary to evaluate the program to measure and maintain its quality during implementation. This study aimed to evaluate the Bachelor of Healthcare Management educational program at Yazd University of Medical Sciences using the CIPP (Context, Input, Process, Product) evaluation model. Methods: This descriptive-cross-sectional study was conducted at Yazd University of Medical Sciences in 2023, with the participation of 8 faculty members, 95 students, and 327 graduates. Three questionnaires, encompassing four dimensions and nine domains, were used as data collection tools. The data were analyzed using the SPSS software (version 27), employing the ANOVA test and descriptive statistics. Results: In terms of context, input, process, and product, the overall scores for the educational program of the course were 3.03, 2.96, 3.06, and 2.99, respectively. Concerning the overall score across all dimensions, faculty members scored higher than students, and a statistically significant difference was observed between the overall scores of faculty members and students. A significant correlation was found among the dimensions investigated. In general, this training course was evaluated at a relatively favorable level. Conclusion: Given the course’s low score in the ‘appropriateness of curricula with approved topics’ section, it is feasible to increase the program’s overall desirability by implementing some modifications in the course topics and accentuating the students’ skill development.


Introduction
Educational planning, often referred to as curriculum development, is one of the main areas of education and training.It plays a crucial role in the effective development of human resources.
Preventing a significant portion of the national capital from being wasted and maintaining the ability to compete in a future world -where quality is the most crucial factor for any organization's survival -is undeniably essential. 1n higher education, "quality" refers to the alignment of current educational factors and elements with established standards, goals, and expectations. 2herefore, the ultimate goal of health and medical human resources planning should be the development of efficient and high-quality personnel across all health and medical fields.These resources should be utilized proportionately and under proper supervision. 3Given the recent developments in the higher education system, it has encountered numerous challenges.These include a mismatch between the curriculum and societal needs, a lack of coordination between universities and society, an increased rate of unemployed graduates and job seekers, and a lack of need-based targeting.It is essential to focus on maintaining, improving, and promoting quality in the higher education environment, with particular attention to effectiveness and efficiency.A systematic and accurate evaluation is required to understand the quality indicators of the higher education system. 1,4Peter Drucker defines the terms "effectiveness" and "efficiency" as follows: "effectiveness" refers to doing things that are worthy or appropriate, while "efficiency" refers to doing things properly. 5iven that "quality" is a subjective and somewhat nebulous concept, it can be challenging to assess.Therefore, evaluation becomes a crucial tool enabling such judgment. 6valuation involves the use of search and judgment methods to gather information.This information is then used to determine value, establish standards for judging quality, and make decisions.It assesses the usefulness and effectiveness of a program. 7hroughout the history of evaluation, various approaches have evolved.These educational approaches can be categorized into seven general types: goal-based approach, management-based approach, consumerbased approach, expert opinion-based approach, expert disagreement-based approach, naturalistic approach, and participant-based approach. 8Management-based approach or decision-oriented approach assists managers in making decisions on educational activities. 9he CIPP (Context, Input, Process, Product) model, one of the most renowned and practical approaches to managerial educational evaluation, was introduced in the early 1970s by Stufflebeam and his team at the Center for Studies and Evaluation at Ohio University. 10The CIPP evaluation model utilizes four dimensions -context, input, process, and product -as factors for evaluating the quality of educational programs. 11The CIPP model addresses the following issues: 1. Identifying the needs and determining how well the project's goals reflect these needs (Context); 2. Considering the plan and budget to meet these needs, investigating various methods, and explaining why a particular solution was chosen over others (Input); 3. Evaluating the extent of program and project implementation (Process); and 4. Observing both positive and negative outcomes, identifying any unexpected occurrences, and assessing how well the target population's needs have been met (Product). 12he CIPP evaluation model guides policymakers and program stakeholders on the regular collection of evaluation data, thereby facilitating informed decisionmaking. 13Furthermore, this model directs available resources toward more effective programs and offers valuable feedback to managers.It is utilized to prioritize the fundamental needs of an organization. 14Therefore, the CIPP evaluation model accentuates constructive evaluation throughout the evolution of a program, from the inception of the idea, through the planning and design stages, to its implementation and completion.This approach assists decision-makers in making appropriate choices. 15s health and medical care provision becomes increasingly advanced, scientific, and complex, the demand for management specialists escalates at all levels.In this environment, the challenge for all specialists is to concentrate on improving the scientific and professional competence of individuals.The overarching aim of education in this profession is to prepare for the future, recognizing that education plays a pivotal role in this transformation.The dynamics of human societies, rapid technological changes, and the impact of these changes on all healthcare workers have underscored the importance of education.
Healthcare management is an interdisciplinary field within medical education.Unlike most disciplines that focus on a specific set of processes and activities, students in this field are required to engage with a variety of processes and activities.This is due to the unique nature of the field.They are expected to have comprehensive knowledge of both clinical and non-clinical management issues.The field of healthcare management was first established in Iran at the Faculty of Medicine, University of Tehran, as a short-term training course under the Hospital Affairs Department.In 2013, the program was approved to offer a bachelor's degree, making it one of the recognized fields within the medical department.
The curriculum for this field underwent revisions in 2014 and 2015, with modifications to its course units to more effectively address societal needs and expectations. 16t is noteworthy that this project was implemented at Kerman University of Medical Sciences in 2013 and at Tabriz University of Medical Sciences in 2016.The conclusion drawn was that the CIPP model could effectively identify the strengths and weaknesses of this training course, thereby facilitating its improvement. 10,17,18urthermore, in a study titled 'Evaluation of the Integrated Curriculum of Basic Sciences in Shiraz Medical School Using the CIPP Evaluation Model', Amini et al. (2016) demonstrated that 75.77% of the students believed the educational system encouraged active class participation, as per the context evaluation using the modified DREEM questionnaire. 19oreover, in a study entitled 'Evaluating the Quality of Educational Programs in Higher Education Using the CIPP Model: A Qualitative Study', it was shown that the highest level of total desirability was associated with the factor of faculty members (4.84) in the field of input, specifically human resources.Conversely, the lowest level of total desirability was linked to the budget factor (2.50) in the field of input, particularly financial resources. 20he Department of Health Care Management at the Faculty of Health, Shahid Sadoughi University of Medical Sciences and Health Care in Yazd was established in 2003 through the efforts of the honorable head of the Faculty.It commenced its operations in February 2003 by admitting 20 undergraduate students.Furthermore, in 2013, the department expanded its program to include a Master's degree, accepting 6 students, a practice that continues to this day.Each year, a significant number of students, drawn to the discipline, enroll in this field.The primary objective is to cultivate and educate skilled and specialized personnel in the management of organizations and healthcare centers.This aims to enhance the efficiency and effectiveness of healthcare services, thereby improving both the quantity and quality of these services.
This study, conducted in 2023, was designed to evaluate and identify the strengths and weaknesses of the educational system in the Faculty of Public Health at Yazd University of Medical Sciences and Health Services.This evaluation, the first of its kind at Yazd University of Medical Sciences, utilizes the CIPP evaluation model.This model aids decision-makers and policy-makers in determining whether to continue or revise the program by identifying the strengths and weaknesses of the educational system.

Materials and Methods
This descriptive and cross-sectional research was conducted in 2023.The study population included all faculty members (10 participants), students (95 participants), and graduates (327 participants) from the Faculty of Public Health at Yazd University of Medical Sciences.These individuals have been part of the faculty since February 2003 for the undergraduate degree and since 2013 for the Master's level.They were included in the study using the census sampling method, with students, graduates, and faculty members considered separately.Of the 460 questionnaires that were distributed, 430 were completed and returned, yielding a response rate of 93%.
The implementation method involves preparing a specific number of questionnaires.These will be distributed to the faculty members and students of the Faculty of Health at Yazd University of Medical Sciences.Participation in this study is voluntary and requires informed consent.For the graduates, the questionnaires will be sent virtually via an online platform.
The primary variables of this research include: • Contextual dimensions: These encompass the objectives of the educational course and the specialized scientific services required by society.• Input: This includes the curriculum, as well as educational and research facilities and equipment.• Process: This involves student activities, teaching and learning activities, and research activities.• Output: This includes specialized skills and characteristics, as well as the educational performance of the course.This research is conducted using three questionnaires whose validity and reliability have been confirmed in similar studies. 8Each questionnaire is designed for a specific group of participants: faculty members, graduates, and students.The questionnaire is divided into two sections.The first section collects demographic information about the participants.The second section contains specialized questions designed to measure the respondents' opinions in nine areas, using a five-point Likert scale (1 = very low, 5 = very high).
The questionnaires encompass four components of the CIPP evaluation model: • Field dimension: This is for faculty members and graduates.It includes the area of educational course objectives (8 questions) and the area of scientific services needed by society (7 questions).
• Input dimension: This is for faculty members and students.It covers the curriculum area (7 questions) and the area of facilities and equipment (6 questions).• Process dimension: This is for faculty members and students.It includes 9 questions in the area of students' activities, 12 questions in the area of teaching-learning activities, and 8 questions in the area of scientific and research activities.• Product dimension: This is for faculty members and graduates.It includes the area of specialized skills of the graduates (13 questions) and the field of characteristics and educational performance of the course (8 questions).After the questionnaires had been completed, the research proceeded to the next stage: data analysis.The collected data were analyzed using SPSS statistical software version 27.An ANOVA test was utilized to compare and correlate the dimensions of the CIPP model.The data were summarized using appropriate statistical indicators such as mean and standard deviation.Furthermore, in sections requiring comparison, appropriate tests like the t-test and correlation coefficient were employed.

Results
Out of the 430 respondents, a significant majority were women, accounting for 70% (N = 304).The demographic characteristics of the participants in the research are detailed in (Table 1).
Table 2 provides an evaluation of the context dimension.It reveals that the average score achieved in the area of introductory course objectives and the domain of scientific-specialized services required by society suggests a relatively favorable situation.The overall average evaluation of the context dimension of the healthcare management educational program at Yazd University of Medical Sciences was also found to be relatively favorable.In terms of input, both the curriculum and the educational and research facilities as well as equipment were favorably positioned.The overall average score for the input dimension in the educational program under study was similar to that of the context dimension, indicating a generally favorable situation.
The process aspect findings indicated a relatively favorable situation in the students' activity area, which encompasses both teaching and learning activities as well as research activities.
The healthcare management training program demonstrated a relatively favorable situation, as evidenced by the overall average score obtained after the process.
The healthcare management training program demonstrated a relatively favorable situation, as evidenced by the overall average score obtained after the process.The output dimension findings indicated that the students' specialized skills and their characteristics and educational performance were relatively favorable.The overall average approximately in agreement only in these dimensions.
Moreover, it is worth noting that the highest score was attributed to the course's effectiveness in training managers, while the lowest score pertained to the appropriateness of the curriculum with the approved topic, as detailed in Table 3.

Discussion
This study, conducted in 2023, aimed to evaluate and identify the strengths and weaknesses of the health and medical services management educational system at the Faculty of Public Health, Yazd University of Medical Sciences.The results revealed that the context dimension evaluation, specifically in the areas of course goals and scientific-specialized services needed by society, yielded an average score indicative of a relatively favorable situation.As a result, the context dimension of the health and treatment management educational program at Yazd University of Medical Sciences received a generally favorable average evaluation.Okhovati et al 10 demonstrated in their research that the context dimension of the healthcare management educational program at Kerman University of Medical Sciences received a generally favorable average evaluation, a finding that aligns with our study results.
Similarly, Jannati et al 17 found in their research that the context dimension of the healthcare management educational program at Tabriz University of Medical Sciences received a generally favorable average evaluation, a result that is consistent with our study findings.In the input dimension, both the curriculum area and the area of educational and research facilities and equipment were in a relatively favorable state.Consequently, the overall average score of the input dimension in the studied educational program was also relatively favorable, mirroring the situation in the context dimension.Okhovati et al 10 demonstrated in their research that the overall average evaluation of the input dimension of the healthcare management educational program at Kerman University of Medical Sciences was relatively favorable, a finding that aligns with our study results.Conversely, Jannati et al 17 found in their study that the overall average evaluation of the input dimension of the healthcare management educational program at Tabriz University of Medical Sciences was favorable, a result that diverges from our study findings.
The results in the process dimension indicated that all three areas -students' activities, teaching and learning activities, and research activities -were in a relatively favorable state.Consequently, the overall average score of the process dimension of the healthcare management educational program was also relatively favorable.However, Okhovati et al 10 found in their study that the overall average evaluation of the process dimension of the healthcare management educational program at Kerman score for this dimension further underscored the relatively favorable situation of the output dimension within the health and medical service management course's training program.In summary, based on the points obtained, the undergraduate course in health and medical services management received a relatively favorable evaluation across all four areas of the CIPP model evaluation, as shown in Table 2.In general, given the total score across all dimensions, faculty members achieved higher scores than students, a statistically significant difference (P < 0.001).
When examining each dimension individually, only in the areas of scientific-specialized services needed by society and educational goals was there no significant difference between the scores of faculty members and graduates (P > 0.30).Thus, faculty members and graduates were University of Medical Sciences was favorable, a result that diverged from our study findings.Similarly, Jannati et al 17 demonstrated in their research that the overall average evaluation of the process dimension of the healthcare management educational program at Tabriz University of Medical Sciences was favorable, another finding that does not align with our study results.The results in the output dimension indicated that both the field of students' specialized skills and the field of characteristics and educational performance were relatively favorable.Consequently, the overall average score of this dimension reflected a relatively favorable situation for the output dimension in the health and medical service management course.Okhovati et al 10 found in their study that the overall average evaluation of the output dimension of the healthcare management educational program at Kerman University of Medical Sciences was relatively favorable, a result that aligns with our study findings.Similarly, Jannati et al 17 demonstrated in their research that the overall average evaluation of the output dimension of the healthcare management educational program at Tabriz University of Medical Sciences was relatively favorable, another finding that is consistent with our study results.
In general, based on the scores obtained, the bachelor's program in health and treatment management was evaluated at a relatively favorable level across all four evaluation areas of the CIPP model.In terms of the areas of each dimension, there was a difference in scores between faculty members and students/graduates in all dimensions, except for the training course goals and the scientific-specialized services required by society.A significant difference (P > 0.05) was observed in the field of educational and research facilities and equipment between the scores of faculty members and students, with faculty members giving higher evaluations.

Conclusion
The evaluation results of the educational program indicate a relative favorability across all dimensions, underscoring a gap from the ideal situation.In the context dimension, weaknesses were pinpointed in the provision of scientific services to society and the alignment of educational goals with the promotion of growth and innovation in management.A significant deficiency was observed in the curriculum dimension, particularly in the program's relevance to approved topics, its responsiveness to student needs, and its alignment with course objectives.The study underscores the pivotal role of up-to-date curricula in ensuring the efficiency and effectiveness of an educational system.
In the process dimension, there is a relative weakness in student activities, including participation in problem-

Suggestions
The study proposes improvements such as establishing networks for the development of graduates' skills, updating curricula to align with modern educational objectives, providing more effective guidance for student activities, and organizing collaborative sessions to address current issues.Ongoing quality improvement in higher education, especially in areas like healthcare management, necessitates continuous evaluation using models such as CIPP.
We recommend that managers and faculty members establish networks and scientific-research activities focused on skill development for graduates' postgraduation.Additionally, curricula should be updated to reflect the missions, goals, and roles of higher education in the current era.Student activities should be guided appropriately, taking into account their abilities and capacities.To address existing problems, brainstorming sessions or educational workshops involving students, officials, and graduates from this field should be conducted.Ultimately, it is essential to develop a program that aligns with the professional needs and expectations of students, emphasizing the importance of skill learning in student education.

Table 1 .
Demographic characteristics of the participants

Table 2 .
Comparison of score and desirability level of CIPP model

Table 3 .
The highest and lowest evaluation scores of the educational program based on the CIPP model , involvement in extracurricular activities, and use of supplementary scientific resources.The output dimension uncovers a significant shortcoming in the specialized skills of graduates, particularly in the course's effectiveness in generating new knowledge and meeting the scientific and skill requirements of management specialists.The CIPP evaluation model, focusing on context, input, process, and product, underscores the interrelation of these aspects.Although the overall scenario is somewhat favorable, it is crucial to address shortcomings and enhance the generation of knowledge and skills.All stakeholders, including graduates, educators, students, planners, and policymakers, should work together to improve the program's results.Strengths of the program• Effectiveness of the course in training managers • The level of familiarity of professors with statistical and research methods • Coordination of the program with the prerequisites of the students Weaknesses of the program Comprehensiveness and sufficient clarity of the course in providing specialized services Suitability of curriculum with approved headings Development of technical knowledge and experience solving