Comparing the efficacy of problem-based learning vs. lectures on the academic achievement and educational motivation of nursing students: A 3-year quasi-experimental study

1Department of Medical-Surgical Nursing, School of Nursing and Midwifery, Mashhad University of Medical Sciences, Mashhad, Iran 2Nursing and Midwifery Care Research Center, Mashhad University of Medical Sciences, Mashhad, Iran 3Department of Nursing, School of Nursing and Midwifery, North Khorasan University of Medical Sciences, Bojnurd, Iran 4Department of Internal Medicine, Faculty of Medicine, North Khorasan University of Medical Sciences, Bojnurd, Iran

as the primary teaching method in the Nursing Diploma Curriculum since 1999 by the United Kingdom Ministry of Health and has been included in the curriculum of nurses, midwives, and health care providers ever since. 3 PBL was adopted in response to student dissatisfaction with their clinical performance, which they attributed to an emphasis on memorizing biomedical science in traditional education. Another chief reason for dissatisfaction with clinical performance was that students were not trained for clinical problem-solving and life-long learning. 4 PBL is the foundation of constructivist theory and provides many opportunities to transform the classroom into an active learning environment. 5 This incorporates PBL as part of the curriculum, intertwined with the main body of the curriculum and not easy to separate. The connection created by PBL among different subjects helps learners with holistic, comprehensive, and systematic thinking. Such thinking is necessary to live in the 21st century and solve complex and interdisciplinary problems. Through PBL, students find solutions to real-world problems by examining them or the subject, gathering information, drawing conclusions based on findings, and reporting results. 6 In this method, instead of using knowledge as a dynamic tool to solve problems, knowledge itself is considered the end product of education. In fact, instead of the conventional method of one-sided learning and practice of the tutor, the student is encouraged to construct knowledge and understand it. In traditional learning methods, teaching is done using external motivators, and it is usually the instructor who directs the learner to learn and guides his or her learning. In PBL, the motivation for learning is internal, and the learner actively directs his/ her own learning with the instructor's help. Schlett et al. have recommended PBL as an educational method for acquiring knowledge to improve education in medical sciences. 7 Rideout et al reported in a study in Canada that undergraduate nursing students were more satisfied with PBL teaching than with traditional lecturing, especially for communication skills and independent learning subjects. 8 Hwang and Kim compared PBL and lecturing methods, and found that knowledge, attitude, learning, and learning motivation increased in the PBL group. 9 According to Günüşen et al, PBL is more effective than lecturing in improving different skills, including problem-solving skills. 10 Luo et al conducted a study to examine nursing students' knowledge and attitudes about PBL with 1200 nursing students from eight different faculties in China. They found that 97.6% were interested in PBL, but 66.7% of them believed that this method would be difficult for students who are not good at solving problems. 11 The current study aimed to compare the effectiveness of two methods (lecturing and PBL) on academic achievement and educational motivation in nursing students at North Khorasan University of Medical Sciences (NKUMS) from 2017-2020. However, what is dissimilar between the current study and other studies is the use of the Solomon four-group scheme. The purpose of using this design was to eliminate the effect of the pre-test in sensitizing learners and to prevent damage to the external validity of the research (i.e., generalization of experimental findings to external conditions) using the Solomon four-group design.

Study design
The current quasi-experimental study was conducted over three years in four academic semesters. Due to a planned curriculum change in the third year, the "Respiratory System Diseases and Disorders" theoretical course was presented in two consecutive semesters to compare the effects of lectures and PBL on academic achievement and educational motivation.
This quasi-experimental study used the Solomon four-group design to remove the sensitizing effect of the pre-test and avoid compromising the external validity of the research (i.e., the possibility of generalizing the experimental results to other conditions). 12 The Solomon four-group test was implemented because the pre-test may have had random effects on the experimental and control groups.
After obtaining informed consent, participants were randomly assigned to experimental and control groups. Two classes (PBL group) formed the experimental group, and two classes (lecture group) formed the control group (Table 1) based on their academic semester.
A faculty member at the Department of Internal Medicine and Surgery instructed the four groups over three academic years (four semesters). The groups were matched in terms of the instructor and topics. All the groups received ten two-hour sessions of instruction. The groups were first briefed on study objectives and methods of instruction before consenting to participate.
An academic achievement test and the Inventory of School Motivation (ISM) questionnaire were administered to control group 1 (C1) and experimental group 1 (E1). The items in the pre-and post-test versions of the academic achievement test had similar content but slightly different formats. At the end of the sessions (four months later), post-tests were administered in all the groups. In the control groups (C1 and C2), lectures were given after delineating the objectives and explaining the material orally using examples, questions and answers, and PowerPoint presentations. For the experimental groups (E1 and E2), PBL was implemented based on Achicke's method. 13

Setting and samples
The study participants were sampled from all Baccalaureate undergraduate nursing students at NKUMS who took the "Respiratory System Diseases and Disorders" theoretical course over the 2017-2018, 2018-2019, or 2019-2020 academic years. They were from four classes in four different semesters. Of the 142 students who enrolled in this study, 34 dropped out; a total of 108 students finished the course and participated in the study through the end and completed the questionnaires. Although PBL is commonly used in small classrooms with small groups, its effectiveness in large classrooms for nursing education has been demonstrated. 14,15

Inclusion and Exclusion Criteria
The inclusion criteria consisted of taking the "Respiratory System Diseases and Disorders" theoretical course for the first time and not working in nursing-related occupations. Students who were absent from more than two sessions or did not wish to complete the questionnaire were excluded from the study.

Measurements
The problem-solving approach in education was presented in the first half of the 20 th century and is rooted in progressivism. The development of PBL from an educational approach with principles and processes is linked to an educational philosophy with several theoretical frameworks that include adult learning (andragogy), experiential learning, and constructivism. The philosophical foundations of PBL, along with these theories, include student-centeredness, the learning environment, student empowerment in the learning process, the development of lifelong learning skills, and the encouragement of independent, active, and self-centered learning. Achike and Nain 13 introduced and implemented PBL and promoted its use. Proposed steps include: 1. Breaking the ice: This step began in the first session when the instructor and students introduced themselves. The instructor then explained the teaching method and objectives, the steps of PBL, and the educational goals set and obtained the students' consent for beginning the program. 2. Group formation and representative selection for each class: The students were randomly divided into five-to six-member problem-solving groups. One student was selected as a representative for each group to record the discussions and coordination purposes. 3. Using triggers: Triggers are methods for concentrating on problems in PBL. They must be complex and expansive and provide learners with health-related social, personal, and care information. In the present study, the content of the triggers was designed based on the curriculum topics, the instructor's experience, and the review of sources. At the beginning of each session, the students received the content of the triggers in the form of scenarios, diagnostic test results (e.g., chest X-ray), videos, slides, and parts of inpatient record documents. 4. Prioritizing the learning problems: The groups explored the problem(s) discussed by the triggers in this step. The group members re-examined the problem from their perspective using their words.
A list of what the group members had discussed as a solution or answer was recorded. Based on that list, each group implemented the solutions in the presence of other groups in the classroom through role-play or concept maps. 5. Feedback by the instructor, facilitator, or resource person: As a facilitator, the instructor supported the students, reduced their stress, guided their presentations, stopped irrelevant topics, created collaboration among them, had them share their experiences, asked questions, and created challenges. The tool contains 49 phrases in school motivation, and participants are asked to select their agreement on each question using a Likert-type scale (from 5 = strongly agree to 1 = disagree strongly). 16 In previous studies assessing the validity of the school motivation test using factor analysis method, its results have been shown to be satisfactory. 17 The reliability of this tool in Iran has been measured by two methods of retesting and calculating Cronbach's alpha. The total reliability coefficient of the test was r = 0.948 16 and in the present study, the Cronbach's alpha coefficient was r = 0.93. For social science research, a Cronbach's alpha above 0.7 is considered acceptable.
Academic achievement: The academic achievement criterion in this study was the students' score on an academic test, which consisted of 20 four-choice questions with a total of 20 possible points. The questions were examined for content validity. First, the test questions were adjusted according to the topics selected by the Ministry of Health and Medical Education and in line with the educational goals. They were then provided to three professors in the School of Nursing (Internal Medicine Department) for comments. After corrections, the reliability of the questions was determined using the Koder-Richardson method (83%).

Statistical analysis
Data were analyzed using SPSS 11.5 for Windows (SPSS Inc., Chicago, IL, USA). Descriptive statistics are used to illustrate demographic information. Additionally, results are reported as mean ± standard deviation and number (percentage). The t test, ANOVA, and the Scheffe post hoc tests were used to compare variables between groups. A P value of less than 0.05 was considered statistically significant.

Results
Of the 142 students who took the "Respiratory System Diseases and Disorders" theoretical course over the three years of the study, 108 students participated until the end and completed the questionnaires. The distribution of quantitative data was normal for all the groups based on the Kolmogorov-Smirnov and Shapiro-Wilk tests (P > 0.05). A total of 100 women (70%) and 42 men (30%) with a mean age of 22.43 ± 8.2 years participated in this study ( Figure 1). E1 and E2 groups were instructed using PBL and C1 and C2 groups through lectures. A pre-test was given to the E1 and C1 groups, and a post-test was given to all groups. Table 2 shows the means of motivation in academic achievement and the academic achievement final score in all groups.
The effect of the pre-test was examined. Then, posttest values were compared between the E1 and E2 groups (the PBL groups) to examine the combined effect of the pre-test and the intervention (PBL). If this comparison showed no significant differences, familiarity with the pretest was thus not associated with negative impacts. If the comparison showed significant differences, the pretest either directly affected post-test scores or the PBL intervention. An independent t test was used to compare post-test values of academic achievement and educational motivation in the E1 and E2 groups.
Based on the independent t test, the pre-test had the same random effect on all the participants (P > 0.05). Therefore, the two control groups were taken as one general control group (n = 52), and the two experimental groups were taken as one general experimental group (n = 56). A t test showed a significant difference (P < 0.05) in the mean scores of academic achievement and ISM between

Group 4 or C2 (n=23)
The first half of 2019-20 academic years.   the experimental (n = 56) and control (n = 52) groups. The mean post-test scores of academic achievement and ISM were compared among groups using ANOVA.
According to the ANOVA test presented in Table 3, the F-statistic was 37.3 for the ISM and 69.0 for the academic achievement scores, which was statistically significant (P < 0.05). The four groups thus differed significantly in terms of the mean scores of both measures in the pre-tests. At least one group differed from the other groups in terms of the two measures. To determine differences between the two groups, the Scheffe post hoc test was used.
As shown in Table 4, there was no statistically significant difference between the mean post-test scores of academic achievement in the E1 (17.5 ± 1.4) and E2 (17.4 ± 1.7) groups who received PBL as their method of teaching (P = 0.998). Likewise, there was no significant difference between the mean test scores of educational motivation in the E1 (151.44 ± 14.8) and E2 (153.7 ± 14.42) groups (P = 0.998). There were no statistically significant differences between the mean post-test scores of academic achievement in the C1 (14.82 ± 2.9) and C2 (15.17 ± 3.5) groups who received training through lectures (P = 0.998) or between the mean posttest scores of educational motivation in the C1 (117.65 ± 15.78) and C2 (114.56 ± 16) groups (P = 0.992).
The mean post-test scores of educational motivation in the E1 group (151.44 ± 14.8) were significantly higher than those in both the C1 (117.65 ± 15.78) (P = 0.001) and C2 (114.56 ± 16) (P = 0.001) groups. In addition, the E2 group (153.7 ± 14.42) showed significantly higher mean post-test scores of educational motivation than the C1 (117.65 ± 15.78) (P = 0.001) and C2 (114.56 ± 16) (P = 0.001) groups. The mean post-test scores of academic achievement and educational motivations were higher in both the E1 and E2 groups (trained using PBL) compared with the C1 and C2 groups (trained using lectures).

Discussion
The current study was conducted to compare the effect of PBL compared with traditional lectures on academic achievement and educational motivation scores in undergraduate nursing students. Our results indicated that both scores were significantly higher in the groups instructed using PBL than in those instructed using the traditional method of lectures. Students appear to learn better given their active involvement in the PBL teaching method. As a result, they scored higher on an academic achievement and on an ISM. This may in part be due to students participating in the lecturing method sometimes forgetting the material due to relying on their memory, thus they may not remember all the material during a test. This result provides evidence that the implementation of PBL has a dramatic effect compared to the lecture method in increasing both the learning and understanding of students. In other words, in the PBL method, more serious participation and increasing the level of involvement of learners provide a better ground for understanding relationships, which leads to more sustainable learning. Therefore, students can have a deeper understanding of educational materials by using their power of reasoning and judgment in the topics discussed in class. Hence, it seems that all the mentioned factors have been involved in strengthening the effectiveness of this method in comparison with the lecturing method. 1 Numerous studies have been conducted in this field, corroborating the results of the current study. [18][19][20] Looking at the studies around the variables affecting the success of PBL, sequential relationships and interaction effects of problem-solving steps, as well as the existence of features such as group participation, self-study, and knowledge building in the PBL process, contribute to its effectiveness. However, to obtain more accurate analyses of performance of this useful and likely superior teaching method, qualitative research is needed to reveal its hidden constructions. 21 In the current study, triggers were presented to students in the form of scenarios, diagnostic results (CXR), video and slides, and excerpts from the records of hospitalized patients by the facilitator (instructor), and triggers of the students were more in the form of concept maps and role-playing. Triggers are used to stimulate and recall situations. In nursing, triggers should be designed based on holistic care. 22 Robert and Ousey acknowledge that although triggers are brief, they provide a broad view of patients' problems. Evaluation and selection of the triggers are critical. By selecting the correct trigger, the learning speed in PBL will increase. 23 PBL may use more than one trigger and these can be presented at different times. Sometimes PBL may start with a clinical scenario of the patient's condition and end with roleplaying. 22 According to Biley and Smith, perceptions of nursing students trained using the PBL approach fall into three themes: 1. Understanding the true importance of everything; 2. Achieving the required knowledge; and 3. Familiarity with the process of change in teamwork. These three themes in PBL lead to familiarity and applying selfdirected learning (SDL), which is what nursing education seeks to do. 24 Given the expansion of plans and the focus on making the nursing professional, nursing training using PBL can prepare nurses for responsibility and adaptation to professional roles. New teaching methods, especially PBL, help improve self-direction and evidencebased learning. 25 There is a strong link between PBL philosophy and adult learning, experiential learning, and constructivism. According to the latest Barrows classification, PBL is a set of special simulated domains of a patient case used to detect a problem. Cases are ideally presented to illustrate the complex situation of a real situation. The student is at the core of this educational approach, and the instructor is to be a facilitator and avoid providing information directly to the student. In Table 4. Multiple comparisons of mean difference of the four groups on the posttest ISM-score and academic achievement score (Scheffe) (post hoc test) this situation, both the process and previous knowledge context are critical to solving the problem. Based on what has been said, andragogy philosophy and constructivism are related to PBL. Andragogy philosophy is considered as a science and art that helps adult education. SDL is at the heart of andragogy. According to this premise, adults want to learn and have the attitude, motivation, and responsibility for their own learning. Adults are motivated to learn based on life experience, task orientation, and/or problem-based orientation. In fact, andragogy stands as evidence in the implementation of PBL. When students use basic knowledge and previous experience to identify and analyze issues ahead, they are, in fact, using the andragogy philosophy, which would be to create SDLs. On the other hand, the result of PBL is SDL, and this expresses the concurrence of PBL and andragogy. Another philosophy related to PBL is constructivism. According to the constructivist paradigm, learning is an active process of constructing meaning and transmitting and understanding. As a philosophy or theory of knowledge and learning, constructivism describes both what is knowing and how one comes to know. Based on the perspectives of psychology, anthropology, and philosophy, this theory describes knowledge as temporary, evolving, and non-purposeful, with an internal socio-cultural structure focused on the individual and society as a whole.

Dependent Variable (I) Group (J) Group Mean difference (I-J) Standard Error
On the other hand, knowledge is not an accumulated product but an active process, based on which the learner tries to understand the world. 26 By providing PBL training, nurses' cooperation with the team and their colleagues increases; moreover, patients' challenges are raised and solved through group discussions. Active learning style during the discussion, responsiveness, and role-playing also increase. 26 Learning about patient interaction and patient care skills based on PBL improves attitude and increases the average long-term care. 27 Having a clear goal for students and faculty, adequate resources (budget, tutor, equipment, educational support, adequate space, managerial support), inviting the willingness of the faculty and a management system to implement PBL, acceptance of more responsibility in learning by students, presence of the Curriculum Planning Committee for the connection with faculty members and addressing the challenges of PBL implementation, appropriate leadership for the implementation and follow-up of the PBL project, and commitment to the implementation of PBL will assist the PBL implementation with positive consequences. 28 According to the results of this study, it is suggested that the following be considered for more effective application of PBL methods in teaching: 1) Emphasis on applying knowledge and functional skills as training goals in the PBL class; 2) Coordinating the complexity of the problem situation with learners' level of expertise, providing special training and guidance for beginner learners, and gradually reducing these supports as the learners progress; and 3) Simultaneous use of other active teaching methods in the PBL class to enjoy their benefits.
The limitations and challenges of the current study include a restriction in available time for implementing PBL and slight problems in various stages of the implementation. Additionally, students were influenced by the learning environment. In addition, their personal and mental differences might have also affected their response to the questions.

Conclusion
Based on our findings, it can be said that PBL effectively improves the educational system and increases learning motivation in nursing students. While strengthening the student's inner motivation, it also provides a basis for increasing the quality of education and sustainable learning. Therefore, nursing instructors should be more diligent in applying this approach. However, it should be noted that PBL may not be recommended for all circumstances and for all educational subjects. PBL requires experienced, highly knowledgeable, flexible, and interested professors. This educational approach requires sufficient time and involvement of students and professors in challenging issues. Therefore, the requirements of this educational approach are of particular importance during the implementation of training. Further studies are needed to assess PBL in courses with large undergraduate students.