Anesthesiology residents’ perception towards educational environment using ATEEM in a medical school in Thailand

Introduction Educational environment is an important factor in effective teaching and learning. Positive educational environments result in better academic achievements. Administrators are responsible for education at the department and faculty levels, and attention is given to the educational atmosphere by regularly measuring and evaluating various related factors including the subjective level of awareness and perception of residents. A teaching and learning system should include learning and environment assessment tools to allow residents to take good care of patients. Many educational environment assessment tools have been created to assess the quality of educational environments. Examples include the Dundee Ready Education Environment Measure (DREEM) with 50 items,1,2 the Postgraduate Hospital Education Environment Measure (PHEEM) with 40 items3-7 and the Anesthetic Trainee Theatre Educational Environment Measure (ATEEM) with 40 items.1,8,9 Anesthesiology is a fundamental branch of medicine. There are specific tools for different disciplines that are used only for each study group. These have a variance in the measurement of psychological quality (psychometric) but are considered the best basic tools for easy measurement. In 2004 Holt and Roff1 developed the ATEEM, a standardized tool used to assess the perception of the educational atmosphere among anesthesiology residents. Medical schools in many countries have endorsed its use in the assessment of educational environments. At present, ATEEM has not yet been translated into Thai. This research is therefore a good starting point for systematically enhancing the strengths and improving the weaknesses of the educational atmosphere in the Department of Anesthesiology using ATEEM (Thai version). The researcher has chosen this tool to measure the level of perception of anesthesiology residents in their Abstract

course of study. It is the first time in the Department of Anesthesiology where the perception of the educational environment in all dimensions has been assessed using ATEEM as a guideline to adjust the teaching and learning environment and assist the authorities to improve the quality of education.
Validation of a Thai version of ATEEM in a local anesthesiology department will allow the assessment of the past and present educational environment that our anesthesiology residents have been exposed to and provide a basis for intervention and plans for improvement. To begin, an accurate assessment of the reliability and validity practicality of the Thai version of ATEEM is needed to measure the quality of the educational environment in the development of anesthesiology residency programs.

Study design
This study is a descriptive cross-sectional study using a questionnaire based approach

Materials
The ATEEM is a questionnaire used to measure the perception of the educational environment among anesthesiology residents. It consists of forty items with each item having a possible maximum score of 4 points, divided into five domains as follows: • Perceptions of Autonomy: items V, X, XV, XX, XXV, XXIX, XXXIII, and XXXVI. There are 8 items with a maximum score of 32 (8 x 4). • Perceptions of Atmosphere: items IX, XIV, XIX, XXVIII, XXXII, XXXV, XXXVIII, XL, IV and XXIV are negative items. There are 10 items with a maximum score of 40 (10 x 4). • Perceptions of Workload/Supervision/Support: items III, VIII, XIII, XVIII, XXIII, XXVII, and XXXI. There are 7 items with a maximum score of 28 (7 x 4). • Perception of teachers and teaching: items II, VII, XII, XVII, and XXII. There are 5 items with a maximum score of 20 (5 x 4). • Perception of Learning opportunities and orientation to learning: items I, VI, XI, XVI, XXI, XXVI, XXX, XXXIV, XXXVII, and XXXIX. There are 10 items with a maximum score of 40 (10 x 4). Respondents were asked to rate the items on a 5-point Likert scale of 0 to 4: 4 for Strongly Agree (SA), 3 for Agree (A), 2 for Unsure (U), 1 for Disagree (D), 0 for Strongly Disagree (SD).
The ATEEM instrument has a maximum possible score of 160, which would indicate an ideal educational environment. A score of 0 is the minimum score and would be a very disturbing result that would need to be resolved .

Copyright request procedure
The researcher contacted the owner of the ATEEM questionnaire to obtain permission to translate and use the original instrument according to the cross-cultural attachment.

Translation
The process followed three steps. First, it was translated into Thai and back to English (back translation method) to get items that maintained their meaning with linguists from the Center for Translation and Language Services of the Asian Language and Culture Institute of Mahidol University. This board has expertise in translation and specifically has experience in translating tools and surveys used in medical research. Secondly, the Thai version of ATEEM was piloted with 8 second-year residents. The quality was assessed for reliability using Cronbach's alpha and split-half reliability. In addition, content validity was assessed through allowing three experts to determine the item-objective congruence (IOC) index and item content validity index (I-CVI). The IOC and I-CVI of the questionnaire was greater than 0.6 for all questions.
Finally, the researchers proofread and reviewed the different aspects of ATEEM (Thai version) again after piloting.

Data collection procedure
After approval from the Committee on Human Rights Related to Research Involving Human Subjects, the researcher collected data to measure the perceptions of anesthesiology residents around the educational environment through responding to the ATEEM questionnaire (Thai version), sent through the mail to the address of the residents with attached postage-paid envelopes from the Department of Anesthesiology to facilitate return.

Population
Data were collected from the population of all anesthesiology residents who graduated between 2006-2018 in Ramathibodi hospital, Mahidol University, Bangkok, Thailand, a total of 170 graduates. The response rate was 55.9% (95 of 170).

Statistical data analysis
Descriptive analysis presents group data as numbers and percentages. Normally-distributed continuous variables are reported as mean ± standard deviation, minimum values, and maximum values.
Inference statistical analysis was done with Student's t test to compare residents' perceived scores between sexes and age groups. P values of less than 0.05 were considered statistically significant.
Cronbach's alpha and split-half reliability were used to evaluate the internal consistency of the instrument as a whole and for each domain. The IOC index and I-CVI were calculated to evaluate content validity. The instrument was tested for validity and reliability, and data was analyzed using the Statistical Package for Social Sciences (SPSS) version 18.0 (SPSS Inc., Chicago, IL., USA).

Determining the confidence of the tool (reliability)
Internal consistency was analyzed using Cronbach's alpha. A coefficient greater than 0.7 indicates that a tool is sufficiently reliable. The questionnaire using this method must be Likert scale with scoring scale. 10 Many methodologists recommend a minimum alpha coefficient between 0.65 and 0.8 (or higher in many cases); those which are less than 0.5 are usually unacceptable.
Split-half reliability is a statistical method used to measure the consistency of scores of the perception, applicable to questionnaires, to divide the test into even and odd questions and compare the results. 10 Split-half estimates has a three-step procedure: 1. Arbitrarily divide the scale into two halves and create total scores for two halves, 2. Correlate the two total scales, and 3. Adjust the correlation upwards using the Spearman Brown formula. Split-half reliability is quantified by the Spearman-Brown formula 10 R sb = 2r sh / (1 + r sh ) , r = reliability of observed scale Instead value, R sb = 2 * 0.906 / (1 + 0.906) = 0.95

Content validity
This is done using a consistency index called IOC that has the following criteria: • A score of 1 when making sure that the question is consistent with the purpose • A score of 0 when unsure whether the question is consistent with the purpose or not • A score of -1 when making sure that the question does not correspond to the purpose. The formula governing this is IOC = Σ R / N Σ R is the consistency index between questions and purposes.
N is the number of experts (three in this case). The criteria for determining the consistency index level between questions and objectives should be between 0.00 and 1.00. A minimum IOC value of 0.05 validates the tested question, while values less than 0.5 rejects the tested question.

Item content validity index
There are 4 levels of consistency assessments in item content validity: 1: not relevant, 2: somewhat relevant, 3: quite relevant and 4: very consistent. To calculate the content accuracy index only items that have been evaluated at 3 or 4 will be considered valid. The confidence index (I-CVI) can be calculated by dividing the number of experts who assess the level of conformity (assessment level 3 or 4) divided by the total number of experts. The formula is I-CVI = Nc / N, where Nc represents the number of experts that evaluate the proposal at the corresponding level and N represents the total number of experts. 11 In the event of at least three experts, a question that should be considered consistent should be assessed as consistent in two out of three opinions. If the value is 2/3 = 0.67, it is acceptable. These criteria should be the lowest for individual content and levels are considered acceptable when the value is 0.8 or higher and excellent when the value is 1.0.
S-CVI/Ave is calculated by taking the sum of the I-CVI divided by the total number of items. 11 S-CVI/Ave ≥ 0.9 indicates excellent content validity. 11 In the current case, S-CVI/Ave = 39.33 / 40 = 0.98.

Results
This study reports an assessment of the educational environment using the ATEEM inventory. The English translation of the ATEEM inventory was revised and refined. It was produced in a combined bilingual format (Thai version). One hundred and seventy questionnaires were distributed among anesthesiology residents at different graduation years. Ninety-five returned complete questionnaires for a response rate of 55.9%. The overall mean ATEEM score was 122.9/160 (76.8%), shown in Figure 1.
The maximum and minimum sum scores were 89 and 150. Respectively, indicating more positive than negative perceptions. Baseline characteristics are reported as numbers (percentage) of respondent with respect to the study population. In all, 20 (20.1%) respondents were male and 75 (78.9%) were female. Respondent questionnaires included graduation year and population. The age range of the population participating in the two phases of the research did not differ. The mean age was 29.4 (SD 2.26), with a minimum of 25 years and a maximum of 39 years, shown in Table 1.
Note: Mean age: 29.4 (SD 2.26), minimum 25 and maximum 39. Figure 2 displays the average total scores of all 95 participants in their different graduation year. Table 2 shows results of "more positive than negative" perceptions. All scores of anesthesiology residents' perception towards the educational environment in the operating room were either excellent (46; 48.4%) or more positive than negative (49; 51.6%), and none indicated many problems or very poor.
With regards to the assessment of each subdomain, all subdomains scored close to or above 70%, as shown in Table 3, with the overall total reaching almost 77%.
All ATEEM domains had Cronbach's alphas greater than 0.7, suggesting that questions in each domain correlates well and that internal consistency is acceptable ( Table 4.) The overall Cronbach's alpha for all domains is 0.931, and the split-half reliability was 0.95.
The mean score and standard deviation, Cronbach's alpha if item deleted and IOC for each individual item are shown in Table 5. All items had range values of IOC and I-CVI between 0.67 -1.00, and S-CVI/Ave had value 0.98 showing discriminative power was in accordance with the criteria.
An independent t test revealed no significant difference in terms of gender (Table 7), although perceptions of atmosphere trended towards significance. Both females and males rated perceptions of atmosphere as the lowest among the domains.

Discussion
The results from this study support the use of the Thai version of ATEEM as a reliable and simple questionnaire for assessing the clinical educational environment in the operating room for anesthesiologists as well as giving a useful indication of the priorities for curriculum development. Anesthesiology residents displayed more positive than negative perceptions about their learning environment. With regard to the assessment of each subdomain, the perceptions of learning opportunities and orientation to learning were highest, with a mean percentage of 81.0% (mean score 32.4/40), while perception of the atmosphere was the lowest (mean score 27.7/40, 69.2%). Sex was not a statistically significant variable, although it trended towards significance in the domain of perception of atmosphere, which received the lowest score for both sexes.
These results show internal consistency of the instrument demonstrated by a Cronbach's alpha of 0.931. This finding is similar to that seen in previous studies. [3][4][5][6]8,9 The split-half reliability was 0.95. Taken together, these indicate that the Thai version of ATEEM has acceptable reliability. The domains of this tool show suitable psychometric properties, with Cronbach's alphas ranging from 0.71 to 0.81 for each domain. 2,4,12 The instrument  underwent evaluation and examination for content validity from three experts, who found that the index of IOC and the I-CVI had values between 0.67-1.00, with S-CVI/Ave having a value of 0.98, indicating acceptable validity. The calculated mean total score for the ATEEM data was 13.1 (corresponding to a raw score of 122.9), which falls into the category of "Excellent" and "more positive than negative" according to the interpretation of the questionnaire's domains. 8 These are listed in the subsequent paragraphs. Perceptions of Autonomy is the first domain, and showed the highest ATEEM scores at 83.3%. This means the anesthesia education model is providing high levels of responsibility for patients to the anesthesiology residents. Due to the high sensitivity in the types of activities performed in the operating room, the learning objectives of each session are well defined and the teaching is directed to the suit the learning needs of the residents. The highest item score for this domain was Item 20, "I have an appropriate level of clinical responsibility, " with a mean value of 3.22 ± 0.42 out of 4.0.
Perceptions of Atmosphere is the second domain and showed the lowest ATEEM scores at 69.2% for both males and females. Possible reasons for this domain scoring lowest may include co-operation with the team, improper scheduling, stress, lack of a motivational environment, and lack of opportunity to develop interpersonal skills. Being able to work well with other residents and nurses fosters a good learning climate. Atmosphere factors also affect the efficiency of the teaching and learning process of the residents. In addition, this domain affects the effectiveness of teaching and learning in the operating room. Although the operating room environment is reasonably homogeneous in various countries, this study was conducted in Thailand, which is a high relationship according to cultural dimensions. Being supportive of each other is an important factor in fostering learning by creating a positive learning environment. 9 The highest item score for this domain was Item 32, "I have good collaboration with theatre staff, " with a mean value of 3.20 ± 0.48 out of 4.0.
The perceptions of workload/supervision/support is the third domain with a score of 76.7%. In Thailand, the Doctor of Medicine program includes teaching in many forms, including lecture teaching, small group teaching or self-learning, and patient bedside teaching. Bedside teaching is still a popular platform that is designed for all anesthesiology residents to experience. It is also an opportunity for clinical teachers to be taught and to think about their knowledge, attitude and skills in caring for patients. In addition, it is an activity that allows for exchange of information and care between residents and patients in real life. Clinical teachers must be trained to provide constructive feedback so that the anesthesiology residents are encouraged to take responsibility for themselves. The highest item score for this domain was Item 27, "I receive the necessary clinical supervision, " with a mean value of 3.27 ± 0.52. The inflow of patients in public hospitals is more than in private hospitals, thereby a higher workload is perceived by the trainees. 8 The residents also gave a high score to Item 31, "My workload in this job is fine, " with a mean value of 3.09 ± 0.51 out of 4.0 in both cases.
Perception of Teachers and Teaching is the fourth domain with ATEEM scores of 73.4%. Attitudes of teaching include giving attention, effort, and determination to teach, whether as a medical professor or a fellow resident. These attitudes were expressed in the form of spoken and body language of the residents. The highest item score for this domain was Item 2, "The teaching helps to develop my confidence, " with a mean value of 3.52 ± 0.54 out of  There is an open mind to listening to comments and an opportunity to ask answers with the residents during bedside teaching to the extent that suitable for learning. In addition, acting as a role model in taking care of patients appropriately (Good role model), reveals that two-way communication is more useful than one-way communication. Supervision is mostly oneto-one with a strong relationship between trainees and trainers. 13 An equally high item score for this domain was Item 7, "The teacher helps to develop my competence, " with a mean value of 3.52 ± 0.54. Finally, the perception of learning opportunities and orientation to learning was the fifth domain with an ATEEM score of 81.0%. Since residents have different personalities and background ideas, they benefit from different approaches. Anesthesiology residents who are diligent in their pursuit of capital participated in bedside teaching with enthusiasm and often reap more benefits than residents who are idle or do not have good study habits. Learning is not all that matters but generating enthusiasm helps the undeveloped residents or those who may need to study more. The anesthesiology residents' perception of the professionalism of medical professors was positive. Their medical professors' awareness reflects positive aspects that help them become confident and ready for a future career. The highest item score for this domain was Item 30, "I have the opportunity to acquire the appropriate practical procedures for my level of training (e.g., fibreoptic intubation/subtenons nerve block), " with a mean value of 3.21 ± 0.65 and also item 21, "There are good opportunities for trainees who fail to complete their training satisfactorily, " with a mean value of 2.89 ± 0.69, both out of 4.0.
Results of this study and other similar studies in the United Kingdom, Australia, New Zealand, Pakistan and Thailand 1,8,9,12,13 found that the ATEEM instrument was culture-independent. Realizing that behavior does not refer solely to individual personality, but the perception of the individual's education environment as well is key to focusing on the education environment. These results did not show a statistically significant difference between males and females for overall ATEEM scores, similar to findings in previous studies. 3,8,13 Limitations of the study The number and size of medical schools can influence the educational environment. This instrument was first used in Thailand to assess the multi-dimensional environment of the operating room for an anesthesiology resident and conducted in a medical school in Thailand. The results of the study may not be conclusive in general for the anesthesiology department in other medical schools. Therefore, a combined study in multiple medical schools is recommended in order to obtain educational results that are beneficial to the educational program across medical schools. Similar to findings in previous studies, 4 these results were based on a relatively small sample size and in some cases there were not enough participants to do cross-group comparisons.
The fact that fewer than 60% of the questionnaires were completed returned may indicate that residents who graduated were not enthusiastic enough to participate in such a study. Nonresponse bias may affect the results. 2 Residents who had already graduated may not have considered that the results of such studies would lead to any changes in their education. Therefore, more should be done to assess anesthesiology residents' perception of the educational environment during their educational tenure, possibly comparing in a year-to-year approach during their studies, including residents from all years of training representing different stages in the educational process. 2

Conclusion
Anesthesiology residents' perception of the educational environment in the anesthesiology residency curriculum of Ramathibodi hospital is on the positive side, as assessed by ATEEM. The Thai version of the ATEEM instrument was evaluated to have acceptable reliability and validity. This indicates that the Thai ATEEM instrument is a useful tool to access the perception of the educational environment in the operating room for curriculum development and improvement in medical schools in Thailand.

Ethical approval
Approved by the Committee on Human Rights Related to Research Involving Human Subjects, Faculty of Medicine, Ramathibodi Hospital, Mahidol University. ID. 12-60-48