Effectiveness of problem-solving exercises in radiology education for undergraduates

Introduction

Medical education in India is at crossroads. While India is one of the largest contributors to medical human resource in the world, the quality of medical education is a matter of constant debate. Several reports have pointed out deficiencies such as the curriculum being overloaded with facts and information with less emphasis on development of higher cognitive abilities, skills and attitudes, besides shortage of teachers, predominance of didactic instruction and faulty assessment.^{[1],[2],[3],[4],[5],[6]}

The Medical Council of India (MCI), which prescribes the undergraduate curriculum, has recognized limitations of the lecture method and advocated the use of active methods. The Regulations on Graduate Medical Education (1997) emphasize that ′curriculum objectives are best taught in a setting of clinical relevance′. Further, MCI recommends that ′every attempt should be made to de-emphasize compartmentalization of disciplines so as to achieve both horizontal and vertical integration in different phases′.^[7] The question is: are we moving towards this goal?

The teaching of radiology in undergraduate medical education needs to be examined in the light of global trends in this discipline vis-ΰ-vis the Indian context. No doubt radiology has seen unprecedented growth as a specialty as witnessed by the manifold increase in investigative modalities.^[8] This is largely due to progress in imaging techniques, which has made diagnosis easier, faster and more accurate. However, teaching of radiology has not received as much attention. This is evident from the deliberations of the European Society of Radiology (ESR), which conducted a comprehensive survey covering 430 teaching hospitals from 26 countries and published a white paper on training in radiology.^[9] The Society noted that teaching of radiology in Europe varied from one medical school to another depending upon the hospital′s patient load, the availability of competent faculty and ′dedicated′ time given for teaching. The use of problem-based learning (PBL), or its nearest ally, case-based leaning (CBL), incorporation of digital imaging and picture-archiving communication system (PACS) were used in varying degrees within the format of radiology as an independent subject (classical model), integrated with clinical teaching (modular approach) or a combination of core and optional modules (hybrid approach). Compared with this, the teaching of radiology in the Indian context takes place within the framework of the curriculum prescribed by the MCI (classical model) with didactic lectures supplemented with a brief posting of about 2 weeks in radiology.

From the perspective of students, studies have shown that training in radiology should encompass all phases of undergraduate training, delivered through a case-based interactive learning using electronic sources.^[10]

Our objectives were to observe and compare the effect of innovative methods with conventional teaching on the performance of students, in the domain of knowledge and interpretive skills. We specifically studied whether integrated teaching of radiology using two methods-problem-solving exercises and PACS-had any beneficial effect on the students′ gain in knowledge and problem-solving skills.

The batch of students admitted in 2013-14 (n=40) was used for the pilot study for developing and validating tools for assessing the learning outcome.

Methods

We focused on undergraduate radiology training as one of the authors (CS) is a faculty member of radiology. We recruited the 2014-15 batch of undergraduate students (n=120) who entered the 5th semester preclinical studies.

The study protocol was approved by the Institute Ethics Committee. Informed consent was obtained from students of batches 2013-14 and 2014-15 for the purpose of recruiting them as study subjects. Considering the issues of confounding variables associated with experimentation, and ethical implications for optimizing the intervention benefits, we chose to conduct a prospective, quasi-experimental study to expose the whole batch to conventional and innovative methods by dividing the course content into two halves.

Ten topics related to common diseases or health conditions were chosen for the study. These were divided equally into innovative and conventional teaching methods, depending upon the weightage given to them in teaching and assessment, each containing five topics [Table - 1].

Table 1: Topics assigned for the innovative and conventional teaching methods

Validation of assessment tool

The tool used for assessment was a multiple-choice question (MCQ)-based written test and objective structured clinical examination (OSCE) stations for testing recall of knowledge and interpretive skills, respectively. The method of validation used was item analysis, which is considered a reliable method.

The MBBS batch of 2013-14 was used for validating the tool. Initially, a bank of 70 questions including 50 questions of recall type (MCQ-based) and 20 questions for testing interpretive skills (OSCE) was administered to students of the 2013-14 batch.

Item analysis was done as per the standard procedure leading to computation of difficulty level, discrimination index and distracter effectiveness.^[11] The questions that deviated from normal values were removed. We were left with 45 questions including 30 questions for testing recall and 15 questions for testing interpretive skills. Of the 30 recall-type questions, 16 pertained to the innovative teaching part and 14 questions to the conventional teaching part. Similarly, of 15 questions for testing skills, 6 were related to innovative teaching and 9 to conventional teaching.

Intervention

Students of the batch of 2014-15 (120), while they were posted in the radiology department for 2 weeks, in small groups of 10-15 were exposed to the innovative methods and the conventional lecture method and included system-wise teaching of radiology with didactic lectures aided by PowerPoint presentations. It was mainly for image analysis.

Innovative methods selected were based on a plethora of interactive methods discussed in the literature. ^{[12],[13],[14]} We picked up problem-solving exercises, CBL and integrated teaching.

The problem-solving exercise was related to clinical cases. It consisted of a case scenario with appropriate laboratory results given as a task. Students were required to arrive at a diagnosis of the said case and the differential diagnosis of the cases were discussed. During the discussion, apart from image analysis, interpretation of images was taught. Emphasis was given to knowledge of pathophysiology. The session ended with a brief line of treatment.

The problem-solving exercise was conducted in an integrated manner with participation of specialists from different subjects. Horizontal integration was achieved through participation of teachers from pathology and forensic medicine. Similarly, vertical integration was achieved through teachers from medicine, surgery, gynaecology, otorhinolaryngology, paediatrics and orthopaedics. For instance, when a case of pneumonia was discussed, the pathologist explained gross pathology and histopathology of pneumonia. The clinician discussed the signs and symptoms, besides treatment of pneumonia and the radiologist discussed the radiological investigation and findings, thereby giving a holistic picture of pneumonia.

The investigator (CS) conducted a pre-test and post-test for each group of students by administering the assessment tool validated earlier, viz. MCQs for assessing knowledge and OSCEs for assessing interpretive skills.

Statistical analysis

The students′ pre-test and post-test scores were entered in Excel sheets. Scoring was done for each student, item-wise, by assigning 1 mark for the correct answer and 0 for the wrong answer. We used SPSS to obtain the average number of correct answers given by the students in the innovative teaching part and the conventional teaching part separately for the knowledge-based questions and interpretive skills questions. This included minimum, maximum and median number of right answers/scores, comparison of gain/ improvement in knowledge scores, vis-ΰ-vis, interpretive skills scores by innovative and conventional methods, and we applied the paired t-test for determining statistical significance (p<0.05).

Results

Of the 16 questions used to assess knowledge in the innovative teaching part, the median number of questions correctly answered was 7 in the pre-test and 11 in the post-test [Table - 2]. Thus, there was a gain in the median by 11 and the difference between the mean (SD) of the pre- and post-tests, was statistically significant (p=0.001).

Table 2: Correct answers in the innovative teaching part

Of the 6 questions used to assess interpretive skills in innovative part, the median number of questions correctly answered was 2 in the pre-test and 5 in the post-test. There was also a gain of 3 points in the median in the interpretive skills and was statistically significant (p=0.001).

Of the 14 questions used to assess knowledge in the conventional teaching part, the median number of questions correctly answered was 5 in the pre-test and 11 in the post-test [Table - 3]. The difference between the correct answers in pre-test and post-test, was also statistically significant (p=0.001).

Table 3: Correct answers in the conventional teaching part

Of the 9 questions used to assess interpretive skills in the second part, the median number of questions correctly answered was 2 in the pre-test and 4 in the post-test. Though the difference between correct answers in the pre- and post-test was statistically significant (p= 0.001), the gain was the smallest.

We wanted to substantiate whether the gain in knowledge and in interpretive skills scores were statistically significant or not. For this purpose, we calculated the difference between the post- test and pre-test, which was found to be significant ([Table - 4] and [Table - 5]).

Table 4: Comparison of gain/improvement in knowledge scores by innovative and conventional methods

Table 5: Comparison of gain/improvement in interpretive skills scores by innovative and conventional methods

We infer that both conventional and innovative teaching methods led to gains in the students′ scores in knowledge and interpretive skills, which are statistically significant.

However, innovative teaching led to slightly higher gains (2 to 5) in the scores of students on items testing interpretive skills, compared with such gains made in conventional teaching (2 to 4).

Discussion

We explored the possibility of using innovative methods of teaching radiology to undergraduates and assess whether these had a beneficial effect on the students′ acquisition of knowledge as well as interpretive skills--the main objectives of our undergraduate programme. Our results indicate that innovative methods are useful in enhancing problem-solving and interpretive skills of undergraduates.

The contributory factors for this effect appear to be the role played by PACS in image recognition by visual perception. The problem-solving exercises and integrated teaching methods too may have contributed to the increased performance in interpretive skills.

Our findings are consistent with the theories of learning and their implications on medical practice. ^[15] Acquisition of knowledge can take place through behavioural modifications aided by practice and reinforcement, whereas higher cognitive abilities require processing and application of knowledge. This is aided through ′contextual learning′, which is the major plank of PBL or CBL. ^[16] The problem-solving exercises, coupled with integrated teaching might have played a role in enhancing students′ performance in this domain.

Many studies have validated the benefits of innovative teaching. As early as 1990, Dichter et al. showed that clinical problem- solving exercises provide an orthogonal approach to organizing medical knowledge. ^[17] Mamede et al. reported how students′ diagnostic competence could benefit most from the clinical case scenario approach. ^[18] Brigid and Linda reported a deep understanding among students subjected to collaborative methods. ^[19]

As regards integrated teaching methods, most medical schools in India organize integrated seminars in which the concerned specialists talk about their particular content, rather than a comprehensive approach accompanied by assessment. Sood and Sharan have reported integrated teaching of mental health in which faculty from Community Medicine and Psychiatry participate. ^[20] There is scope to see how best the integration can be achieved within the disciplinary structure of the MBBS course.

From the students′ perspective too, innovative methods have been acceptable and even preferable to conventional teaching methods. Kourdioukova et al. in their analysis of radiology education in Europe claim that an integrated approach leads to more effective education from the students′ perspective and helps them to develop a positive attitude towards radiology. ^[21] A study by Arjun Singh applying patient-oriented problem-solving system has shown that most students were in favour of the new method of teaching compared to didactic teaching. ^[22] Nyhsen et al. claimed that interactive teaching sessions offer better results in learning compared with the non-interactive ones.

However, it is necessary to identify the challenges and constraints in introducing innovative teaching methods and find ways to overcome these issues. Innovative teaching methods cannot be grafted in a vacuum. They requires a fertile soil in the form of a good infrastructure and resources for managing learning, and an assessment system aligned with the learning outcomes. ^[23] Achieving integration in the context of ′departmental feuds′, which exist in Indian medical schools, appears to be a challenging task. In our case, this was achieved through committed leadership, motivated faculty and persuasion by individual efforts.

A well-planned faculty development programme is indispensible to prepare the faculty to implement innovative methods in letter and spirit. Several articles have highlighted the importance of such programmes globally ^[24] as well as in the Indian context, including one study reporting highest priority assigned to the theme ′integrated teaching′. ^[25] The MCI has recommended establishment of a medical education unit in each college, and a comprehensive plan for nationwide faculty development programmes. ^[26] While these developments are laudable, there is a need to organize discipline-specific programmes, which can address the needs of faculty to apply best practices in their respective disciplines. Notwithstanding these challenges, the introduction of innovative methods is the need of the hour.

Our study has several limitations. We did not include an experimental design, which is considered gold standard for proving the efficacy of an intervention. Division of the curriculum in two parts would leave much doubt regarding their interference with the methods employed. Other limitations are the study sample limited to a single institute, and a single batch of students (excluding the batch of students 2013-14, used for the pilot study). There might be some pitfalls in our method of assessment and hence our results may not be what they appear to be.

Next, we plan to ′cross-over′ the methods for the new batch so that we have some additional data to support our claim. We recommend that more such studies are needed to provide a sound evidence base for optimizing the kind and quality of intervention.

Conclusion

The innovative approach to teaching radiology is an effective learning exercise. Teaching of radiology or any other subject is beset with the issue of developing competency of graduates in appropriate settings. There are challenges in terms of resource- constraints, shortage of trained, committed faculty and departments functioning in silos. However, the recent exploration of e-learning, new modalities of assessment and stimulus to faculty development, are promising signs.

Acknowledgements

We thank the students of 5th semester MBBS batch of Mahatma Gandhi Medical College and Research Institute (MGMCRI) for their active participation in this study, Deans, MGMCRI and management of Sri Balaji Vidyapeeth for their support.

Conflict of Interest. None declared