Importance of critical appraisal skills training in medical students

Critical appraisal is defined as ‘the process of carefully and systematically examining research to judge its trustworthiness and its value and relevance in a particular context’.¹ Critical appraisal allows one to interpret the relevance, validity and trustworthiness of the conclusions drawn from a particular study. Since medical science is a rapidly evolving field, clinicians from all specialties are required to read and interpret upcoming evidence for effective application in their practice. However, a variety of factors including a lack of emphasis in medical curriculum, rigidity and ignorance of practising clinicians, and a paucity of constraints on malpractice preclude the implementation of an evidence-based approach to healthcare in India.² The magnitude of this problem was realized during the Covid-19 outbreak in India. For example, favipiravir—an antiviral drug originally developed against influenza, has been widely prescribed in India despite poor evidence related to its efficacy.³ Similarly, convalescent plasma therapy has been indiscriminately used to treat severe Covid-19 despite conclusive evidence of its futility⁴ and an advisory by the Indian Council of Medical Research.⁵ Besides being costly, such interventions lead to mental harassment of patients’ attendants as they try to find suitable donors and sources.

The new National Medical Commission (NMC) syllabus for MBBS (implemented in 2019) includes critical appraisal of medical literature, but only as a single competency in clinical pharmacology ‘PH3.3. Perform a critical evaluation of drug promotional literature’.⁶ However, this placement is flawed as biostatistics and study designs will not be taught before this, which are essential for interpreting literature. The MBBS syllabus at the All India Institute of Medical Sciences (AIIMS), New Delhi is different from that of NMC and does not include critical appraisal at all.⁷

Recognizing this problem, the Scientific Society of AIIMS New Delhi organized an intramural ‘Online critical appraisal competition’ for second-semester MBBS students (batch of 2020) in May 2021. The competition could not be held in person owing to the Covid-19 situation. Each student had to appraise any randomized controlled trial (RCT) on a candidate drug or therapy for Covid-19. A total of 18 students volunteered to participate in the competition with topics including favipiravir, ivermectin, CD24Fc, doxycycline, azithromycin, dexamethasone, hydroxychloroquine, baricitinib, lopinavir–ritonavir, remdesivir, convalescent plasma therapy, low molecular weight heparin, Patanjali CORONIL, vitamin C and zinc, tocilizumab, inhaled budesonide, REGEN-COV and even a vaccine—ChAdOx1 nCoV-19. A professor from the Department of Neurology, AIIMS, New Delhi presided over the presentations to point out errors or missed points and to teach important concepts of evidence-based medicine (EBM). Each student was also assigned a ‘mentor’ who would be a more senior MBBS student having prior experience in research. The mentors were supposed to help the participants in searching the literature, understanding the methodology and finding resources to learn basic biostatistics. The overall schedule of the competition was flexible with 2–3 presentations held in the evenings.

Despite having no experience in reading medical literature, the participants tried to learn various concepts and interpret the studies. Initially, participants did experience problems in interpreting biases and presenting their appraisals. However, with due guidance from the presiding professor, they learned to interpret CONSORT diagrams, define the population, intervention, control, outcomes of a study, point out and classify biases and evaluate conflicts of interest.

Feedback from the participants was overwhelmingly positive. The following are excerpts of the feedback, which was collected through electronic messages:

This helped me gain some basic knowledge of statistics and terms used in clinical studies. This made me realize the whole process of RCTs and meta-analysis and how they ultimately culminate in clinical guidelines.

Now I can read and understand research articles. Earlier they were Latin and Greek for me.

It gave an idea—how to read research papers and interpret them. Instead of skimming through the conclusions, I can now understand the methodology and interpret p values, confidence intervals, etc.

A meta-analysis including 10 previous studies has shown that training of critical appraisal skills leads to an overall positive effect on knowledge of epidemiology/biostatistics, attitudes towards medical literature and reading behaviour.⁸ Further, of the 4 studies that assessed critical appraisal ability, results were negative in 2, inconclusive in 1 and positive in 1. Before concluding that the effects of training on critical appraisal ability are dubious, it is important to note that these studies were conducted in the USA, Canada and Mexico. In these countries, research experience is an important determinant in securing residency spots which may lead to a general awareness regarding research papers in most of the students. This may prevent formal training in critical appraisal from creating a major influence on the critical appraisal ability of these medical students. On the contrary, admissions to residency programmes in India are through entrance tests, and research experience is not taken into account. Overall, medical students’ research activities in India are in a poor state.⁹ Against such a background, formal training in critical appraisal can have a pronounced positive effect in India.

Further, instead of didactic teaching of critical appraisal, students should be tasked with appraising and presenting research papers. This form of problem-based, active learning facilitates an interesting learning environment and better academic performance in medical students.¹⁰ A study has shown that near-peer tutors were more effective and readily accepted in teaching critical appraisal.¹¹ This is similar to the model used in this competition by the Scientific Society where senior medical students were ‘mentors’ to juniors who had to present the appraisals.

In summary, a dedicated space for critical appraisal in the MBBS curriculum is needed for facilitating the practice of EBM in India.