Just-in-time teaching and peer-assisted learning in enhancing the flipped classroom

INTRODUCTION

The flipped classroom is an innovative approach to blended learning, which enhances student participation and allows a greater breadth of the subject to be covered in a short period by allowing a bulk of the core learning to take place outside of ‘class time’, thus enabling higher-order learning in the classroom. However, poor student compliance and lack of engagement impair the effectiveness of this technique. A study conducted by Hobson et al. showed that 80% of students do not read preassigned material in college courses.¹ In addition, despite the introduction of newer pedagogical techniques, active and collaborative peer learning are poorly represented in flipped classroom sessions.

These barriers can be overcome by introducing two additional elements: just-in-time teaching (JiTT) and peer-assisted learning (PAL) through concept tests, which can further enhance student engagement with course content and elevate the role of the teacher from a dispenser of information to an enabler of higher-order thinking.

JiTT is a two-step instructional technique that encourages student ownership of teaching–learning as well as provides a ‘just in time’ feedback, so lessons can be tailored to meet student-specific needs. PAL, through concept tests (conceptual multiple-choice questions [MCQs]), provides students with the opportunity to address problem areas in the course material through peer interaction and derive solutions through teamwork. The JiTT and PAL approach supplements the flipped classroom pedagogy by allowing for better student engagement with preassigned reading and providing students and teachers the opportunity to recognize background knowledge. It can smoothly lead students from understanding basic ideas to more comprehensive concepts through content tailoring, targeted questioning and peer interaction and can also serve as a formative assessment tool.

Since its introduction by Novak et al. to promote student engagement in college education, the JiTT model of instruction has been used in various disciplines.^2,3 Peer discussion in the classroom teaching promotes critical thinking and encourages collaborative learning.⁴ Mazur et al. have recommended combining the use of JiTT and peer instruction (PI) as a means to make the student learning process transparent both before and during class sessions and to allow for more ‘personalised’ learning.⁵

A pilot study in a general surgery residency programme utilized a JiTT and PAL instructional strategy—more than 70% of students had a favourable response; 90% faculty reported positive perceptions of aspects of the JiTT and PAL strategy. Resident engagement time was significantly greater, and more MCQ responses were correct for JiTT and PAL attendees.⁶ Madiraju et al. implemented the JiTT approach, integrated with conceptual MCQs, and reported that overall satisfaction and endorsement of JiTT were favorable. JiTT-based courses had better and more consistent performances in examinations than non-JiTT courses.⁷

MEDICAL EDUCATION

We investigated the impact of the dual interventions, JiTT and PAL on learner outcomes in undergraduate medical students. We determined the effect of JiTT in increasing learner engagement with course content, critical thinking and self-reflection before a flipped classroom session for undergraduate medical students, determine the impact of concept tests-based PAL on learners’ performance, problem-solving and collaborative skills, in the flipped classroom and to determine learners’ satisfaction with the JiTT and PAL approach in the flipped classroom setting.

METHODS

This study was done in the Department of Obstetrics and Gynecology, ESIC Medical College and Hospital, Faridabad, from May to August 2024. Ethical clearance was obtained from the Institutional Ethical Committee (DHR Reg No: EC/NEW/ INST/2023/HR/037) vide letter number 134 X/13 November 2024-IEC/DHR/71 dated 29th April 2024.

Twenty-five Phase III MBBS students posted for 4-week continuous clinical rotations in the department of obstetrics and gynaecology were invited to participate in the study. Two students withdrew later for personal reasons. The remaining 23 students participated in the study after written informed consent.

The intervention was administered as four JiTT and PAL enabled flipped classroom sessions (Fig. 1) in a small group setting, covering topics related to family planning and contraception competencies as detailed in the National Medical Council undergraduate curriculum document. All teaching sessions were conducted by a single teacher with 10 years of teaching experience in undergraduate medical education.

Close

FIG 1.

Just-in-time teaching (JiTT) and peer-assisted learning (PAL) intervention framework MCQs multiple choice questions

Export to PPT

Departmental peers collectively reviewed the pre-class reading assignments before the intervention to assess for relevance to teaching–learning objectives and current clinical practice. The student response questionnaire was validated independently by 6 experts. The content validity index for the questionnaire was 0.967.

Preparatory assignments included reading material, presentations and/or videos. Warm-up assignments included 3–5 recall-based MCQs related to the pre-assigned preparatory material. Two additional questions were asked:

1. Minute paper: Learner was asked to summarise the key points of the study material in 3–4 sentences (50–80 words).

2. Muddiest point: Learners were asked to write the area/ concept that was ‘most unclear/needs further clarification in the pre-assigned reading’. The learner could respond in one of three ways––(i) submit their ‘muddiest point’ if identified, (ii) submit ‘none’ if there was no ‘muddiest point’, or (iii) submit ‘can’t say/don’t know’ if they were unclear about ‘muddiest point’ or ‘did not wish to submit’. This option was created to prevent the answer from being rigid and dichotomous, allowing for the representation of ‘uncertainty’ and ‘reluctance’ in responding.

The content of the classroom sessions was tailored to the responses received through the JiTT submission of warm-up assignments. The class began with a brief introduction and a statement of learning objectives. The learners’ responses to warm-up MCQs were included in the PowerPoint presentation in the form of pie charts, and these were discussed through peer interaction. The content, flow and complexity of the class were determined based on the performance of learners in the warm-up MCQs and ‘muddiest points’ submitted. The second part of the intervention was concept-based MCQs (3–4) or concept tests that were administered as clickers through an online, real-time survey application. Each concept test included a stem containing a life-like clinical scenario, and multiple options reflecting common clinical conundrums of moderate complexity. Most concept tests had multiple correct answers to stimulate the learner to arrive at conclusions through iteration, and to prevent calculated guesses. The learners were asked to submit their answers individually first. If <30% answered correctly, the topic was re-visited by the teacher in detail. If more than 70% answered correctly, the correct explanation was provided, and the teacher summarized the topic to move on to the next concept test. If 30%–70% learners responded correctly, peer interaction was done through ‘Think– Pair–Share’ and learners were asked to re-vote. The learners were then asked to vote again individually, and the pre- and post-intervention scores were noted.

At the end of the study period, student feedback was collected through a Google form survey containing 10 rating scale-based questions and an open-ended feedback option.

All classroom sessions were video recorded for analysis.Primary analysis of time spent in teacher-led and PAL sessions was conducted by a research assistant in real-time during the classroom sessions. This was later re-checked for discrepancies by the principal investigator from the video recording of the teaching sessions.

RESULTS

Of the 23 participants in the study, 19 (83.7%) students submitted warm-up assignments and minute papers before the submission deadline. This was significantly above the established benchmark of more than 70% submissions. Overall, 6 (28%) of learners were able to identify the ‘muddiest points’ after reading/completing the prescribed pre-class reading, 10 (41.6%) did not have any doubts/‘muddiest points’ while 7 (29.9%) chose not to respond to the question.

As per the study protocol, PAL was enabled through concept tests. Of the 11 concept tests posed in the teaching sessions, 8 fell in the 30%–70% correct student response category in the initial vote. These were then discussed through peer interaction, and a re-vote was done. Fig. 2 depicts a session-wise break-up of the proportion of correct responses pre- and post-PAL. The increase in the proportion of students answering correctly was significant, ranging from 21% to 60% (Table 1).

Close

FIG 2.

Details of learners in just-in-time teaching (JiTT) and peer assisted learning (PAL) enabled flipped classroom sessions

Export to PPT

Data pertaining to individual student scores for all concept tests was checked for normality using the Shapiro-Wilk test, and was found to be normally distributed and mesokurtic. A right-tailed paired sample t-test was applied to determine if the intervention caused a statistically significant positive change in mean scores. The pre- and post-intervention mean test scores were 49.9% (14.8%) and 87.2% (10.1%), respectively (p<0.001) and were significantly different.

The average normalized gain is a measure of the effectiveness of an intervention in promoting conceptual understanding. It was calculated using the formula <g>=<Post>– <Pre>/100– <Pre> where <post>=post-intervention mean class score=87.2 and <pre>=pre-intervention mean class score=49.9. The average normalized gain was 0.75, which is significant and is likely to improve the learners’ progress. Overall, 68% of the time was spent in PAL, while 32% of the total teaching time was spent on teacher-led learning (Table 2).

Learners’ responses to the Student Response Questionnaire are depicted in Table 3. All respondents felt that the pre-class reading assignments were adequate and relevant, and their ‘muddiest points’ were sufficiently addressed in the classroom session. 95% learners felt ‘safe and comfortable’ in a PAL environment, and that their problems were ‘sufficiently addressed’ through peer interaction. The mean (SD) learner’s satisfaction score (on a numerical score of 1–10) for the entire intervention was 9.33 (0.86).

DISCUSSION

We conducted this study among Phase III Part II MBBS undergraduate students who had previously been taught the topic of Family Planning and Contraception using the traditional didactic approach. Contraception as a topic requires a thorough understanding of the normal physiological processes, a knowledge of the hypothalamic–pituitary and ovarian axis, a broad understanding of pharmacology, as well as the application of scientific knowledge in a wide-ranging clinical context. Student engagement with pre-class reading assignments was gauged by submission of warm-up assignments and minute papers. Both parameters exceeded the pre-determined benchmarks for good engagement. However, the proportion of students who were able to identify their weaknesses and problem areas was relatively low (28.5%). There was greater interest in the classroom sessions, with a mean classroom attendance being 81.5%. Students felt that classroom sessions were well attuned to their learning needs, indicating the importance of JiTT in designing course content effectively.

It is noteworthy that the average normalized gain for the intervention is substantially high. This means that this intervention enables a learner to achieve a 75% improvement in their pre-test scores, which is a major gain in learning. The effect size, which emphasizes the impact that the learning gain makes, is high at 2.95, indicating that an improvement of 2.95 standard deviations can be expected using JiTT and concepts that enable PAL in the flipped classroom.

The mandate of the medical educator is ‘not to transmit knowledge to a passive recipient, but to structure the learner’s engagement with the knowledge, practicing the high-level cognitive skills that enable them to make that knowledge their own’.⁸

The flipped classroom is an innovative and interactive approach that encourages the use of in-class time for higher-order thinking and collaborative learning. However, the lack of student engagement and poor application of higher-order cognitive skills in in-class sessions often mar the impact of this approach. The JiTT intervention aims to create a bridge between pre-class and in-class activities, ensuring learner engagement with pre-class reading assignments.

JiTT enables both the student and the teacher to become aware of the student’s background knowledge. By assigning JiTT exercises, an instructor can gauge the level of knowledge and discover the learners’ weaknesses and difficulties to create appropriate class content. Thus, classroom time can be effectively managed to facilitate engagement in problem-solving and critical thinking.

An efficient way to do this is through the use of PI. PI can be effectively delivered through concept tests or conceptual MCQs. These are higher-order MCQs that focus on one key aspect of the instructor’s major learning goals for the session. When coupled with student interaction through PI, concept tests represent a rapid method of formative assessment of student understanding, require minimal changes to the instructional environment, and introduce many of the recognized principles of effective teaching that enhance student learning. The use of concept tests for PAL encourages faculty–student contact, promotes active learning, provides prompt feedback, and respects diverse talents and learning styles.⁹ A study done among students of the doctor of pharmacy curriculum observed a significant improvement in correct responses (90.4% v. 51.2%) in pre- and post-PI polling. This increase was observed across all phases of the curriculum and was higher in cohorts that had previously participated in PI sessions.¹⁰

The success of the JiTT and PAL approach is heavily dependent on the quality of the concepts that are used for PI and collaborative learning. JiTT comes to the rescue here by providing the instructor with relevant feedback on the student’s knowledge and comprehension of the material, allowing customization of the classroom sessions to the learner’s unique needs. Engagement with the pre-class reading material by students not only prepares them for the class but also fosters a habit of self-reflection and critical thinking. JiTT prepares the learner to face the concept tests in our proposed format, as factual knowledge is a prerequisite for meaningful PI and problem-solving.

While the JiTT and PAL approach is well-researched in the basic sciences and undergraduate and postgraduate university programmes, there is a notable lack of research of its application in medical education. We attempted to combine two evidence-based interventions that are backed by a solid theoretical framework. We found encouraging results in terms of student engagement, peer collaboration, self-reflection and enhancement of academic outcomes. There are a few limitations of our study, the primary one being the short duration and the application of the method to only a single module in obstetrics and gynaecology training. Another drawback of the JiTT and PAL framework is that it requires both motivation and expertise from the faculty. Furthermore, all sessions were conducted in a single discipline by a single instructor.

The JiTT and PAL approach can be potentially adapted to all disciplines of medical education, with some effort and planning. This pedagogical technique is most suitable for cognitive competencies that require application and analysis, typically targeting clinical skills with ‘knows how/shows how’ level of proficiency. This approach also provides an in-built and tailored formative evaluation mechanism for teaching/learning. Pre-classroom assignments and concept tests, once created, can be reused over time with little or no modification. The use of IT-enabled tools, such as real-time surveys through mobile-based applications and online-response collection and interpretation systems, greatly eases the effort involved in developing and dispensing pre-class reading material and warm-up assignments, as well as facilitates easy data collection and evaluation of responses, thus making the intervention less labour-intensive and cost-effective.

We believe that this intervention holds great promise in transforming the largely didactic ‘tradition’ in the Indian medical classroom; therefore, it needs to be tested in larger populations over extended periods and across various disciplines to identify and harness its potential fully.

Conclusion

The JiTT and concept-based PAL approach improves student engagement, peer collaboration, self-reflection and critical thinking. It enables a more efficient use of in-class time for higher-order learning and problem-solving. This approach promises to be an excellent addition to the flipped classroom.