Evaluation of the preclinical hospital visit programme with students’ feedback at the University of Nottingham, UK

INTRODUCTION

Hospital visits are a crucial component of the early clinical and professional development course at the medical schools in the University of Nottingham, UK. In this programme medical students achieve competencies in basic clinical skills and ethics by hands-on experience of performing clinical examinations on patients. Early clinical and professional development course was established in the medical school with the aim to address broader issues of professional awareness at an early stage as advocated in the GMC publication Tomorrow’s Doctors (2009) by integrating clinical practice with teaching of basic sciences. Similar programme for fresher medical students was introduced in 1971 in the department of Family Practice at Howard University College of Medicine to expose medical students to patient care early in their careers. Almost 89% students had rated this programme as good to excellent.¹

The structure of the hospital visits comprises an introductory lecture on history taking and clinical examination of the relevant system, briefing and demonstration of the examination by the teaching fellows (TF) in smaller groups. This was followed by hands-on patient examination by students under supervision. Medical students are expected to attend practice sessions at the skills centre before the visit.

Hospital visit programme is not only beneficial to the medical students but also helps the participating junior doctors and teaching fellows to develop presentation and communication skills through teaching.² Clinical experience is an important part of the integrated medical curriculum. It provides a foundation for experiential learning in the early medical years by which the students understand the relevance of the basic science knowledge and clinical practice in a real hospital setting.^3,4 During the debriefing, medical students can put forward questions and hence identify gaps in knowledge so as to address and improve one’s learning by consolidating the understanding of clinical topics.⁵

A systematic review by Dornan et al. included 73 studies done between 1992 and 2001 evaluating the influence of early clinical experience in hospital and community settings in developing professional attitudes, empathy, confidence and motivation in medical students. The American studies in this review had noted that early clinical experience in preclinical years led to increased recruitment to primary care and rural medical practice.⁶ However, the gap in the research is the need to evaluate the outcomes of the early clinical experience programme to understand their effectiveness in shaping the clinical careers of medical students and plan the medical curriculum. Hakim et al. in 2014 emphasized the role of integrated model of medical curriculum by incorporating clinical experience earlier in medical education to reinforce cognitive, psychomotor and affective domains of learning and promote professional and social development.⁷ Johnson et al. noted positive relationship of early clinical exposure and students’ satisfaction with medical education by significance tests.⁸ Clinical elective licence (CEL), a practical course for early medical students in Austria was highly rated by 1st and 2nd-year students in their feedback based on evaluation in 2011. The students had reported increased confidence with patient interaction and communication.⁹ The Medical Council of India ‘Vision 2015’ aims to produce medical graduates at par with global standards by reforming the preclinical curriculum with early clinical exposure. Govindarajan in a study on learning objectives of 150 preclinical students posted to various specialist clinics and intervention laboratories noted significant increase in post-test scores in all the specialist areas where early medical teaching was associated with clinical experience. This was based on analysis of open feedback from students, which showed significant impact on their cognitive, psychomotor and affective domains. This study model has introduced new medical students to professional, interpersonal, scientific and ethical aspects of medicine earlier in their careers.¹⁰ Study on preceptorship in internal medicine at preclinical level has shown to improve the students’ performance during internal medicine clerkship.¹¹ Further studies on clinical experience achieved by introduction of various programmes to promote clinical skills in preclinical year students have reported improved performance, professionalism and satisfaction with the curriculum.^12,13 Although all these studies have shown improved student experience and satisfaction with learning clinical skills, it is crucial to conduct course evaluation to understand the effectiveness of the programme. Goldfarb and Morrison in 2014 showed the effectiveness of collecting students’ feedback in real time, that is while the course is ongoing rather than wait till the end of the course to enable course modification effectively. This maximizes student involvement in course development, improvements in curriculum and identifying weaknesses in the programme and in students’ experience.¹⁴ A study on students’ course evaluation has mentioned that courses which are well organized, clearly communicated with helpful teaching staff and delivered goals and objectives of curriculum were generally rated highly by medical students. This study has further described the limitations of students’ evaluation mainly that the quantitative ratings are subjective; students may not appreciate value of the curriculum content which ultimately should be finalized by the medical educators and not just based on students’ ratings. Hence, the end of the course evaluation questionnaires should include effectiveness of faculty teaching, educational material, course organization and achievement of the course objective.¹⁵ Benton and Cashin, 2014, have recommended that students’ evaluation of teaching (SET) is effective for assessing the teaching programme or faculty. SET is a multidimensional tool which includes several factors such as course organization, planning, communication skills, teacher– student interactions, course difficulty and grading systems. These factors are scored, so far a single global score has not been identified to validate teaching effectiveness.¹⁶

With the literature review, we have identified the gap in the research related to the effectiveness of the end of the course students’ feedbacks for evaluation of the preclinical skills programme. We hope to bridge this gap by analysing the medical students’ end-of-course feedback suggestions for hospital visit programme and implementing their suggestions to modify the course based on their perceptions and further measuring the effectiveness of the modifications with feedback from the students’ experiencing the changes to the training programme.

Our study was to evaluate the preclinical (1st and 2nd year) medical students’ experience of hospital-based clinical teaching programme between 2011 and 2013 on their feedback forms based on the students’ feedback. The students’ suggestions were implemented to bring about suitable modifications to the hospital visit programme. The modified programme was similarly evaluated with students’ feedback. The objective of the research was to gauge the effectiveness of the SET by comparing their pre- and post-modification ratings of the programme.

METHODS

This quasi-experimental pre- and post-test intervention design study involved comparisons of students’ feedback obtained before and after suitable modifications in the hospital visit programme with 337 feedback forms available in each group. The pre-test group feedbacks were collected from February 2011 to December 2011 and the post-test group feedbacks from February 2012 to May 2013 (Table I). This work was carried out in accordance with the Declaration of Helsinki with no potential harm to participants, and the anonymity of participants was maintained. As this study was not a part of a clinical trial and did not involve patients or research participants, ethics approval was not sought.

The pre-test group had experienced hospital visits based on an old format, and the post-test group had experienced modifications in the programme based on the feedback suggestions of the previous group. In the pre-test hospital visit programme, all the year group students had an introductory lecture, following which groups of 7 students were formed and each group was supervised by one TF. Based on the feedback, it was noted that the number of patients seen by each student was unpredictable, ranging from 1 to 4; also, the amount of time spent on the ward was left to the discretion of the TF. This resulted in dissatisfaction among the students, which was reflected in their feedback. A major issue identified was of a high student to TF ratio which made it difficult to provide one-to-one supervision for students in the group. To improve on the old format, modifications in the hospital visit programme were introduced in the first quarter of 2012. The important changes incorporated were 2 TFs were allocated to a group of 7–8 students; hospital visits time was stipulated from 9 a.m. to 12 noon and not finishing before 12 noon; each student had to see no less than 3 patients; debriefing was conducted at the end of hospital visits by the TFs; TFs were mandated to collect student feedback at the end of the hospital visit.

The feedback form used is shown in Table II. The questionnaire format of the feedback form is similar to a tool for the evaluation of clinical learning environment questionnaire. Five main factors considered by this tool are cases, authenticity of the clinical learning experience, supervision, organization of the doctor–patient encounter and motivation to learn.¹⁷

In the free text section, a total of 898 comments were noted for both the groups. These comments were on two questions asked in the feedback, first ‘What did you enjoy about the hospital visit?’ and ‘What could be improved?’ These comments or codes were grouped into pre-test and post-test with four main themes of introductory talk, hospital visit organization, patient experience, and briefing and debriefing. Salient comments are shown in Tables IIIa and IIIb.

Quantitative response questions in the feedback were statistically analysed using Welch’s t-test, and free text questions were qualitatively analysed and grouped into themes.

RESULTS

The feedback forms of the preclinical 1st and 2nd-year medical students for the hospital visit programme were studied retrospectively before and after the changes were introduced in the programme.

DISCUSSION

The changes brought about in our preclinical hospital visit programme were based on the recommendations of pre-test students’ feedback, mainly improving students to TF’s ratio, committing at least 3 patients to each student for bedside examination and hospital visit session lasting not less than 3 hours. Evaluation of the post-test group feedback forms has shown positive response for the modified programme. This was mainly brought about by motivation of the students in both the groups to actively provide feedback on the teaching programme.

The student’s giving feedback identified and rectified the factors affecting the current programme to overall improve the management of the current programme, course content and grades. As the feedbacks were anonymised, the students were more open about their views. The advantages of teaching feedbacks were to give teachers the opportunity to improve the learning and teaching experience of the students.¹⁸ We noted that to improve the hospital visit programme and the post-test feedbacks of the students, the TFs had brought about changes to the programme based on pre-test students feedback such as modifying the introductory lecture to include demonstration of practical skills, coordinating with other TFs to improvise oneto-one observation of students performing clinical skills, ensuring enough number of patients are made available to students for examination; these modifications in the teaching programme had positively influenced the feedbacks from the post-test student groups. Hence, our study shows that SET programme has motivational influence on the teaching staff thus improving the teaching programme. These findings illustrated that the students were keen and eager to learn and gain the maximum from their clinical visits. The feedback was taken seriously, the issues addressed and improvements implemented in early 2012. Analyses of the post-test feedbacks revealed positive responses and satisfaction with the programme. Our study shows the validity and reliability of student feedback for improvement of various programmes in higher education.² The students who provided the initial feedback were not the same who experienced the changed programme and provided the subsequent feedback. This illustrates that it was not just a question of one group of students ‘getting what they asked for’ but a genuine improvement in the programme as experienced by the second set of students who had no knowledge of the concerns of the first group of students and the changes implemented as a consequence. Wong and Moni had observed that SET is an effective tool for improving the education programme only when the teaching and training staff take into account the feedback and implement appropriate actions. Salient features of this study are institutional requirements, operational practices, personal biases and provision of support.¹⁹ There is limited evidence about how the teachers in higher and medical education in particular, systematically apply student feedback to improve the quality of their teaching practice.²⁰ In our study, the concerted effort of the programme organizers to introduce and implement changes as suggested by the students’ feedback had resulted in major improvement in the post-test groups feedback. Our findings are supported by observations from studies by Scot et al. and Schiekirka and Raupach that the concept of pre-test and post-test feedback design is relatively simple and effective for constant improvements of concise medical education modules.^21,22

This is one of the first studies involving clinical experience of preclinical medical students to demonstrate effective utilization of students’ feedback to evaluate and accordingly modify the programme leading to increased students’ satisfaction. It was performed retrospectively due to which certain biases experienced by the participating students could not be eliminated or controlled by the researchers. It was also noted that due to organizational limitations of resources, not all students’ requirements were fulfilled in spite of the popular demand such as each student getting to examine more than 3 patients independently in every hospital visit or allowing students more than 3 hours on the ward for each visit as it compromised patients’ eating and resting times (Table IIIb).

The evaluation of medical education based on students rating assesses three of the four main features, structures, processes and teacher characteristics. Schiekirka and Raupach have noted that evaluation activities are not mandatory at all the universities; hence, selected students providing course ratings might create a bias.²² However, at our medical school, the preclinical students attending the hospital visit programme had to compulsorily submit the feedback to evaluate the programme. Evaluation studies have also valued evaluating learning outcomes. However, difficulty in generating valid and reliable tools to assess this feature has been noted. In our study, learning outcomes were not evaluated as there was no summative assessment at the end of the programme.¹⁶ SET was effectively used to evaluate the hospital visit programme in preclinical years by assessing important factors of students’ satisfaction such as teaching performance, introductory talk, demonstration of the systems examination, briefing, debriefing, available facilities, number of patients seen, and time spent on the ward.