Competency-based learning and training for medical postgraduates within regulatory guidelines in India: The SBV Competency-Based Learning and Training Model©

Introduction

The majority of students qualifying in the MBBS examinations in India opt to do postgraduation rather than accepting other avenues of employment. Keeping this in mind, as also the fact that the postgraduates in training are going to be the specialist workforce in the country after completion of the course, it becomes mandatory to ensure that there is uniformity in postgraduate training across institutions in India and a minimum acceptable level of knowledge, skills and attitude is reached by all. This requires a criterion-based summative evaluation which does not exist at present. Therefore, the exit standards remain uncontrolled with large variations between institutions and between universities.

Introduction of the National Eligibility cum Entrance Test (NEET) examination has to some extent tried to standardize the level of graduates entering the postgraduate system. This, however, is undermined by the fact that NEET has had to lower the percentile progressively to fill up postgraduate seats in view of the inability of the selected candidates to pay the prescribed fees in private institutions or their discontent with the subjects or institutions offered. Last year, according to reports in the press, the percentile was lowered to 21% before all available seats were filled. The proposed licentiate examination after the undergraduate course may bring about some uniformity in the level of outgoing undergraduates, but postgraduate training would still remain widely different. The licentiate examination would add to the problem by accumulation of candidates who pass the university examination but do not clear the licentiate examination.

For decades now, no major changes have been introduced in either the postgraduate syllabus or evaluation process in India by the regulatory agency. The few measures that have been taken are more cosmetic than substantial.^[1] The Medical Council of India (MCI) no doubt has recommended that the ‘postgraduate curriculum shall be competency based’. In spite of this core principle being enshrined in the Council's postgraduate medical education regulations, so far, no concrete steps have been taken in the country to establish and implement a competency-based medical education programme. There are a number of reasons for the present state. First, there is no universally agreed list of skills and competencies required to be attained by residents in India during the period of training. The expected standards vary from institute to institute, and even within the same institute, from one faculty to another! Second, there is no mechanism for recording or monitoring the progress of individual students on a regular and continuous basis. The logbook is a poor substitute for this purpose since it is often filled up by postgraduates just before the final examination, does not record the progress of the candidate on a regular basis and does not give scope for reflection. Third, there is no scope for tailoring intervention based on the levels attained by the individual postgraduates at various intervals of training. Last but not the least, the assessment is based on the final examination when it is too late for interventions.

A postgraduate working group constituted by the board of governors of the MCI, after a year-long discussion, brought out a report recommending transformational changes in the postgraduate training process to meet the requirements of the country.^[2] The fate of this report remains unknown after 6 years. With its attention totally focused on its regulatory function, the MCI has little time to devote to curricular reforms. However, since it has overwhelming powers in recognition or de-recognition of courses, efforts in the country to be innovative have been few and far between.

Any change to bring about quality in the postgraduate training process must conform to regulatory guidelines and, therefore, should avoid tinkering with the exit summative process which is controlled by the MCI. The system should also provide for individualization of monitoring and training of postgraduates with opportunities for additional training inputs to those who lag behind. A conventional competency-based curriculum provides for ‘fixed standards with a flexible time schedule’. This obviously is not possible within the MCI system which is largely ‘flexible standards within a fixed time’. The third model described by us allows for ‘fixed standards within a fixed time’, which is what is possible within the regulatory norms. This model evolved and was implemented in Mahatma Gandhi Medical College and Research Institute under Sri Balaji Vidyapeeth (SBV), Puducherry. The name and the process have been copyrighted solely for getting credit for the name and the particular process for accreditation purposes. Copyrighting, in the Indian context, does not prevent any other institution from using it as it is or modifying it to suit their own situation and does not involve payment of royalty to the copyrighting institution but merely is a record that the particular model was first introduced in the institution from which it has been copyrighted. It is not intended to serve as a model prescribed for all others.

SBV introduced a unique Competency-Based Learning and Training Model© (CoBaLT) for setting up a high standard of postgraduate medical education in India to ensure that the outgoing postgraduates have reached a minimum level of prescribed competency and can function independently as specialists, researchers or medical teachers, when they complete their course. This model is a step that addresses the current variability of standards within and across postgraduate departments, with overemphasis on final examination (summative evaluation) of the residents and lack of opportunities for feedback and improvement to individual candidates. To the best of our knowledge, such a process has been set in place for the first time in India in postgraduate medical education. The model is described herein so that other institutions may consider whether it offers any advantages and if they feel it does, whether they would like to adopt it as a model or develop another to suit their own individual circumstances.

Description of The Process

The process involves several sequential steps as described below.

Definition of competencies

‘Competence has been defined as the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values and reflection in daily practice for the benefit of the individuals and communities being served’.^[3] Competency, therefore, refers to attributes or qualities of the outgoing graduate and may include all components of education, namely knowledge, skills and attitudes and communication skills.

Several models of competencies used abroad are available for use in India.^[4],[5],[6] Of these, the Accreditation Council for Graduate Medical Education (ACGME) model is the most comprehensive and simplest to understand. It divides competencies into six domains as follows:^[5]

1. Patient care
2. Medical knowledge
3. Interpersonal and communications skills
4. Professionalism
5. Practice-based learning and improvement, and
6. System-based practice.

Each domain includes a set of subcompetencies.^[7] The newly released attributes of the Indian Medical Graduate by the MCI,^[8] the competencies of the ACGME and the Canadian list of competencies match in many aspects although the nomenclature may vary. These are listed below.

Entrustable professional activities.

Since competencies are attributes or qualities, it may not be possible to directly measure them and certify whether the graduate has attained that particular competency. Hence, it is the practice to rewrite the competencies in a more measurable form as an entrustable professional activity (EPA) which the graduate can be safely expected to perform at the end of the training at the desired acceptable level.^[9] EPAs ensure that the performance of the learner in an authentic learning environment can be observed and measured.

EPAs can be divided into core EPAs which may be common to all departments providing postgraduate education and subject-specific EPAs which are relevant only to the concerned department. Each department may have a list of EPAs amounting to 30–40 in number which all postgraduates must show evidence of acquiring during the course. EPAs for different subjects as employed at present in different countries are freely available in the public domain and have to be tailored to suit the requirements of India. This requires a cooperative endeavour of the concerned faculty.

Having made a list of EPAs for the subject, it is necessary to mark which of the domains the EPA involves. This is required so that appropriate methods of evaluation may be matched to the domains. This step may require the help of a medical educationist. Each EPA may involve one or more of the six domains as per the ACGME norms. It must be remembered that the whole of the curriculum would not be covered by the EPAs and some portions may still require other methods of training.

The EPA list is not considered to be written on stone and requires revision by all faculties with external experts on a regular basis.

Levels

Having rewritten competencies as EPAs, it is necessary to fix the acceptable levels of performance of the postgraduates at various intervals of time. The Dreyfuss model of level of competencies appears best suited for this purpose.^[10] In this model, the levels range as follows:

1. Is new to the competency: Level I (Novice)
2. Able to perform under strict supervision: Level II (Advanced beginner)
3. Able to perform under loose supervision: Level III (Competent)
4. Able to perform independently without supervision, Level IV (Proficient) and
5. Able to perform professionally and teach others: Level V (Expert).

The list of EPAs and the levels and milestones must be made available to the resident at the time of joining.

Milestones are the intervals during the course, at which the levels are assessed to determine progress. In the SBV model, the EPAs are self-assessed at the time of joining by the candidates and at 1 month by the faculty to demonstrate to the students the difference between their judgement of their own ability and that judged by the training faculty. Thereafter, the levels are assessed at 3-monthly intervals during the 1st year and 6-monthly intervals thereafter. Frequent assessment in the 1st year and if required in the second and third years enables interventions to be planned if the candidate lags behind the required level. The acceptable levels at each milestone are fixed by the faculty for their subject. The acceptable exit levels are generally fixed at Level IV for most EPAs and at Level III for complex EPAs like the ability to perform a pancreatectomy. For postgraduate diploma courses which are 1 year shorter as per regulatory norms, the levels have to be fixed at one level below, where appropriate. This distinction has become irrelevant since the postgraduate diplomas have been discontinued by the regulatory agency. As a model, examples of EPAs developed for 3 of the 20 postgraduate disciplines, namely, the disciplines of otorhinolaryngology and psychiatry for the diploma and degree courses are shown in the annexures (available at www.nmji.in). Others are available at the SBV website for use of postgraduates.

Assessment of professionalism and multisource feedback

Assessment of professionalism requires a process of observation in the workplace by all members of the healthcare team involved with the concerned postgraduate and not only the subjective impression of the unit or department head. Hence, it is necessary to evolve a system of obtaining multisource feedback (MSF) from peers, patients and relatives, nursing staff, other healthcare workers and for pre- and paraclinical postgraduates from laboratory staff in addition to the faculty. In view of the cultural issues involved in India due to perceived difference in status symbol between doctors and other categories of health workers, which results in reluctance on the part of these healthcare workers to give a written feedback on doctors, it is necessary to ensure that the feedback form is completely anonymous, is simple to fill and does not involve a detailed description of criteria or rubrics of assessment. It would make this step difficult to work since the return of filled forms from other healthcare workers would be very few. In any instance, since several feedbacks are taken from several categories of health workers, which is only part of the information available to the heads of unit and department, objectivity is not considered of overwhelming importance. After all, this information is to be used only for private counselling if it is found to be consistent and not used for ranking or other purposes in public. Detailed Likert scales are unlikely to function in the Indian context. Examples of the MSF forms designed by SBV for this purpose are shown in [Figure 1],[Figure 2],[Figure 3]. By trial and error, it has been found that the simple 3-point format works best in practice as it is easy to comprehend by those giving feedback. Some departments such as psychiatry may require a more specific MSF. Such a feedback should be obtained not less frequently than once in 3–4 months to be meaningful. Those who are underperforming need to be called and counselled in confidence by the head of the unit and the head of the department.

Reflective learning

For a system to monitor and provide feedback to the postgraduates regarding their shortcomings and deficiencies, and also offer them help with their learning difficulties, it is necessary to allow for reflection on the part of the concerned postgraduate on his/her teaching–learning experiences and environment, and provide a forum for expressing his/her doubts as also seeking clarifications. This should be a continuous ongoing day-to-day process. The importance of reflection on learning has been well documented.^[11]

For enabling a continuous process of reflection and feedback on reflections, all postgraduate students on admission are allotted to a mentor from among the faculty. They are encouraged to constantly record their reflections on the learning process and the mentors are obligated to respond as soon as possible. In addition, all the other faculty are also encouraged to respond to the issues raised by the postgraduate student even if they are not the concerned mentor if they have something to contribute to the discussion. The resident on receiving the feedback from the faculty is encouraged to raise additional issues if he/she chooses to clarify further issues. The reflective process also enables identification of students who may require additional curricular or psychological support. In addition, it provides an opportunity for the faculty to provide additional learning resources to answer the questions raised.

Monitoring: The e-portfolio

A competency-based programme which involves continuous monitoring cannot be based on the conventional log book. It requires, therefore, an electronic record linked with the learning management system of the institute which captures all activities of the residents in addition to demographic data obtained at the time of their admission and a unique ID number. The residents record their daily activities on a day-to-day basis and reflect upon their learning which will be monitored and assessed by the faculty on a weekly basis. This e-portfolio includes postings including extra-departmental rotations with dates, a patient log of cases seen on a daily basis, tasks performed in patient care including operative and procedural work along with activities such as treating patients under supervision and independently, clinical governance and audit, awards and recognitions, teaching–learning activities, academic presentations, critical incidents with reflection on the same, research, academic publications, training courses attended, community/outreach activities and extracurricular activities. All these are recorded electronically. A unique Wiki-based portfolio has been developed by SBV for this purpose. The portfolio is a confidential document only open to the mentor and departmental faculty. On successful completion of the course, it can be sealed by the candidate to be opened only on his/her request for testimonials, etc.

The whole purpose of the e-portfolio and reflective learning is to provide regular feedback to the candidates as mentioned under the section on milestones, identify those with issues in professionalism based on MSF and call them for personal confidential counselling, identify those who are lagging behind on milestones, discuss problems with them and, for those with learning issues, provide additional learning experiences such as additional opportunities to assist in procedures and practice in skill laboratory. The ultimate aim is to ensure at the penultimate milestone assessment that all students have reached the acceptable level of performance.

Process, hurdles and pitfalls: The Sri Balaji Vidyapeeth experience

The system, although better than the one which exists at present, is to some extent still subjective and requires education of both faculty and residents. At SBV, in addition to agreeing on a need for a competency-based system, major hurdles along the way included developing an e-portfolio, linking e-feedback to the residents, implementing MSF in the Indian cultural context and understanding the concept of reflective learning.

Detailed planning has to be carried out before the competency system can be implemented. All academic stakeholders must be convinced that no regulatory norm is being violated. This process of preparation took more than 1 year at SBV. The preparatory phase comprises workshops to educate the faculty and students on the competency-based learning system and its merits over the conventional process. It is also required to prepare a detailed list of departmental EPAs and an acceptable format of MSF from various stakeholders. The preparation of EPAs must be participatory and democratic.

Major requirements for a competency-based training programme to succeed in India include buy-in from students and faculty, arranging regular student–faculty meetings and developing a system of reflection and feedback and a reflection on the feedback. Residents' reluctance to spend time daily with the portfolio was overcome by discussing with them at workshops how the system was developed primarily for their benefit to ensure that at the time of going for the summative examinations they have achieved what they came to achieve while joining the course. Students' buy-in is facilitated by mentioning that the main aim of the process is to ensure that all students reach the same acceptable level in the same period and those who lag behind get additional training inputs. Other healthcare workers had to be brought on board by group discussions to explain the need for the process, explain the format for giving feedback and ensuring total confidentiality and anonymity. A major issue initially was faculty resistance since this involved additional work. Multiple faculty development programmes were required to get all faculty members on board. Faculty motivation is helped by building in a performance-linked incentive in their remunerations or official recognition at the annual college day. The duties of the faculty members include identification and definition of competencies, deciding on acceptable level of performance and providing mentoring and feedback. Students' responsibility is to collect and docu-ment evidence, defend accomplishment and reflect and respond to feedback. The faculty mentor should initiate regular interaction, interpret data in making decisions on the need for further intervention, give feedback, plan individualized intervention, coach and support self-regulated learning and be able to monitor reaching of acceptable level of performance.

Monthly planning and monitoring sessions with randomly picked portfolios in the presence of the concerned students, their mentors, educationists and administrators were used to identify lacunae, suggest measures for improvement, identify good e-portfolios and use them as models for educating others and offering incentives for satisfactory performance. After a year of planning and considerable ongoing monitoring on a regular monthly basis, and revision sessions, the system was introduced in 2015 and has been functioning successfully since then. It is an ongoing process and each year the outcome seems better than the previous year.

To the best of our knowledge, SBV is the first institution to formally introduce a competency-based postgraduate programme for residents in India. The programme called CoBaLT^© had to be tailored to the Indian milieu and could not be complemented by a criterion-based summative evaluation process since this is completely under the control of the regulatory body. However, it serves as a good model for India, within regulatory guidelines, enabling a training and intervention process which ensures that all residents are aware of their goals at the beginning of the course and reach the minimal acceptable level of skills at the end of the course with a facility for additional inputs for those who require additional curricular or psychological support and carries the additional benefits of continuous mentoring and monitoring and an introduction to the reflective learning process.

It is not offered as the only model for the Indian scenario, rather as an example of one which has been introduced, which is in a process of continuous development and improvement for other institutions to adopt or modify it, or prepare another model as per their requirements with the aim to better train postgraduates in medicine to ensure competency.

Conflicts of interest. None declared