The checklist manifesto revisited

Semrau KEA, Hirschhorn LR, Delaney MM, Singh VP, Saurastri R, Sharma N, Tuller DE, Firestone R, Lipsitz S, Dhingra-Kumar N, Kodkany BS, Kumar V, Gawande AA, for the BetterBirth Trial Group. (Ariadne Labs-Brigham and Women’s Hospital, Harvard T.H. Chan School of Public Health, Divisions of Global Health Equity and General Internal Medicine and the Department of Surgery, Brigham and Women’s Hospital, Department of Medicine, Harvard Medical School and the Department of Health Policy and Management, Harvard T.H. Chan School of Public Health—all in Boston, USA; Feinberg School of Medicine, Northwestern University, Chicago, USA; Population Services International and the Community Empowerment Lab, Lucknow, and Jawaharlal Nehru Medical College, Belgaum—all in India; Population Services International, Washington, DC, USA; and the World Health Organization, Geneva, Switzerland.) Outcomes of a coaching-based WHO Safe Childbirth Checklist Program in India. N Engl J Med 2017;377:2313-24.

Summary

This was a matched-pair, cluster-randomized, controlled trial investigating the impact of implementing WHO’s Safe Childbirth Checklist (SCC) versus the current practice at 120 government (primary health centres, community health centres and first referral units) health facilities, conducting more than 1000 births annually, in 24 districts of Uttar Pradesh, India. The SCC is a list of 28 evidence-based birth practices broken up into four pause points: on admission, during maternal pushing, within 1 hour post-delivery and at discharge. The intervention was rolled out through the Better Birth programme^[1] which utilises local leadership engagement and longitudinal peer coaching for checklist use and resolving barriers within the existing health system. After engagement of facility leadership, the SCC was introduced to facility birth attendants during a 2-day orientation followed by twice-weekly nurse coaching visits which were tapered to monthly over an 8-month period. The study enrolment began 2 months after the intervention with 157 689 women enrolled overall. Data were collected via birth registers, follow-up phone calls and home visits as necessary, with 99.6% of the enrolled participants reporting 7-day outcomes. Audits as part of the study’s data quality assurance system reported 98.3% accuracy of the collected data. The primary composite outcome included stillbirth, early neonatal death, maternal death and severe maternal complications within 7 days of delivery. Adherence to the SCC was measured via independent nurse observers at selected study sites at 2-month at 12-month time points. SCC adherence was significantly higher in the intervention group compared with the control group (72.8% v. 41.7%) at 2 months and (61.7% v. 43.9%) at 12 months, yet significant difference was observed neither in combined maternal and neonatal mortality as well as severe maternal complications and the primary composite outcome (15.1% v. 15.3 %), nor secondary maternal and perinatal adverse outcomes.

Comment

The senior author of this paper has an extremely popular book titled The checklist manifesto, in which he shows how a simple list of items makes complex procedures such as surgeries, aviation or skyscraper construction safer than ever before. He says there that, despite all of our expert knowledge, resources and efforts, we have innate human inadequacies in dealing with complexities in care packages that have many steps or involve many people and lead inevitably to ‘errors of ineptitude’.^[2] Thus, even the most skilled and well-resourced surgeon will occasionally forget a key step or fail to plan for a certain eventuality. Subsequent evidence has supported his theory. A WHO surgical checklist has been shown to consistently reduce perioperative mortality in a variety of settings, and this strategy has now permeated other areas of healthcare delivery with some success.^[3],[4],[5] Meanwhile, maternal and neonatal mortality rates have improved in low- and middle-income countries though not to the degree one would expect as facility-based birthing rates have increased and strategies for further improvement have been elusive.^[6] Could improvements in evidence-based birthing practices using the checklist strategy lead to reduced perinatal mortality and morbidity? That is what the study authors set out to investigate in Uttar Pradesh.

The Better Birth study implemented the SCC through a peer coaching model rolled out over an 8-month period. The adherence to practices that were being ensured by this checklist improved substantially in the interventional arm, but did not lead to improved outcomes such as better maternal or child survival or reductions in severe maternal complications. This finding may seem disappointing, yet not altogether surprising. The efficacy of the checklist model relies on the user(s) having adequate knowledge, skills, professional support and resources to act; in this case, emergency obstetric/newborn care to save the life of a mother and/or child. Yet, it was rolled out in a region where the most basic necessities such as water, electricity, routine medications and adequate personnel are often unavailable at the time of birth.^[7] Even with adequate supplies, the skilled birth attendants (SBA) providing care in this trial may not have had the skills, support and referral systems necessary to intervene in a timely and effective manner.^[8] Only 14% of the observed births in this study were attended by physicians; yet in this and similar regions, non-physician SBAs frequently do not have the training to perform life-saving procedures.^[9] The overall caesarean section rate (listed as <2% in both groups at primary facility sites, though likely somewhat higher due to intrapartum referral) reflects what the Lancet series on Global Maternal and Newborn Health has termed ‘Too Little Too Late’^[10] as does the low blood transfusion rate (0.8%) in the face of frequent maternal haemorrhage (7.4%).

Clinical procedure checklists, like the SCC, can be a useful marker of processes in birth practices. However, it has obvious limitations in assessing both the quality and timeliness of the interventions provided. For example, the use of a partogram is a strong recommendation by the WHO despite limited evidence and is one of the 18 measures in this study.^[11] However, ‘partography started’ does not indicate whether this tool was used effectively to identify dystocia and intervene appropriately. Were patients with pregnancy-induced hypertension treated with antihypertensive therapy when indicated? Were the appropriate medications available? We believe that a checklist can only be part of a larger package of interventions that could improve survival. Birth attendants must be effectively trained to not only employ a checklist, but also understand how to interpret findings and intervene in a timely and appropriate manner. In the era of increased task-shifting in primary care, physicians must take ownership as team leaders in these settings and push for greater adherence to evidence-based practices. One must also acknowledge that determinants of perinatal outcomes are operating well before the actual moment of conception. Adequate nutrition, sex equity, income level, education status and safe, healthy home environment are basic determinants that would independently affect the survival of mothers and their newborns through the stress of birth.^[12],[13] The high rates of preterm birth and low birth weight in the study (~21% and 28%, respectively) raise concerns about the underlying maternal health status as well as the quality of antenatal services mothers are receiving.

Finally, we feel that there were some weaknesses in the implementation of the intervention and assessment of impacts. While the peer coaching model did have some success in changing practice behaviour, we believe greater adherence could have been achieved with a more sustained, consistent coaching in the study facilities. Further, the percentage adherence outcome loses significance when it does not take into account the differential impact of the checklist items. Having a birth companion present at admission and at the time of birth achieves two points on the checklist scale, but we would argue that it is less likely to impact outcomes than measuring maternal temperature. In addition, tracking outcomes only up to 7 days following birth does not capture the full picture of postpartum maternal complications. The high rates of puerperal sepsis (5% in both study arms) perhaps suggest further mortality beyond the 7-day follow-up period. Finally, the effort to accurately capture checklist adherence was laudable but limited by the Hawthorne effect, the number of births witnessed (2% of the total), the limited number of facilities assessed and particularly the lack of any nocturnal data, a time period which frequently includes further limitations in the available resources and different staff members from the day. This does bring into question the internal validity of the adherence data, which has been trumpeted as clear evidence that birth practices can be improved.

The study authors should be commended for completing one of the largest such studies in a low-resource region while also developing a data quality assurance system which can act as a model for future studies in similar settings. It also provides useful data not directly related to the hypothesis being explored. For instance, in both the interventional and control arms, the maternal mortality rate within 7 days of delivery was 100 deaths per 100 000 births, despite the setting in a high-volume, health facility setting. Other emerging economies such as Russia, China and Brazil all report rates <50 deaths per 100 000 births as of 2015 WHO data, while European countries are commonly <10 deaths per 100 000 births.^[14] The neonatal mortality rate of 30 per 1000 live births is similar to the reported national rate though the national figures include all deaths within 28 days of birth.^[15] These data further indicate the gaps in access and quality of services in the public sector in India, particularly at the primary care level. It is ironic to see these results shortly before the Government of India announced a massive investment in health spending for secondary and tertiary care services, while primary care remains underfunded.

The coaching-based Better Birth intervention did result in considerably higher rates of evidence-based birth practices though the level of adherence decreased over time. However, it did not positively impact clinical outcomes. To do so is likely to require substantial organization, resources and investment in skills.

Conflicts of interest. None declared