Measure for Measure

Bill and I are used to journalists calling us data nerds. Sometimes it’s meant as a compliment, sometimes it’s not, but we always take it in a positive way. At Microsoft, we analyzed mountains of data to make the best possible business decisions. We have tried to bring the same culture to our foundation. However, we’ve had to come to terms with the fact that in certain areas of global health, the data just aren’t very good.

My primary focus is women and children’s health. I’m enthusiastic when I look at child health statistics because child mortality is declining steadily. When I look at maternal health statistics, though, it’s more frustrating. For years, the number of maternal deaths worldwide remained relatively constant at just above 500,000 per year. In 2005, a big World Health Organization study put it at 536,000. It was agonizing watching the numbers hold steady. The Millennium Development Goals (MDGs) had set a very ambitious target of a 75 percent reduction in maternal deaths by 2015 (compared to the 1990 figure). The problem wasn’t just that we were going to fall short of the target. It was that we weren’t making any progress toward it whatsoever.

Then, in 2010, a new study in The Lancet put the number of maternal deaths at 342,000. The change reflected a new and better methodology for doing the estimates. According to more up-to-date statistical techniques, we’d been making gradual progress on maternal health for years; we just hadn’t been able to measure it. Now, there is consensus in the field that maternal mortality is down by about a third in the past three decades — well short of the MDG target, but much better than total stagnation.

Obviously, everyone is pleased with the newer estimates. Fewer mothers dying is the goal. But the data are still far from perfect, and our ongoing difficulty in accurately measuring maternal mortality also makes it harder for us to know how best to prevent it.

In his annual letter, which came out Wednesday, Bill explains "how important measurement is to improving the human condition." When we set a clear goal, intervene in the ways we think will best help us achieve the goal, and then measure the impact of the intervention as we go, Bill argues, we essentially have a report card. We can see which interventions work and which don’t, and we can keep doing what works and fix what’s not working. When measurement is public, as it is with the MDGs, countries have a strong incentive to focus on important global health goals. No country wants to be in last place on the list of maternal deaths.

But if we don’t know how many mothers are dying, where, or why, then we have to make educated guesses about which interventions to pursue.  And if we can’t measure the impact of those interventions, then we can’t identify best practices in a rigorous way.

The poor maternal health data is not for lack of trying. Fundamentally, thankfully, maternal mortality is a rare event, so the sample sizes are low. It’s also systematically underreported. Many countries with high maternal mortality rates also have poor systems of registering deaths. As a result, we have to produce estimates using multiple data sets and complicated statistical modeling, but the confidence intervals are large enough to belie the use of the word confidence.

In response to these problems, the maternal health community has pioneered innovative survey techniques to obtain more accurate mortality data. One of these techniques is a relatively low-cost survey called the "sisterhood method," which involves asking women questions about the health status of their siblings. During these interviews, participants often recall deaths that occurred a decade or longer before, providing surveyors with valuable historical data. Eventually, the goal is to replace these surveys with better registration and certification of deaths, or where this is not possible, maternal mortality estimates based on census data.

Now, we are getting better at using proxies for maternal mortality — basically counting important steps toward better maternal health that are easier to count, like how many women give birth with a skilled attendant. The field needs to focus on understanding which proxies are most reliable. Figuring out how many women have access to prenatal care, to take another example, is measurable, and if we can establish how that’s correlated to better health for mothers and their children, then we can track prenatal care access in real time to help inform our decision-making in the short and medium term.

One possible proxy is the number of women giving birth at health facilities instead of in the home, and the data show that this number is going up globally. There is a good reason for health systems to encourage facility births. Research shows that it is virtually impossible to predict obstetric emergencies. For example, numerically, more women without risk factors have atonic postpartum hemorrhage compared to those with risk factors.

Any woman giving birth is at risk of serious complications. Since mothers with quick access to expert, high-quality care are more likely to survive emergency situations, it makes sense to ensure that mothers have access to that kind of care.

Still, recent data suggest that in poor countries the quality of care in many health facilities is so low that not enough women, especially the most disadvantaged, are getting the care they need. As more women choose to give birth in facilities, the issue of improving the quality of care is and should be a top priority for maternal and child health advocates.

The tragedy of women dying uncounted — their invisibility to members of the global health research community — is a reflection of their invisibility in the societies they call home. The lives — and deaths — of women and children often go unseen.

But I see signs that women themselves are starting to fight against this invisibility. When I talk to women in places like Kenya about their desire to plan the number of children they have, I hear evidence that they’re starting to believe they can and should have power over their futures and the future of their families. When I talk to women in self-help groups, I hear evidence that they’re finding powerful role models among their peers.

In Bill’s letter, he tells the story of Sebsebila Nassir, an Ethiopian mother who recently gave birth to her second daughter. When her first daughter was born several years ago, Sebsebila followed the Ethiopian custom of waiting to give her a name, for fear of becoming attached to a child who might not survive her first month. But because the Ethiopian health system is improving, mothers are starting to abandon the tradition. Sebsebila named her daughter Amira on her birthday. It’s a powerful story about a revolution in child survival and the commitment and innovation of the Ethiopian government. It’s also the story of one mother’s growing sense of empowerment and optimism.

Measurement in the field of maternal health will continue to be a massive challenge. The recent, lower estimates of maternal mortality show that years of work by dedicated maternal health advocates and practitioners had a greater impact than we understood. They also show that poor women themselves are full of ambition and have been doing the hard work of saving their own lives. If we invest in better measurement, we will get a lot more impact out of all that courageous effort.