Report: Nations exaggerated numbers of vaccines in Gates-funded program

Microsoft co-founder Bill Gates has described a program to boost vaccination of kids around the world as his best investment. But a new analysis — by another Gates-funded organization — says only about half as many additional children as reported got the shots.

The vaccine program pays poor nations $20 for every new child immunized, and that financial incentive has apparently led dozens of countries to exaggerate their statistics, says Christopher Murray, director of the University of Washington's Institute for Health Metrics and Evaluation.

Instead of reaching nearly 14 million additional children, as estimated by the World Health Organization (WHO), the study by Murray and his colleagues concludes about 7.4 million additional children were actually vaccinated through 2006.

Official estimates claim 90 percent of children in the developing world were getting the DTP vaccine against diphtheria, tetanus and whooping cough by the end of 2006. The Seattle study agrees coverage has increased steadily, but only to 74 percent.

A standard vaccine for infants, DTP is considered a yardstick of overall vaccine coverage.

A few countries, including Pakistan, appear to have inflated their numbers by a factor of four or more. Many others, including the African nations of Niger and Mali, reported twice the number of actual vaccinations, the paper says. But in Uganda, Sierra Leone, Cambodia and several other countries, vaccination numbers were accurate or even underestimated.

"The good news is that in a lot of countries, their systems seem to be working," Murray said. "The bad news is that there is quite a lot of over-reporting, and it does seem to be related to the presence of performance-based payments."

If the study results are correct, it means governments in developing countries were overpaid by about $140 million.

Even before its publication today in the British medical journal The Lancet, the study touched off consternation in the global-health community. Not everyone agrees with Murray's conclusions, and some experts fear the critique will undermine support for childhood vaccines.

A special meeting was convened in London partly to discuss the results. Murray sent an advance copy of the study and a note to Bill and Melinda Gates, to alert them of the impending blow to the GAVI Alliance — one of their first and favorite global-health efforts.

Formerly the Global Alliance for Vaccines and Immunization, the GAVI Alliance was launched in 1999 largely due to a push by the Bill & Melinda Gates Foundation to reinvigorate stagnant vaccine programs.

The Gates Foundation committed $1.5 billion to the organization, and donations from wealthy countries have pushed total funding to more than $3 billion. GAVI estimates its multiple programs have saved more than 2.8 million lives in the developing world.

"Overwhelmingly, GAVI is a success," said spokesman Jeffrey Rowland.

Most of GAVI's money goes to buy vaccines that are distributed to poor countries. The incentive program, which accounts for about 11 percent of spending, rewards countries for expanding vaccination and allows them to use the money to improve health care, Rowland said.

"It's performance-based, which is the new way of doing aid programs."

GAVI does require independent audits of vaccine data from countries but nevertheless announced it would suspend the incentive payments until it can fully evaluate the data.

Murray's study compared government data from 193 countries with door-to-door surveys conducted as part of general health evaluations. Conducted by independent contractors, the surveys are less likely to be biased, he argues, because they question mothers directly and reach even the poorest households. They are also removed from the government bureaucracy and pressure to meet vaccine targets.

"I think our estimates are the best that are out there," Murray said.

The gap between official figures and the household surveys widened after GAVI began paying the per-child fee. The study does not accuse governments of deliberate fraud, but subtle pressures throughout a health system can lead to unintentional inflation, Murray said.

Even data from household surveys can be fraught with error, cautioned David Bishai, of Johns Hopkins Bloomberg School of Public Health, whose commentary on the study also appears in the Lancet.

"It's not a gold standard," he said.

Some of Murray's conclusions are based on a selection of "worst-case" countries, which may not represent what's going on everywhere, Bishai said. He's also worried that the Gates Foundation and other organizations could read the study as a broad indictment of GAVI's work and pull their money.

"You could end up throwing out the baby with the bath water," Bishai said. "There is a good-news story lurking underneath, and that is: Vaccine coverage is up."

Ethiopian Health Minister Dr. Tedros Adhanom vehemently denied his country had inflated its vaccination rates, officially reported as about 60 percent. Murray's study said the number may be closer to 30 percent.

Two independent surveys, both conducted door to door — and one of which sampled nearly 1 million people — came up with results very close to the official numbers, Adhanom said. Yet Murray's study ignored that data, relying instead on a survey that sampled only about 2,000 households.

"Thirty percent is very, very wrong," said Adhanom, who serves on the Institute for Health Metrics and Evaluation board. "I have been to the villages. Perhaps the people who wrote this report never have."

The Gates Foundation declined to comment on the study. But in a Monday board meeting at the institute, the foundation's global-health chief Tachi Yamada pointed out ways Murray could put a more positive spin on the report. Yamada also grumbled a bit about the institute's penchant for "sensationalistic" analyses that expose flaws in health programs.

But the Gates Foundation has vowed to judge its programs based on results. That's the main reason the foundation lured Murray away from Harvard and put up $105 million for the institute. Its missions include rooting out bogus health statistics and evaluating whether the enormous sums being poured into global-health programs are making a difference.

"They are being paid to be the cop, and that is what they're doing," Bishai said.

GAVI spokesman Rowland and a representative of the WHO, which compiles vaccine data, said they both agree data quality need to be improved.

Murray said that's his goal, too.

"By getting on top of this issue, it's very feasible for GAVI to end up with an even better design, and even better results."

Sandi Doughton: 206-464-2491 or sdoughton@seattletimes.com