Cooking the Iraq body count books
Francis Sedgemore, Thursday 17 June 2010 at 12:27 UTC
Listen to any ‘Stop the War’ spokesman, and you’ll likely hear quoted a number representing the excess mortality count in Iraq since the multinational military invasion and removal of the Ba’athist dictatorship. By excess mortalities we mean deaths resulting from the invasion and occupation, and subsequent terrorist activity carried out by those who refuse to recognise the authority of the elected Iraqi government.
The number of excess deaths quoted by political activists opposed to the invasion of Iraq is based largely on a study published in 2006 in the medical journal The Lancet. In this highly controversial paper, Gilbert Burnham, Riyadh Lafta, Shannon Doocy and Les Roberts claimed, based on a door-to-door survey, that over 650,000 people had died as a consequence of the war, of which some 600,000 suffered violent deaths. Add to this the statistical uncertainty, and the grand total is around a million. As you might expect, if you are of a cynical disposition, one million is the figure quoted by innumerates with a political agenda as the actual number of excess deaths due to ‘western imperialism’.
Burnham et al.‘s findings have been challenged on a number of fronts, yet owing to repeated quoting in popular news media, the one million excess deaths figure is widely accepted outside the scientific community. Among experts there is a general acknowledgement that the number of excess deaths exceeds 100,000, but is nowhere near the total presented by Burnham and his colleagues and quoted with monotonous regularity by political commentators.
The latest contribution to the debate surrounding the number of war-related deaths in Iraq comes from the economist Michael Spagat, who in a paper in the journal Defence and Peace Economics questions the ethical and data integrity of the Lancet study, presents evidence of data fabrication and falsification in that work, and calls for an investigation. Spagat’s conclusions are damning, and his criticisms of Burnham and his colleagues go way beyond previous accusations of ‘main street bias’, which results in an over-estimation of a measured quantity through sampling only the more accessible urban locations within a polled area.
In addition to the Burnham et al. study we have a survey carried out by British opinion pollsters ORB, which in 2007 published an estimate of 1.2 million (±2.5%) excess deaths in Iraq. That figure was later revised downward to one million, but it remains an order of magnitude greater than the number of excess deaths accepted by the majority of experts. Michael Spagat, along with Josh Dougherty of Iraq Body Count, an organisation which has regularly published estimates of Iraqi deaths due to the war, looked in detail at the ORB survey, and found serious irregularities in its mortality data, and systematic errors that render those data worthless.
Spagat and Dougherty’s dissection of the body count over-estimates was discussed last week by Independent columnist John Rentoul, for whom the scientifically credible number of excess deaths should be enough for the ‘stoppers’ to make their case without having to exaggerate the count by a whole order of magnitude. Indeed, but Rentoul would do well to acknowledge that his own trade of newspaper punditry is no stranger to the use of hyperbole in the service of political rhetoric and vulgar point-scoring.
Further reading
Burnham et al., “Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey”, The Lancet 368, 1421 (2006)
Michael Spagat, “Ethical and data-integrity problems in the second Lancet survey of mortality in Iraq”, Defence and Peace Economics 21, 1 (2010)
ORB opinion poll data on excess deaths in Iraq, (2007)
Michael Spagat & Josh Dougherty, “Conflict Deaths in Iraq: A Methodological Critique of the ORB Survey Estimate”, Survey Research Methods 4, 3 (2010)
Feed the writer! 

Thursday 17 June 2010 at 18:22 UTC
[...] Francis Sedgemore – Cooking the Iraq body count books [...]
Thursday 17 June 2010 at 19:53 UTC
On the other hand we have this:
http://scienceblogs.com/deltoid/2008/02/spagat_goes_off_the_deep_end.php
So who am I, a wandering pedant, to believe?
Friday 18 June 2010 at 09:22 UTC
Tim Lambert’s been on this topic for some time, and he’s clearly familiar with the details. But I’m not altogether sure where Lambert is coming from, as, as far as I can se, he has been associated with those for whom the lower body count figures (~100,000) are the more credible, yet he is also cited by as supporting the Burnham et al. work against that of Iraq Body Count.
What bothers me about Lambert and others who support the Lancet study is the vitriol directed toward Iraq Body Count and others who have questioned the Burnham figures. It’s far more than the robust divergence of views that I experienced as a research scientist working on a particularly controversial topic within my own field of expertise. Such disagreements can very easily become personal, but not like the personal-political attacks we see when it comes to public discussions of excess mortalities in Iraq. Lambert’s blog is a case in point.
As for Lambert’s 2008 objections to Spagat, he lost me with his get-your-retaliation-in-first tactics, which included comments about “absurd assumptions” for main street bias (not absurd; it’s a credible criticism of the Lancet study, and deserves addressing in a grown-up manner, not dismissed with a haughty sniff); and all this stuff on both sides about backers and supposed conflicts of interest.
I’m not sure what to make of Spagat’s allegations of scientific misconduct on the part of Burnham et al., but for Spagat to come out so and call for an official inquiry would suggest that he’s confident there is a case to answer, with further evidence to be presented at the appropriate time. A malicious allegation of fraud could so easily destroy Spagat’s career. This is not like calling a politician a liar.
On the detail of Lambert’s 2008 criticism of Spagat’s work, he’s lost me again. And I’m a former research physicist with much experience of using experimental statistics, and teaching students about the uses and abuses of regression analyses. To be honest I haven’t a clue what Lambert’s on about here; my only problem is the paucity of data with which to carry out a credible regression analysis, linear or otherwise.
Friday 18 June 2010 at 09:52 UTC
I should also point out that Spagat’s 2010 paper (the second in the above list) goes way, way beyond the simple graph objected to by Lambert, and contains detailed criticisms of Burnham et al. that deserve detailed answers.
Spagat’s new paper is a long and difficult read, but is certainly worth reading in full if this subject is of interest to you.