Here's one more showing WHY the study was so flawed:
Counting Iraqi War Casualties
S. Robert Lichter
How reliable is a new study claiming 100,000 deaths from the U.S. invasion?
A controversial new study claims that the invasion of Iraq was responsible for far more deaths than heretofore recognized. The study was released ahead of schedule, in the final days of the American presidential campaign, by The Lancet, a British medical journal. Media accounts have focused on the researchers' conclusions that:
- "the death toll associated with the invasion and occupation of Iraq is probably about 100,000 people, and may be much higher"
- "violence accounted for most of the excess deaths, and air strikes from coalition forces accounted for most violent deaths"
- "the risk of death from violence ... was 58 times higher" after the invasion than it was before.
Does the study warrant the attention it is receiving? More importantly, do the findings warrant the authors' conclusion that their results "demand a re-evaluation of the consequences of weaponry now used by coalition forces..." not to mention the Lancet's editor's broader conclusion that, "Democratic imperialism has led to more deaths, not fewer"?
To be sure, the researchers showed ingenuity and courage in collecting public health information in Iraq at a time of great violence and social disruption. Their use of survey methods is largely impeccable given the limitations under which they had to work. Unfortunately, they are on shakier ground when they use their sample results to estimate the number of deaths and increased risk of violent death associated with the invasion and occupation.
Their method was to estimate the death rate before and after the invasion and compare the results. To do this they surveyed 988 households in 33 randomly selected neighborhoods (or "clusters"), asking residents about any deaths since January 1, 2002. Then they tabulated the number of deaths reported prior to the invasion on March 19, 2003, and compared the total to the number of deaths reported after that date. (They added a statistical correction factor to take into account the greater length of the post-invasion period.)
Some limitations of this method are unavoidable. First, the researchers assume that people were able to accurately recall information about the timings of deaths over the past three years. Second, the "cluster sampling" method made necessary by the continuing unrest in Iraq increased the chance of selecting unrepresentative neighborhoods that skewed the findings. This may have happened in the Falluja cluster, which represented 3 percent of the sample but accounted for 71 percent of all post-attack violent deaths. The researchers recognized this problem and excluded "this extreme statistical outlier" from their calculation of excess deaths.
More important was the way the researchers chose to determine whether reported deaths were associated with the war and occupation. They used the pre-invasion death rate as a baseline and assigned responsibility to the conflict for any increase in deaths from all causes after that date: "we estimated the death toll associated with the conflict by subtracting pre-invasion mortality from post-invasion mortality and multiplying that rate by the estimated population of Iraq....") From this they concluded, "the mortality rate was higher across the Iraq after the war than before, even excluding Falluja. We estimate that there were 98,000 extra deaths... during the postwar period."(Notwithstanding their wording "after the war" and "postwar," these figures included deaths that occurred during the invasion.) The margin of sampling error surrounding this estimate is quite large, ranging from 8,000 to 194,000 deaths.
The crucial assumption is that any increase in deaths after the invasion began on March 19, 2003 is associated with the conflict and subsequent occupation, to the exclusion of any other factor. Specifically, their sample included 46 reported pre-invasion deaths, only one of which was violent, and 89 post-invasion deaths outside Falluja, 21 of which were violent. According to a table that breaks down the causes of death, fewer than half of the "excess deaths" (45 percent) resulted from violence. One in five was accidental, one in six was due to heart attack or stroke, just under one in 10 was caused by infectious disease, and the same proportion consisted of neonatal or infant deaths. Yet all these deaths without exception were attributed to the war and occupation.
The researchers noted that their overall totals include "12 violent deaths not attributed to coalition forces," including seven murders, two of unknown origin, two from anti-coalition forces, and one "from the previous régime during the invasion." Because this listing includes violent deaths from Falluja, we cannot say how many were among the 20 "excess deaths" from the other 32 clusters. It is troubling, however, that there was only one reported violent death during the entire pre-invasion period. Wen a single instance represents the sum total of violent deaths committed by criminals, "the previous régime," and all unknown parties during the 14 months prior to the invasion, it suggests the instability of the data, the probability of increased measurement error, and the danger of calculating changes in mortality rates from such a low base.
Yet that is precisely what the researchers did in estimating the proportion of excess deaths and increased risk of death attributable to violence. In fact, they upped the ante by including the figures from Falluja in these calculations, after excluding it from their estimate of 100,000 excess deaths for being an "extreme statistical outlier". In the 97 percent of the country sampled apart from Falluja, it is not true that "most of the excess deaths" were due to violence. Similarly, their conclusion that the rate of death from violence was "58 times higher" after the invasion was based on comparing the one pre-invasion violent death with 52 post-invasion deaths in the Falluja neighborhood they sampled, in addition to the 21 deaths in the other 32 neighborhoods throughout the country.
Thus, it is highly problematic to attribute so many violent deaths and such a high risk of death throughout Iraq to totals that depend so heavily on a single neighborhood out of 33 sampled. But it is also problematic to blame the war and occupation for all increases in fatal accidents, heart attacks, infectious diseases, and neonatal and infant mortality that occurred since the invasion. Even if the sampling was flawless and the self-reports entirely accurate, the total number of reported deaths should be the starting point for estimating conflict-related mortality not the finish line.
Dr. Robert Lichter is President of the Statistical Assessment Service
http://www.stats.org/record.jsp?type=news&ID=481