Innumeracy At The Times Or Laziness At The Economist?

Via Glenn we are led to this posting at The Economist site:

Innumeracy, thy name is New York Times reporter

A dire article from the New York Times indicates that infant mortality is rising in the American south, particularly among blacks:

To the shock of Mississippi officials, who in 2004 had seen the infant mortality rate — defined as deaths by the age of 1 year per thousand live births — fall to 9.7, the rate jumped sharply in 2005, to 11.4. The national average in 2003, the last year for which data have been compiled, was 6.9. Smaller rises also occurred in 2005 in Alabama, North Carolina and Tennessee. Louisiana and South Carolina saw rises in 2004 and have not yet reported on 2005...

...65 more dead babies is 65 too many.  But it's a small enough number that one needs to consider things like measurement error--did Mississippi change its criteria for infant mortality?--and random variation before leaping to the conclusion, as the article does, that this is some fundamental sea change in operation.  Certainly, generalising the problem to "the south" on the basis of smaller increases in six other states, which are even more likely to be random variation, seems extreme.

And the author suggests other reason to disregard this story.

To which I say, please - there is innumeracy, and then there is laziness.  It is certainly true that other explanations *might* be in play, but that is hardly the same as demonstrating that they *are* in play.

For example, the author waves his hand (OK, keyboard) and invokes "random variation".  Fair enough - let's consider that.  The Times includes a graphic depicting infant mortality statistics in Mississippi from from 1985 to 2005.  Using eyeballs only, I would say that the current level for black infant deaths, 17 per 1000, has not been seen since the late 80's; the level has remained below (or at) 15 per thousand since about 1993.  If random variation explains the jump to 17, why has it not jumped to 17 (or 13?) at all in the the previous decade?  I have seen noisy time series, and this does not look like one.

But we aren't stopping with a simple eyeballometric estimate!  Here is some infant mortality info from the State of Mississippi website.  For 2004 we have 481 deaths, broken down as 153 white and 328 non-white.

So let's brace ourselves for a vaguely numerate moment here and estimate the standard deviation  for a frequency distribution of 15 per thousand (which we will assume is the "correct" infant mortality rate for non-whites.

328 deaths with an assumed rate of .015 implies total births of 21,867.  And recalling some dusty stats textbooks about binomial approximations, I will announce that that the square root of [0.015*.985)/ 21,867] = .082%; that should be roughly the standard deviation of the mortality rate.

From which we further estimate that 95% of the results should fall within a range of
plus of minus 2 standard deviations, or 13.4 to 16.6 percent.  Which means that the result on offer, 17, is outside the range we might expect from random variation.

Now, clearly there is a news-selection bias here - the Times would not be reporting on this this if the result had been closer to the expected level, so maybe this is just bad luck.  On the other hand, the Times does suggest other causes, such as cutbacks in Medicaid:

The setbacks have raised questions about the impact of cuts in welfare and Medicaid and of poor access to doctors, and, many doctors say, the growing epidemics of obesity, diabetes and hypertension among potential mothers, some of whom tip the scales here at 300 to 400 pounds.

However, the Times is being quite coy with their "questions have been raised" formulation  vis a vis Medicaid.  Haley Barbour, Governor of Mississippi, tried to cut Medicaid benefits, mainly to the elderly, but seems to have been blocked until July 2005:

July 2, 2005

In Mississippi, Soaring Costs Force Deep Medicaid Cuts

 

By

SHAILA DEWAN

HAZLEHURST, Miss., July 1 - Starting Friday, most Medicaid recipients in Mississippi will be limited to five prescription drugs at a time, with no process for appeal. The cap appears to be the most restrictive in the nation, but is just one of many measures being taken by states seeking to rein in soaring Medicaid costs.

...

In Mississippi, where more than a quarter of the population is on Medicaid, the cap includes a limit of two name-brand drugs. The only patients exempt from the rule are children, people in nursing homes and patients with H.I.V., who were given an 11th-hour reprieve because virtually none of the anti-viral drugs used were available in generic form.

Gov. Haley Barbour, a Republican who backed steeper cuts to Medicaid than those enacted, said the Legislature had come up with the limit on prescription drugs on its own. But, he added, "states are limited in their options as far as cost control."

The cost of Medicaid, financed jointly by the federal government and the states, has risen 63 percent in five years and is now more than $300 billion a year. Governors from both parties have asked the federal government to allow them more latitude in restricting Medicaid spending.

Advocates for the ill and the disabled in Mississippi say many patients require more medicine than the new restrictions allow. Others who could be affected include diabetics, the mentally ill and cancer patients. Of the 768,000 Medicaid patients in 2004, 80,000 used more than five prescriptions a month.

Nothing about children or mothers.  Baffling.

Left unmentioned by the Times is the possibility of a Katrina effect in the 2005 reporting - infant deaths in the last third of the year may have spiked due to Katrina-related disruptions in pre- or post-natal care.  Yes, that would have to be quite a spike - if deaths were at a normal rate for two/thirds of the year, they would have to rise from 15 to 21 per thousand over the last third  to produce an annual average of 17.  Officially, the answer is that there were no Katrina-related infant deaths. None?  Not one expectant or new mother missed an appointment due to storm dislocations, and later had her baby die?  Extraordinary.  Sort of contradictory to this story, too.

I am not going to be able to deliver a Big Finish here.  I very much dislike the hand-waving "random variation" suggestion, but the Medicaid cuts theory hinted at in the Times does not grab me either.

As to the situation in other states, my annoyance with The Economist remains high.  We are presented with this:

I presume, because they are not mentioned, that Georgia, Virginia, West Virginia, Florida, Kentucky, and Arkansas did not see increases in infant mortality.

Geez.  I presume that when the Times says that

Smaller rises also occurred in 2005 in Alabama, North Carolina and Tennessee. Louisiana and South Carolina saw rises in 2004 and have not yet reported on 2005.

they are leaving open the possibility that Georgia has not yet reported figures for 2005. 

And that would seem to be the case, if the Georgia website is a useful guide.  However, the figures for Georgia for 2003 and 2004 show changes of only +/- 0.1 per thousand.

On my scorecard, the Times has opened a discussion on a legitimate story (and I bet the reporter and the editor would be the first to admit that their story raises more questions than it answers).  As debunkings go, The Economist fell short of the mark.  And I'll rank this post as somewhere in between - I have a lot more sympathy for the confusion at the Times, and I don't have any answers either, so fans of the economist won't be hanging their heads in shame either.

Bother.  The Times story irked me yesterday and The Economist post irked me today, but I am nowhere.  It's great to be back.

Comments

[clarice]

Great to have you back even if it's only for a numbers game.

[JJ]

As usual with these stories, you have to scratch head and wonder about the whats first and then that leads to the whys.

Legitimate story? What? Why?

Or is this just a rabbit? Chase the rabbit. Ignore the rabbit? Instead could someone explain to the RNC that fiscal spending a la Reagan needs to make a return before any more elections happen. A more compassionate conservativism might be to act fiscally efficient.

[Appalled Moderate]

1. Look at comment #4 at the economist posting, which suggests something more than standard deviation fluctuation going on in Mississippi's statistics.

2. The NYT's story is hugely annoying as they conflate Mississippi with the rest of the south, but give us nothing regarding the rest of the south that's hard numbers, or an explanation of what may be going on in Alabama and South Carolina. This is pretty much typical of the NYT's reporting of the South, which does not recognize that the several states of the south are actually quite distinct from each other.

[JJ]

And while I am shaking index finger at the RNC...

Should it be the "War in Iraq"? The "War on Terror"?

Or should it be the "War on Terrorism in Iraq"?

Or should it be the "War on Terrorists Wherever They Operate"?

Better, how about: "The War for Democratic Government in Iraq" or something in a similar line.

Be nice to not have to think about these rabbits that the DNC rabbit farm is constantly releasing. Come on, think up some counter labels! Make them chase...

[RogerA]

I am attaching this link to the United Health Foundation's coverage of infant mortality: http://www.unitedhealthfoundation.org/shr2005/components/infantmortality.html

It gives a very different picture of infant mortality.

From a statistical point of view, one single figure in a trend line running 20 years may or may not have import. The usual statistic by which to check it is a "chi squared" test. However, even that is a bit misleading, and a more sophisticated test of trend lines is done using monte carlo simulations where possible outliers are "smoothed." For any statistics maschists out there (both of you) google "jointpoint" and look for regression analysis.

Another aspect of this whole discussion is the definition of infant mortality which is all infant deaths in the first year of life. About 2/3 of those are attributable to congenital defects and correlate most positively with low birthweight; the other 1/3 could include SIDS or even accidents.

Basically, by pulling one statistic out of a trend line over 20 years that has consistent trended down, the NYT is telling us they havent got a clue. The innumeracy of the journalists (silence of the lambs?) is profound, but, after all, they are journalism majors, who ultimately are art history majors that couldnt hack it. (apologies to all art history majors)

Finally, as you rightly point out, the times piece is just the lede for a story on that nasty ole republican haley barbour and his attacks on abortion and medicare.

Please do look at the link. It will make you feel much better about the health care system's effort to reduce infant mortality in the United States.

Sorry for the long post, but this issue of infant mortality is an important one to me, and I do happen to also be an epidemiologist.

[Rick Ballard]

"About 2/3 of those are attributable to congenital defects and correlate most positively with low birthweight;

Which, not so oddly enough, is supported by additional Mississippi statistics which also include the prenatal medical care numbers that don't really support a contention of medical neglect.

The incidence of low birth rate was 26.7% for whites and 42.4% for nonwhites while the incidence of prenatal care begun by the second trimester care was 93.7% for both groups. There is a disparity in prenatal care beginning in the first trimester versus the second but you have to seek care in order to receive it and it sure doesn't look like any denial of care is occurring.

[RogerA]

AAARRGGHH--Change 'chi squared" to "chi square." BTW, the graphic in the times piece is a classic example of a axis scale chosen to exagerate the effect of a relatively small change of a trend. Plot those same figure on an axis divided in tens from zero to 100 on the y axis and expand the x axis and see what those same trend lines look like.

[cathyf]

What sort of questions do I wonder about?

1) Jiggling statistics by moving deaths from the late-miscarriage column to the infant-mortality column. When a preemie dies, there is a judgement call. A 30-weeker who dies at 8 months old is clearly infant mortality. An 18-weeker who lives for 12 hours after birth is a miscarriage. The line is somewhere between them -- where, exactly? In the environment where there is a huge very active very acrimonious political dispute as to whether a fetus in the second trimester is a baby or a clump of tissue, there are points to be scored in how you classify the 2nd-trimester fetuses who die of natural causes. And in southern states, if there is any change to the classification rules, I expect that it would be a change in the direction of re-classifying these dead as "infants." I have no evidence that anything like this change has happened in any of those states, but it is the first question that I would ask.

2) In first world countries, infant mortality is really quite rare, and is associated with the few remaining intractable problems. Like genetic defects, which rise as a function of maternal, and to a lesser extent paternal, age, and so you would expect a gradual rise related to the long-term trend of older parents. So I wouldn't expect that to be a factor in a sudden one-year change. But their may be a "Katrina effect" where refugee babies with serious congenital problems were less likely to get diagnosed because of the general chaos, or if they were diagnosed there were lots of things to worry about and so non-fatal complications might have turned fatal, or turned fatal earlier (and made it as an "infant" death rather than the death of an older child.)

3) But the question I would look at quite closely are the other first-world intractable causes of infant deaths: failure to breastfeed (approx doubles the probability of death in the first year), exposure to cigarette smoke, stomach sleeping, failure to vaccinate, car accidents, especially with missing or misinstalled car seats. Every single one of those causes of death is something I would expect to have been aggravated by a natural disaster like Katrina. Mothers who decided to risk formula-feeding in a first-world environment where it only doubles the risk of death for their babies all of a sudden found their families in a crisis where clean water and refrigeration were hard to find -- and it was way too late to change their minds. Families where the smokers take all sorts of actions to minimize the amount of smoke that their babies are exposed to suddenly find themselves crammed into a single hotel room, or camping out with friends or relatives with much more important things to worry about than being OCD about smoking around the baby, and many more things making them need a smoke. They may be improvising sleeping accomodations, with babies sleeping on air mattresses or waterbeds, or couches, or bassinets that they are too big for, or left to sleep alone on big beds. The Katrina refugees who were far from their pediatricians didn't have the support of making a quick phone call or bopping into the office if parents weren't sure whether a sickness was serious or not, and making that call wrong can get you a dead baby. Not to mention that babies get a gazillion vaccinations, and being a refugee makes it harder to get them on time. And finally, the circumstances of evacuation put babies at a lot more risk from car accidents. First of all, because their families are spending more time in cars, and secondly, when it came to the decision to leave with more people in the car than seat belts, and/or with babies on laps instead of properly secured, or having to leave people behind, their families made the correct decision to pack into the vehicle that they had and go.

So, yeah, I would wonder about a Katrina effect, too. RogerA, maybe you have a better intuition about the numbers in these states. How many families with infants were affected by the disaster? Would all of these effects together plausibly cause a spike that could get the infant mortality up that high in Mississippi, Louisiana and Alabama?

4) When we add North and South Carolina to the mix, then the next question is to wonder about local maternal/infant care programs. I gave birth in South Carolina, in the poorest region of the state with the highest infant mortality. We had several small, clever, public-private partnerships to address the problem. In one, the state printed up coupon books filled with donated goods and services. One was for pregnant moms, the other went home from the hospital. Each coupon was for a particular prenatal or well-baby visit, and the doctors offices would stamp your coupon when you came in for your visit. I believe that the program started when the state's first lady used her bully pulpit to bring in merchants, doctors, hospitals, etc. Other programs included aggressive campaigns to get mothers to breastfeed, to put their babies on their backs to sleep, to get carseats to every baby and make sure that they were installed correctly. Again, we are talking about small numbers of deaths, so a program that works can have a relatively large effect. And it is usually quite difficult to measure which programs are working and which are not having any effect. So the last question that I would want to ask is whether one or more of these programs got discontinued, or changed, right before the jump in infant mortality. Especially if they had a program that was saving lives, and they got rid of it because they didn't realize that it was working well, it would be really good to figure this out so that they can reinstate the program and expand it to other places.

[Specter]

Clarice,

Thanks for the h/t at AT, but I must defer. Somewhere in that thread I gave the link to the data I posted. It is The Clinton Legacy. I hope that you can get a mention of them into your post. Thanks.

[RogerA]

cathyf: it is an excellent question (re Katrinia effect). I don't know of any studies published, however that would require some very detailed and time consuming medical records review. I suspect, as do you and Rick, that it has to somehow be a factor; the question is how much. I will see if I can find something from CDC.

Having moved from Washington State to Memphis recently, I have been struck by what I consider to be a high rate of smoking in the population. I would certainly want to look closely at that factor (and as you know, the info on prenatal use of tobacco and alcohol is self reported, and there are certainly issues of, ahem, moral hazard.

To Tom's original post, if relatively lay people can figure out what may be going on, or at least looking at possibilities, what does that say about our "fishwrap of record?"

[Rick Ballard]

RogerA,

Which table were you trying to link?

[Ignatz Ratzkiwatzki]

Hi guys,

I'm the artist formerly known as Barney Frank. Decided to become the third stooge (Larry Fine) instead just to post here. Had to pop in and note the repulsive irony that this week the NYT is bemoaning an increase in unintended infant mortality while last week they were bemoaning the SCOTUS denying women the "right" to engage in a little intentional infant mortality. Sick.

[clarice]

Thanks, Specter. I didn't see the reference and will ask the editor to attach an addendum.

[RogerA]

Sorry Rick: table 34, Infant Mortality.

[RogerA]

cathyf: looks like several efforts going on, although nothing definitive published yet. The National fetal and infant mortality review program run by the college of OBGYN docs is collectinv information. See here: http://www.nfimr.org/

[abwtf]

There is no story here without at least a half dozen other accurate statistics. Cause of death, how old, what weight, when delivered, mother's health (including age, smoker, drinker, drug taker, overwieght, etc) among others.

The NYT goes to its standard playbook of 'government good, Republicans bad' and doesn't think to look any further. Sometimes counter-intuitive result can occur from greater access to medicine, for example older women getting pregnant or multiple-conceptions where one or two children are born healthy while another ends up at a very low-birth weight and doesn't survive.

Anyone who really wants to dig deep can try the March of Dimes which has state by state and regional data.

But without a better knowledge of who the would-be mothers are, I think any analysis is somewhat useless.

[TimUSSRR]

Well, I just had to sign in again, so I updated my account.

was quest33
was TimS

Now TimUSSRR

Well I like the old saying.

"There are liars and damned liars, and statisticians."

First as has been indicated, you need to look at the numbers for all the states for at least 10 years, better twice that or more.

The more personal categories you can get data on, the better.
Sex, Race, Demographic, Prenatal Care, Medicare, qualifying for Medicare across the states, drugs in mom, criminal records of mom, etc.

Then of course the disaster type things, like the hurricanes, displacement, etc.

Now once you have all these things (and probably some I haven't thought of) you can apply statistics.

Otherwise you are comparing apples and oranges, and bananas, etc.

I too think that it is very important. We shouldn't lose the babies. That is not who we are.
As for the moms and fathers sometimes, who choose who and what they are, I think we should raise our criteria about who can be a mother and father. But of course that would be frowned on.

[RogerA]

"There are liars and damned liars, and statisticians."

I feel compelled to come to the defense of statisticians and modify Tim's quote: "There are liars and damned liars, and those that have no understanding of statistics. And I am confident that 95% of the american public are in that category (plus or minus 2.5%) :)

[cathyf]

No, no, Roger. It's that there are 3 kinds of people in the world: those who can count, and those who can't.

:)

[SMGalbraith]

As someone who traveled through southern Mississippi after Katrina hit, I can't imagine how the event didn't contribute to these increases in that state.

It would be interesting to see a regional breakdown (i.e., southern vs. northern parts of the state) for Mississippi. The Gulf Coast region of that state - from Pass Christian through Gulfport and Biloxi - was wiped out. As in gone. Since I only traveled on the main road (I-10), I'm agnostic as to the side or back roads but they must have been (largely) unpassable.

Include in this the personal and physical stress (increase in miscarriage?) that any pregnant woman would have gone through.

A combination of less than normal health/medical care (appointments delayed, medicine scarcer for at least several weeks, et cetera) along with the stress, it's somewhat amazing (thankfully) that the numbers were only (sadly) slightly elevated.

Let's hope the numbers start go down again.

SMG

[jimmyk]

One objection to the article, or the graphic in particular, is use of the term "trend" in response to a single data point being out of line.

While your point about news-selection bias is correct, you give it short shrift. Think of it this way: There are 50 states, so the probability that some state in a given year experiences a reading more than two standard deviations above the mean is (by my calculation) 0.94. In other words, in almost 19 years out of 20, the Times could have a story like this even if it is just random.

[Rick Ballard]

Using blacks as a surrogate for nonwhites, this table shows Mississippi with a lower infant mortality rate than the Democrat heavens on earth of Michigan, Illinois and Wisconsin.

Mississippi is dead last overall but this note:

Mississippi is 50th this year, down from 49th in the 2004 Edition. It has been in the bottom three states since the 1990 Edition. The state ranks well in all three health policy measures: 8th for access to adequate prenatal care, which is available to 81.8 percent of pregnant women; 11th for per capita public health spending, at $197 per person; and 14th for immunization coverage, with 84.0 percent of children ages 19 to 35 months receiving complete immunizations. It ranks in the bottom five states on nine of the 18 measures: a high premature death rate, a high infant mortality rate, a high total mortality rate, a high rate of cardiovascular deaths, a high percentage of children in poverty, a high prevalence of obesity, a high rate of motor vehicle deaths, a high occupational fatalities rate, and a high number of limited activity days. It also ranks in the bottom 10 states for two other measures.

makes it clear that it's not for lack of expenditure of state funds on health issues.

[Charlie (Colorado)]

From which we further estimate that 95% of the results should fall within a range of
plus of minus 2 standard deviations, or 13.4 to 16.6 percent. Which means that the result on offer, 17, is outside the range we might expect from random variation.

Oh, nonsense. In fact, it means that about one year in 20, we should expect the total to fall outside that range.

[clarice]

I think we ought to have a field trip to Vegas with Cathy and Charlie acting as our guides.

[Jeff Dobbs]

I think we ought to have a field trip to Vegas with Cathy and Charlie acting as our guides.

Oh, I'm so down with that...but..."what happens in Vegas stays in Vegas"...must be signed in blood amongst all participants. If my adventures end up on The Smoking Gun, there will be hell to pay.

[clarice]

You have MY word.

[cathyf]

I think we ought to have a field trip to Vegas with Cathy and Charlie acting as our guides.

Sorry, I make money the only way you can in the gambling industry -- being overhead for the house. Back at the first trading company I worked for, one of the more memorable rules is "No Wishing, No Hoping, No Praying, No Speculating"

[clarice]

I was afraid of that, cathy.

[Daddy]

I thought the media had reported that they had resorted to cannabilism in the Super Dome during Katrina. Surely they weren't simply chowing down only on grown ups?

[mindsteps]

Check out the following article from the commercial appeal in memphis:

http://media.www.dailyhelmsman.com/media/storage/paper875/news/2006/11/28/TheGameOfLife/Memphis.Infant.Mortality.Rate.Highest.Among.Us.Cities-2551694.shtml

The article starts with this:

People don't usually think of Memphis when they hear the term "third world."

Most probably think of far away places where children are starving and people drink from the same river they bathe in.

But most people probably don't know that in 2002, when the statistics were last made available, 31 babies out of 1,000 didn't see their first birthday in the 38108 ZIP code alone.

An infant mortality rate like that competes with countries such as Syria, Nicaragua and Vietnam.

More than 200 babies died within their first year here in 2002, ranking Memphis No. 1 out of the country's 60 largest cities for infant mortality.

[Tully]

International infant mortality comparisons are one of my pet peeves. Infant mortality is not really that comparable cross-nationally. Every nation uses different measures.

Infant mortality stats are figured differently in every nation, and in the US we count every live delivery including seriously pre-term births, as well as some in-utero deaths (stillbirths). Many European nations do not count any birth in their infant mortality stats unless the baby lives for 48 hours after delivery--and somehow in some "enlightened" nations, babies with crippling birth defects almost never leave the delivery room alive, even if born kicking and screaming, and thus are not counted in the stats. In addition, we have the finest perinatal care for pre-term babies in the world, so we attempt to save a LOT of babies that in other nations would never be counted as births in the first place.

In America, a 20-week 500g preemie (survivability factor approaching zero) that is delivered not breathing and without heartbeat, that a doctor works on to establish respiration and heartbeat and fails, is counted as a "live birth" and becomes an "infant mortality," affecting both the IM and life expectancy stats. In almost any other country, if it were counted at all it would be counted as a "perinatal mortality," which does NOT count towards infant mortality or life expectancy figures.

In Russia, a baby that is not born breathing on its own AND does not live at least seven days AND weigh at least 1000g at birth is not counted as an "infant death." In some Scandanavian countries, infant euthanasia of live-born but "defective" babies is not unknown--and they are counted as perinatal mortalities, not infant mortalities. There is NO consistent international standard for defining either infant or perinatal mortality, and most countries don't track perinatal mortality at all. To properly compare the figures cross-country, you have to have accurate combined PM and IM stats. And they don't exist.

[JorgXMcKie]

cathyf, actually, there are 10 kinds of people in the world. Those who understand binary and those who don't.

;->=

[Rick Ballard]

From which we further estimate that 95% of the results should fall within a range of plus of minus 2 standard deviations, or 13.4 to 16.6 percent. Which means that the result on offer, 17, is outside the range we might expect from random variation.

Oh, nonsense. In fact, it means that about one year in 20, we should expect the total to fall outside that range.

Doesn't "that 95% of the results should fall within a range" say exactly that? Most of the time 95% = 19 out of 20 which kinda leaves the possibility that 1 out of 20 will be an outlier.