Stir It Up
Why media shouldn't accept Detroit statistics at face value.
Published: April 25, 2012
Journalists tend to love numbers. As subjective as anyone's perceptions may be, numbers tend to give the perception of objectivity. If a reporter writes that a band is popular, then the editor wants to know how many records they sold last quarter, or how many people showed up at their last show, or how many girls screamed and fainted when the musicians walked by.
And when someone publishes a Top 10 list, we absolutely go crazy. Even when a list is much longer we'll pick the Top 10 just to make it manageable. And when some study comes up with statistics about a city, it can become gospel truth in people's minds.
Detroit's been on the receiving end of numerous lists and studies that play into perceptions about the city. Those conclusions seem to verify perceptions people already have so there is little motivation for anybody to question them. For instance, in October Forbes magazine listed Detroit as the most dangerous city in the nation, based on an analysis of FBI crime statistics. The rest of the media ate it up.
About a year ago the Detroit Regional Workforce Fund released a report claiming in one bullet point that 47 percent of Detroiters are functionally illiterate. This was actually a rehash of a 1998 report from the National Institute for Literacy. Still the media ran with this "new" data, and CBS Detroit, Fox Nation, the Huffington Post and even African-American news website The Root piled on — as did the UK's Daily Mail. Forbes magazine reported it too, then backtracked and ran a story questioning the report and pointing out that the "statistic" has been a staple of white supremacist websites for at least a decade.
Danny Devries, a data analyst at Data Driven Detroit (D3), took on the myth in a December D3 staff blog post after a caller repeated it to him while he was on WDET-FM's The Craig Fahle Show in December.
"I had seen this 47 percent functional illiteracy rate bouncing around the media for a while," Devries wrote in an e-mail to me recently. "That number just seemed so high. I did some digging and realized the methodology for determining the illiteracy rate for Detroit just wasn't accurate. There was too much extrapolation to determine a specific rate for Detroit based on a national survey from 1993 of 26,000 people. All of this isn't to say that functional illiteracy isn't an issue, but it deserves fair treatment. Throwing around this '47 percent' number isn't doing anyone any favors."
The question is why did these arguably responsible newsrooms take the bait like a big, fat bass going after a juicy nightcrawler? Mostly they took it hook, line and sinker. Devries got into that in his D3 blog post in December.
"Dissecting the origin of this statistic is more about the poor data literacy of some of our news agencies than it is about Detroit's literacy rates," wrote Devries. "Many of them referred to the report as a 'new study,' missing the important detail that the research is far from new. The 47 percent Detroit literacy rate is the result of a 1998 analysis by the National Institute for Literacy, performed on data from the National Adult Literacy Survey, published in 1993. That's right: those 'alarming new statistics' are based on data almost two decades old. Almost all of the media coverage neglected to communicate that fact."
Apparently the cutbacks in newsroom funding are having an effect on the reliability of information reported. But where is reliable information on the adult illiteracy rate in Detroit? The truth is that we don't know. Even the original data from 1993 came with a warning that the information was only reliable at the county level. The most useful information I could find came from the National Center for Education Statistics, which reported in 2003 that the functional illiteracy level for Wayne County is 12 percent. The only way Detroit could be at 47 percent illiteracy while the county is at 12 percent would be if there was zero illiteracy in Wayne County outside of Detroit. By comparison Washtenaw County's functional illiteracy rate was reported to be 6 percent; Oakland's and Macomb's at 7 percent each. The entire state of Michigan came in at 8 percent.
So let's get back to that Forbes list. Is Detroit the most dangerous city in the country? As commendable as Forbes was in debunking the illiteracy rate myth, they regularly fall into the same trap with statistics. For some perspective, let's consider a report by Zack Taylor, a Toronto-based planning consultant who wrote a study ("Lies, Damn Lies and Statistics: A Critical Examination of City Ranking Studies") on the shortcomings of highly publicized city rankings. Taylor's conclusion for journalists is "Be skeptical" when publicizing these rankings. After all, people believe this stuff and it becomes the basis of opinions that can turn into municipal policies.
Is Detroit the most dangerous city in the nation? Not so says a D3 blog post titled "Apples to apples, Detroit ranks 17th not first in crime, or ten reasons not to trust top ten lists." Among the things pointed out here is "Forbes uses the FBI's Uniform Crime Report data in exactly the way that the FBI advises it should not be used: to rank locales." It goes on to point out that the FBI data is incomplete, particularly because cities voluntarily report these statistics and some cities aren't included, like Chicago. So if Chicago happened to be the most dangerous city in the country no one would know based on the Forbes list or the FBI list.
> Email Larry Gabriel