Interest and pressure groups put out garbage masquerading as useful research on a daily basis; it’s the job of the press to recongnise it as such and ignore it. Most of the time, the press does a very good job of that. But there’s a false-positive problem: it only takes one journalist to give that research the imprimatur of a respected business publication.
Let’s say that a researcher in bed with some pressure group — let’s call him Samuel Bornstein — approaches a very large number of journalists with his silly research. And let’s say that the vast majority of those journalists take one look at his research and ignore it. That’s fine, until Bornstein chances upon Randall Forsyth. Next thing you know, there’s a column in Barron’s giving credence to the research, and anybody reading the article has no indication whatsoever of how weak the reasearch really is.
What’s worse, even though the Barron’s column is part of the newspaper’s online offerings, it doesn’t include any link to the research in question — which means that the readers have precious few clues that it might all be based on the crumbliest of foundations.
Here’s Forsyth, cribbing shamelessly from Bornstein and NASE:
A surprisingly large proportion of self-employed entrepreneurs used risky mortgages either to start or to expand their businesses, according to a recent study by the National Association for the Self-Employed. And millions of them face sharp increase in their loan payments — just as the economy tumbles into a freefall.
According to NASE, an estimated 3.7 million small business owners have what the group calls "toxic mortgages: — alt-A loans, alt-A adjustable-rate mortgages, option ARMs and interest-only loans.
But that 3.7 million number just doesn’t pass the smell test. Here’s what Bornstein writes in his official conclusions after conducting the survey of NASE members:
According to this survey, it is estimated that 3,709,800 small business owners hold Alt-A and “toxic” mortgages that are scheduled to “Reset” beginning in 4th Quarter 2008 and continue through 2012. These small business owners will be at-risk for “payment shock” and default as their monthly mortgage payments skyrocket.
According to Inside Mortgage Finance, a trade publication in Bethesda, Maryland, approximately 3 million people hold toxic mortgages totaling $1 trillion. The magnitude of this Alt-A and “toxic” mortgage crisis exceeds the subprime mortgage crisis which had $855 billion of subprime loans outstanding.
Yes, Bornstein really does say, the paragraph after telling us that 3.7 million small business owners hold toxic mortgages, that the total number of toxic mortgages in existence is just 3 million.
This discrepancy is responsible for the first line of the NASE press release, which should itself have raised any number of red flags with Forsyth:
The nation’s small businesses own ninety-three percent of all "toxic" mortgages and are at risk of defaulting on their loans/payments.
I asked Bornstein about this, and he replied:
Forget about the 93% figure. That was based upon 12.2 million self-employed in 2003 SBA figures. I found more recently released stats that had 16.2 million in 2007. NASE missed making my revisions in my submitted revised drafts.
Which completely misses the forest for the trees. Bornstein’s survey came to the conclusion that 22.9% of small business owners have toxic mortgages. If there are 12.2 million small business owners, that means that 2.8 million of them have toxic mortgages. Divide 2.8 million by 3 million total toxic mortgages, and you get the 93% figure.
But using Bornstein’s revised 3.7 million figure, one comes to the conclusion that 123% of all toxic mortgages are held by small business owners. No wonder NASE missed that revision: even they probably saw how silly that assertion was.
Once you start looking at Bornstein’s conclusions with a skeptical eye, they all start looking silly. Take this, for instance:
19.2 % (3,110,400 At-Risk) of all self-employed business owners are at-risk of “payment shock”. They do not know the monthly mortgage payment that they will be required to pay at “Reset”.
It’s unclear where the 19.2% figure comes from. But with mortgage rates hitting all-time lows, people with adjustable-rate mortgages are more likely to see their rates go down than go up at reset. Yes, there’s a handful of people with negative-amortization mortgages who might be shocked when they start having to pay back principal. But according to Bornstein’s own survey results, they only account for 3.7% of all small-business owners — nowhere near the 19.2% figure that Bornstein is throwing about.
And that bit about not knowing the monthly mortage payment they’ll be required to pay at reset? Well, of course they don’t know: their mortgages haven’t reset yet — and most of the mortgages which will reset won’t do so until 2010 or 2011. Bornstein isn’t uncovering shocking ignorance, here, he’s just saying that small business owners have no more idea than anybody else where interest rates are going to be in two or three years’ time.
Which brings us back to Forsyth’s assertion that "a surprisingly large proportion of self-employed entrepreneurs used risky mortgages". Turns out, not so much: fully 70% of them, it turns out, wound up with fixed-rate mortgages, despite the fact that their income was lumpy and hard to verify and that they were exactly the sort of people for whom option ARMs were designed.
And even that number is dubious. Bornstein refused to tell me how many people received his questionnaire or what the response rate was. He did admit that he made no attempt to determine whether the respondents were a representative sample of NASE members, or whether NASE members were a representative sample of the self-employed, or whether the self-employed constituted exactly the same group of people as small business owners. (Clearly they don’t; I’m self-employed, but I’m not a small business owner.) Instead he simply extrapolated with gay abandon from 1,687 survey respondents who hold a mortgage and belong to NASE, and multiplied the resulting percentages by a whopping 16.2 million to get his headline results, which he quite amusingly reported with five-significant-figure specificity. In other words, his entire survey is something of a joke.
My point here is not really to pick on Bornstein and Forsyth in particular. But I did receive an email from Bornstein on Sunday plugging this story, calling it "an interesting finding that impacts the current economic situation, and may provide a key to a solution to mitigate the impending losses expected in 2009". If he’s emailing me out of the blue, you can be sure he’s emailing many other journalists, too.
And so since Bornstein approached me, asking me to write about his research, and since Forsyth had already done so uncritically, I thought it might be worthwhile to show a little bit of how the journalistic sausage is made.
The lesson of this story is that if you come across any piece of journalism which cites official-sounding statistics from expert sources, treat those numbers with a very high degree of suspicion unless you already consider that source to be reliable. It’s a pity, but you can’t just simply believe what you read in the paper. And these kind of reporting errors are never going to be caught by editors — or even by media critics like myself, unless the researcher in question comes to me directly and asks me to look at his research.