Can We Trust Statistics?

The Numbers Game, which was something of a surprise bestseller in England, has finally made it stateside. It’s probably the clearest book I’ve yet seen on all the different ways in which numbers can lie or mislead, and as such it can be a bit depressing. So I sent one of the two authors, Michael Blastland, an email, to see if there was any hope:

I take most if not all of this book to be a work of media criticism.

Now more than ever, we’re surrounded by hugely important numbers:

macroeconomic statistics like GDP and unemployment; enormous dollar

amounts for everything from office redecoration to infrastructure

investment to central bank liquidity lines to CDS notionals

outstanding; stock market indices; credit spreads; mortgage rates; etc

etc. But a reader of your book will naturally, and healthily, be

skeptical of all such numbers and especially any spin or meaning which

is put onto them.

So the question is: what is such a reader to do,

when coming across such numbers? Absent a long and deep rummage in raw

data, how can a consumer of the media know who to trust, if anybody?

And if the messenger isn’t trustworthy, should we just ignore that

news source entirely?

Here’s Michael’s reply:

Good question: is there an unforgiving choice between cynicism or becoming your own complete data nerd? I like to think there’s another way.

It starts by remembering that numbers are like words – surprise, surprise – they too can be ambiguous or even untruthful. Does that mean we junk them?

Of course not, no more than we can junk the language. It means we approach them with our wits about us and apply similar standards to those we’d use on, say, the language of politics.

Or at least, we should try. In practice, many of us tend to say that if numbers admit uncertainty or interpretation, they’re no better than lies and damned lies. Words, on the other hand, well, we have no choice but to try to navigate our way through those, using our wit and experience. That’s a self-defeating double standard and no-one is more hurt by it than those who thereby throw up their hands in resignation at every piece of data. The truth is that we can bring wit and experience to bear on numbers too: a few simple tricks and techniques, sometimes a few daft questions and, yes, a sense of who can be relied on to tell it straight(est).

For example, putting big numbers into human proportions is the simplest of ideas, but how often does anyone stop to calculate how much that $1.2 trillion dollar deficit would mean to them personally? That’s not remotely difficult to work out – you share it between the population – and you’d find that it was roughly equal to a guy on average earnings (about $50,000) taking out a loan for a used car (maybe $4,000), but usually only paying the interest (about $60 a year at the rates paid today by governments), because that’s how governments tend to manage these costs. Suddenly that impossible number is comprehensible. This doesn’t tell you whether the stimulus policy is wise, but it helps you put it in far better proportion than if you are left dumbfounded by a row of 11 zeros. There are all kinds of such tricks.

They’re no magician’s wand, but they sure help. For some reason, they’re seldom taught.

Michael’s dividing by 300 million here — the population of the United States, more or less — while I’m generally more inclined to divide by 100 million, which is roughly the number of households in the United States. After all, average earnings across the 300 million individuals in the US, including the elderly and children and the unemployed, are much less than $50,000.

So for me, a $1.2 trillion deficit works out at $12,000 per household — and that’s $12,000 of new borrowing just this year, on top of all the debt we’ve accumulated so far, which is something north of $10 trillion. By the end of this year we’re likely to have a national debt of about $12 trillion, or $120,000 per household — and now you’re talking real money.

Michael also reckons that the government is borrowing money at just 1.5%, which might be true right now, but clearly isn’t going to be true indefinitely. Sooner or later those borrowing costs are going to rise, and the debt service costs on $120,000 at say 5% work out at $6,000 a year, or $500 a month. Again, real money.

And Michael and I are very much looking at things the same way. Get politicians involved, and the arguments will never end, even about how to frame the question. Should we ask, for instance, how much America’s total net worth would fall in the absence of a stimulus package and a $1.2 trillion deficit? How much of the Fed’s "quantitative easing" should we include in the cost of the stimulus? Should we try to put a value on the loan insurance being written by Treasury? What about the present value of future Medicare and Medicaid and Social Security liabilities?

The upshot is really that there’s no easy shortcut through the hard work of trying to understand numbers for yourself. Everybody has an angle, and no one — journalists emphatically included — is particularly reliable.

Update: Michael replies:

In order to put the cost to the US economy of a 1.2 trillion deficit into proportion, I think that we really should compare it with US national income. That after all is what the US is being asked to do, as a nation, to pay $1.2 trillion on this occasion out of its national resources.

To personalise it, that’s about 14.5 trillion national income divided by 300m people compared with 1.2 trillion divided by 300m people. And that gives about $50,000 per head income compared with about $4,000 per head costs. If you want to divide it by household, it’s about $150,000 income, compared with about $12,000 costs. It’s a rough and ready way of doing the job, but it’s not too bad, I think.

But spot on about the interest rates. They are not going to stay this low, and that’s when the problem might really begin to bite. The cost of servicing the accumulated debt could grow quite quickly at that point.