Explaining Zagat grade inflation

New York magazine’s Grub Street blog points me to a piece at smartmoney.com about the Zagat guides, which has some interesting datapoints:

When the Zagats started selling their 1983 New York restaurant guide, it was no mean feat for a chef to score a food rating of 20 or higher, the benchmark for “very good to excellent” in Zagat terms. Only one in four New York restaurants did so at the time. Today fully 70% reach those heights. It’s as if the bottom tier dropped out: Just over a decade ago 189 out of 1,300 New York restaurants rated 15 or below; today only 23 do, despite the fact that the guide now rates more than 1,500 restaurants.

Is this a function of the Zagats being too cozy with the restaurants they cover? Grub Street, which once edited the Long Island guide, thinks not:

Most people who provide quotes to Zagat eat frequently in just a few restaurants, which they wildly overrate. In the Long Island guide, nearly every sushi restaurant was praised as having (we’re making these up, because we’re clever that way) “sushi so fresh you’ll think you’re swimming in the ocean,” and every local Italian restaurant has “pasta to die for.” The respondents go wild with the numerical ratings as well. The Zagats may well be power mad, and the way they do business may not exactly project an aura of incorruptibility, but the Zagat respondents need no help driving the ratings up through the roof.

I’m sure this is true — but although it explains the high scores in the Zagat guides today, it doesn’t explain the lower scores in the Zagat guides of old. Why the grade inflation?

I think the main reason is hinted at by Grub Street: people used to send in surveys as a service to Zagats, in return for which they received the next year’s guide. Both the surveyers and the company benefitted from this arrangement. Today, however, Zagat is seen as a corporate behemoth, and the number of people who think that sending in a survey is some kind of public good is shrinking dramatically. So the main reason to send in a survey is now different: it’s to do a favor to one’s favorite restaurant, by making it look as good as possible and to see if one can help it beat the competition.

Zagat’s three-point system exacerbates the problem. The scores are given on a 30-point scale, but a score of 25, say, is actually just an average rating of 2.5, multiplied by 10. People naturally give their favorite restaurant a score off 3/3, and then score other restaurants off that benchmark. So if you’re doing a guide to an area where peoples’ favorite restaurants aren’t particularly great, then the scores are likely to inflate. Which may or may not help to explain why the Cheesecake Factory in Las Vegas is rated 21, while the Cheesecake Factory in San Jose is rated 16 — the kind of people who go to the Cheesecake Factory in Las Vegas are more likely to think of it as their favorite restaurant than the kind of people who go to the Cheesecake Factory in San Jose. (And, yes, the fact that the Las Vegas branch has a score over 20 does mean that a significant number of reviewers gave it that 3/3 rating.) As the Zagat guides become increasingly popular, the reviewers cease to be fine diners, and start becoming much more like the general population. As a result, fine dining establishments such as Jean Georges are no longer the benchmark by which other restaurants are judged.

But it’s also surely true that some of the grade inflation is simply due to the fact that restaurants now are better than they were in 1983. Does anybody really deny that?