Consumers love rankings, but they may end up doing more harm than good

This week, El Celler de Can Roca in Girona, Spain, became the world’s finest eatery – at least according to the 2015 World’s 50 Best Restaurant awards – thanks to the “curiosity and creativity” of the three brothers who own it.

Earning such a prestigious award is a sure way to guarantee El Celler and others on the list have packed dining rooms for many years, bestowing on them success and profits. When Noma became the “world’s best” for the first time in 2010, the designation “is credited with catapulting the restaurant to international stardom, resulting in enough booking requests to fill its tables for years to come."

Such rankings abound in today’s society, whether organized by a group like the World’s 50 Best or based on the preferences of consumers on Trip Advisor. Typically they’re considered useful and beneficial, providing incentives for restaurants and other businesses to improve their services and products while giving the rest of us more information to make decisions.

But can they do harm as well and actually make us all worse off than before? New research we’ve conducted shows the answer may, surprisingly, be yes. In some cases, we might be better off without them.

Do rankings have a dark side?Trophies via www.shutterstock.com

Rankings, their critics and sour grapes

Such rankings have long been the target of criticism, especially from those being ranked and when the designation is particularly influential, as is the case with the World’s 50 Best Restaurants.

An “Occupy50best” petition has been circulating recently, signed by more than 400 chefs, restaurateurs and others who have attacked the methodology underlying the World’s Best rankings. They claim there are no established criteria and no consistent and objective gastronomical requirements, while jury members are “appointed by backroom politics, vote anonymously, without ever having to justify their choice of a restaurant or even to prove that they actually ate there!”

Such criticism of the methodology underlying rankings is probably as widespread as rankings themselves. One need to look no further than the never-ending debate about rankings of education programs and universities.

The key question is usually about quality and whether the ranking accurately reflects it. And criticism may be just sour grapes among the unranked (or low-listed).

But even if the ranking method is good and correctly reflects quality, a more basic question ought to be asked: is it good for consumers? To put it another way: is better information (as provided by a correct ranking) good for consumers' welfare?

The interplay among consumer decisions

If rankings were used only in situations where an individual decision affects only the decision-maker, more information would indeed always be better and never harm consumers.

However, in many markets where rankings play a role, consumers' choices do not exist in a vacuum as purely individual decision problems; they affect others' welfare and their decisions in often complex ways.

Consider restaurants. For many customers, the value of a dinner is influenced by the identity of the other patrons of the restaurant. Or education programs. Students learn from their peers, and the network generated at a school is crucial for future professional success.

In other words, these markets are characterized by “consumption externalities" – when consuming a product or service has a positive or negative affect on others.

Furthermore, in some markets, prices are rigid, leading to rationing so that not everyone who wants a product or service gets it. For instance, in many good restaurants, one has to book a table well in advance.

Last but not least, in markets with fully flexible prices and without externalities, firms' price-setting behavior is influenced by demand, and hence again agents' choices can’t be described as individual decision problems.

Foodies and normal consumers

In our recent research, we investigated the welfare effects of rankings in such markets where consumers’ choices are interdependent. We show that in such cases rankings may be harmful for consumers.

How could this be? To understand, let’s focus on markets with rationing (which fits quite well the case of restaurants, especially if one acknowledges that prices are the same every day of the week, but demand is much higher on the weekend).

Now, consider a hypothetical market with two restaurants and no rankings. One of the restaurants, let’s call it A, is more expensive and of higher (expected) quality, while the other, B, is less expensive and of lower (expected) quality. Because there is no ranking, there is uncertainty about which restaurant is indeed of higher quality. There are also two types of consumers: the foodies who value quality highly and the normal consumers who don’t.

Without the ranking, the expected quality difference between restaurants A and B is not sufficient for normal consumers to be willing to pay the extra cost of restaurant A. They therefore book a table at restaurant B. The foodies, on the other hand – because they are all about quality – are willing to pay the extra cost, and so they all book a table at restaurant A.

What happens when a (correct) ranking is published? That depends. Does it confirm that restaurant A is better? Or does it surprise consumers by ranking B higher?

Rankings and welfare

Let’s start with the case in which restaurant A ranks first.

Consumers now know with certainty that restaurant A is better than restaurant B. While this information doesn’t change what foodies would do (they are even more eager to check out the higher-priced joint), it does affect the choice of normal consumers. Indeed, being certain about the quality difference, normal consumers are willing to pay for the extra cost: they thus all call restaurant A to book a table.

The issue is that restaurant A is too small to accommodate both foodies and normal consumers. Therefore, it has to decline some consumers. The foodies that cannot get a table at restaurant A are made worse off by the publication of the ranking, whereas the normal consumers who do get a table are made better off.

Now, let’s consider the case in which the ranking outcome is a surprise: it reveals that restaurant B is better. In that case, the ranking affects the behavior of foodies who all want a table at restaurant B. The same capacity constraint problem arises and only some foodies are made better off by the publication of the ranking (whereas the normal consumers who cannot get a table anymore are made worse off).

Consumers lose in the end

What we show is that, once we take into account the probability of whether the ranking will confirm what we know or surprise us, and the rationing procedure used by restaurant (usually first-come first-serve), the welfare effect of the ranking might be negative for all consumers. A similar negative welfare effect is present in case of consumption externalities.

With flexible prices, neither rationing nor externalities are needed to reach the same conclusion. The reason is that the “winner” faces higher demand and can thus lift prices. If the level of demand depends a lot on quality (and hence on the ranking outcome), this rise in prices can be so large that every consumer loses in the end.

Far from suggesting that rankings are always harmful, our results propose a cautionary tale: contrary to the mantra, more information is not always better.

The question that remains is: when are rankings likely to be harmful? One of the conditions we identify is that consumers care sufficiently about quality. Ironically, it is when the information provided by the ranking is very important for consumers that it is most probable that it will hurt them.

Laurent Bouton is a member of the American Economic Association, the Econometric Society, and the European Economic Association. He has been selected to receive an European Reseach Council Starting Grant for his project "Political Economy with Many Parties".

Georg Kirchsteiger is also affiliated with Centre for Economic Policy Research, the Center for Economic Studies and Ifo Institute, and the Vienna Center for Experimental Economics.