Epic Mickey (Credit: Disney)During his lengthy career as a video game developer, Warren Spector has had his share of commercial hits and misses, but he's rarely had a critical failure.
His 2010 release Epic Mickey split critics, however. Some heaped praise upon it, while others panned it harshly. Those diametrically opposed views gave the game a fairly low Metacritic score -- the lowest mark Spector had ever received.
"The reality is I've made a lot of games," he says. "And if you pay attention to Metacritic, the 'worst' game I ever worked on was 82 -- and that, I thought, was low. ['Epic Mickey'] was in the 70s and that got to me...I learned to take reviews and ratings with a little more of a grain of salt."
While that score might imply a flop, Epic Mickey proved to be anything but. Despite a slow start, it went on to become Disney's best-selling game ever on a single console and inspired a flood of fan letters to Spector and the company. And that dichotomy illustrates the problem a lot of developers (and gamers) have with the ratings aggregation site.
While Metacritic says it gives reviews from major outlets a heavier weighting in its review totals, the site excludes the game reviews of several notable sources (like USA Today and the New York Times). And that, Spector argues, makes the site a poor indicator of a game's quality.
Game publishers tend to rely on Metacritic scores as a barometer of sorts. Various cash incentives are often tied to those scores, which can be critical to a developer.
Earlier this year, role-playing game creator Obsidian Entertainment was forced to lay off between 20 and 30 staffers after Fallout: New Vegas only managed to achieve a Metacritic score of 84. The company's contract with publisher Bethesda stipulated that Obsidian would only receive royalties from the game if it hit 85 or higher. Without those, it was forced to trim its staff and cancel an in-development game, which was rumored to be headed for the next-generation Xbox.
"[Fallout: New Vegas] was a straight payment, no royalties, only a bonus if we got an 85+ on Metacritic, which we didn't," Obsidian creative director and co-owner Chris Avellone reportedly tweeted (before erasing the post).
Even high-profile developers can see fortunes won or lost on their Metacritic score. In the terms of its contract with Activision, Bungie's "Destiny" project must achieve a Metacritic score of 90 or higher to earn the developer a $2.5 million bonus.
Further frustrating developers is the fact that a good Metacritic score doesn't necessarily mean a game will sell well. Quantic Dreams' Heavy Rain earned a respectable 87, but was hardly a blockbuster seller. Cult hit Psychonauts got the same score, but flopped commercially. Wii Fit, meanwhile, only nabbed an 80, but worldwide sales have topped a stunning 22.61 million copies. And while Mortal Kombat vs. DC Universe only garnered a 72 cumulative score, it still sold over 1.8 million copies.
It's far from a perfect system -- and that's something even publishers willingly confess.
"It's accepted and we report their ratings with great glee when they are where we want them to be," says Strauss Zelnick, chairman and CEO of Take-Two Interactive Software. "It's not always [a great barometer]. … It's not a perfect correlation, but it's not a bad correlation."
One of the big complaints about Metacritic is the quality of some sites that contribute to the overall score. While Metacritic says it selects outlets that "are well-regarded in the industry and are known for quality reviews," many of the sites it cites are unknown to readers.
Underscoring the trouble with the system is that other forms of entertainment aren't held to the same standards. Film studios, for example, don't care much about how a project registers on Rotten Tomatoes or Metacritic. Their focus is entirely on the bottom line. If a movie makes money, they're happy — and they do what they can to ensure those involved are happy so a sequel can be produced. Game developers, by and large, don't have the luxury of refusing to do a follow-up to a hit game. Critical adulation is nice, but in the end, virtually any studio would prefer a cash cow like Transformers to a well-reviewed film like Moonrise Kingdom.
That's true in the videogame industry as well. Unfortunately for some developers, a game that doesn't succeed at both can cost a lot of pride — and money.