IGN, Gamespot review score inflation revealed

"Personally, I think a lot of the non-problem that most people have with the score distribution is with the 10-point/percentage rubric."

ha! "non-problem." awesome. a star for all my arachnid friends!

heh.

m3mnoch.

OK, first off, this isn't just something that happens only in game journalism. The same criticism can be applied to movies/music/etc. Most reviews skew higher. There are a number of reasons for this.

Reviewers, in general, don't like to review mediocre products. There are only so many ways to say that a game is lackluster/boring. So you really reserve your strongest venom for the media that are bad/flawed in some truly remarkable way. In other words, it takes a lot more than a mundane, cliche story for an RPG to get a low score. Stuff that's bad, but bad in conventional ways tends to fall into the 5-6 range just because reviewers try to look for stuff that the game gets right. Focusing on the boring stuff makes for boring writing. Games that are out-and-out horrible tends to get four or less.

But then there's the issue of how we probably never get to review a lot of stuff deserving of a 4 or less. It's sent back for more development, to at least get it up to the mediocre standard. Game companies invest a lot of money in their product, and can see pretty easily if a game's mechanics or controls render it unplayable. And they'll order a revision or just kill it outright if they don't think it can be salvaged. That's how Frame City Killer got killed off for the Xbox 360. (Note: My local EB games still has the box for this on display, months after it was canceled.)

Also, I'm with Derbeste. Enough with stupid metrics like thumbs up/down and stars. Scores can be useful, but should be no substitute for reading the reviews. A quick look on Metacritic shows a lot of really good games in the 70/80 range (Psychonauts, for example, which I'd rate among my top 10 games of the last generation.)

Usually, the more options a reviewer has in awarding a score the better. The 100-point scale is useful. A five-star system is not. (Just take any critic who gives out stars and start comparing which movies get which grades, and the system will start to break down.) But still, the system is flawed. Which would most gamers rather play: Aggressive Inline or Fable? (Both have an 85 on Metacritic.)

I'm going to have to disagree with you on the uselessness of a five-star system, Dirk. I'd say it's much more useful than a 100-point scale.

The problem is that when you're scoring a work in an entirely subjective way, the most a numerical score can do is communicate the reviewer's general opinion on the work in question. 5 stars gives you major categories for "bad," "bleh," "okay," "good," and "great." I don't really see the need for much more than that, but if you need it, you can go to half stars, which gives you the same granularity as a non-decimal 10-point scale, but with none of the culturally-inherent weighting that would occur.

Really, can you say there's any noticeable difference between a game that scores an 8.5 or an 8.8, even on the same site? Even with the *same reviewer*? It depends entirely upon the reviewer's tastes, and whether or not they coincide with yours. No, since the best you can glean from a review score is a general overall impression, I think the review rubric should actually be designed with that in mind.

Anyone else remember Daily Radar's scoring system? "Miss," "Hit," and "Direct Hit." It was pretty much perfect, because it *forced* you to read the review to find out what was so great or horrible about the game, not just compare one number to another.

>>>I'm going to have to disagree with you on the uselessness of a five-star system, Dirk. I'd say it's much more useful than a 100-point scale.

The problem is that when you're scoring a work in an entirely subjective way, the most a numerical score can do is communicate the reviewer's general opinion on the work in question. 5 stars gives you major categories for "bad," "bleh," "okay," "good," and "great.">>>

This is a good point. Like I said, I'd prefer no metric at all. But if forced to pick, I'd go with a bigger one. To see how flawed the star system is, pick any critic/site with a large volume of reviews, and start comparing.

Example: Rolling Stone gives Cars 3.5 stars (out of four). Capote also gets 3.5 stars. So does that mean that they're equivalent in quality?

It is possible that games that really suck just get cancelled. I like to think that the truly truly awful games are generally not showing up on shelves.

Then again, there are always the stray "Drake of the 99 Dragons"es of the world.

I think GameSpot is pretty fair in their reviews, and I've never noticed anything out of the ordinary. They have both editor and user reviews, so you can make the comparison. Also, since you now have meta sites, I think anyone truly researching a game these days can easily make an informed decision.

If anything reviews are inconsistent only because you never have the same person reviewing all games, and each person has their own tastes. I would imagine those reviewing the games are playing the consoles and the genres they enjoy most. Again, this is why seeking more than one source for a review is important.

Anyone use Netflix? Their star system is sometimes totally out of whack, but I find the people and critic reviews are far more telling.

@ Dirk

To compare a "Cars" review to "Capote" would be silly. They're different genres. No one deciding what movie to see will be torn between those two pictures. It goes the same for video games.

If I'm looking for a FPS (and I'm an idiot, so I'm going to base my entire decision on a rank) I will line up two games with similar release dates, similar consoles, and similar ESRB ratings.

Who uses video game rankings to see which game is better? You find the game you're interested in and then see how it (individually and for its own merits) was rated.

If I'm in a video store picking movies for my kids and I've narrowed it down to two choices, but one has "two thumbs up" and the other has "two thumbs down" it would definitely help my decision (assuming I trust Ebert and Ropert's taste in kids' movies).

The fewer options a reviewer is given for rating, the better. I would even argue that TWO choices would be best; "Buy it" or "Don't Buy It." It's all opinions anyway.

If only this chart could somehow quantify what those reviewers actually write about the games they review... you know... those multipage documents telling you what's good and bad about a game, outlining what to expect from a game, and comparing to similar games? Do you even know what I'm talking about? Or do you only look at the numbers? Oh nevermind...

I would just say that the scale is not linear but exponential. The difference between 6.0 und 6.5 is much lower that the one between 9.0 and 9.5. Anything between 0 and 5 would be absolute crap, everything above 7 is good to go and anything above 9.0 is pretty much a must have. I don't try games rated lower that 7.0 because as good as they may be there are certain reasons why they are rated so low, usually things that destroy the fun too much as a low frame rate or crappy controls.

Additionally, if you would the IGN guys test *all* the games released the last ten years I am sure that curve would look another way.

It will be interesting to see how the other sites pan out. Video game reviewers each hav e their own value system. But most readers and reviewers would agree a 5 and below is a failed game, 6 is proof, 7 is decent, 8 is good and 9 is great. That's your 5 star system right there. Numbers below 5 are merely to outline why a game failed, or to deride it. In the case of PC games, it makes sense because a patch may fix a flawed game.
Also, is there any grading system out there, anywhere, school, media, wherever that actually uses a bell curve with 50/5 being average? It seems downright un-American to me.

As others before have said, 7/10 is average/fair/decent so it makes sense that the average score is in that area. No stupid conspiracy theories needed. And of course they're getting paid to review games, just like you get paid to blog about them.

m3mnoch worte
I mean, i'm as big a fan of citizen kane as the next guy. do i think it's a better movie than say... syriana? nope. cut out the nostalgic factor and release it today and it's crap. for its time, tho? sheer brilliance.

Though this does not pertain to the question at hand, I guess you arent that big of a Citizen Kane fan if you think that one of the greatest films made is worse than Syriana which wasnt even the best film of 2005. Cut out the nostalgic factor and release it today what you have isnt crap but a film with great acting, and awesome cinematography which is a joy to watch. Sure, the movie would lose a couple of points based on todays standard but it sure doesnt become crap.