Xbox Live Indie Games ratings are being manipulated, devs say

by Christopher Grant on Mar 30th 2011 2:25PM

If you thought jocks versus nerds was a war you left behind in high school, we've got some bad news: It's followed you to a new, less predictable battleground. Yup, Xbox Live Indie Games. Seemingly provoked by a harmless call to action by the developers behind College Lacrosse 2011 on their Facebook page -- they even said, "Please remember to not rate other games low to help CL11" -- some developers have noticed a drastic drop in their XBLIG rankings.

"After hanging out around the #6 top rated slot for a month or two, our game's rating started to drop very quickly," Cthulu Saves the World dev Robert Boyd wrote on Microsoft's App Hub Forums. "In less than a week, we went from being #6 top rated to our current spot of #11." And how are the lacrosse games doing? "In contrast, the Lacrosse games which were in the middle of top 20 best rated (I don't remember the exact positions) are now ranked #4 & #5," Boyd observed.

For its part, Microsoft's XNA Creator's Club has acknowledged the possibility of manipulation, tweeting, "We are investigating a possible misuse of ratings on #XBLIG titles. We'll announce more information here as it develops."

The real issue is how easy it is for a group of individuals (take the 177K fans of College Lacrosse's Facebook page) to influence the popularity of games on the service: Users can rate games on Microsoft's website without owning an Xbox, purchasing a game, or even playing a demo. When download numbers are as tied to rankings as they are on the Indie Games marketplace (just like on the iOS and Android marketplaces) this issue is really critical in maintaining the fairness of that ecosystem.

Via: Armless Octopus
Source: App Hub Forums
More coverage: Facebook, @XNACommunity

Tags: cthulhu-saves-the-world, microsoft, Robert-Boyd, XBLIG, xbox, Xbox-Live, Xbox-Live-Indie-Games, Zeboyd-Games

Reader Comments (49)

It's like that with many services.. they allow people to rate games or product without knowing if those people actually own it! I can see the dilemma if you sell products in a store, but with LIVE it's a very closed system.

I mean you can't get Indie games in a store, so why not just include a rating feature that is only active after you purchase the full game?? This would have been something I'd have implemented right away if I were part of the MS team. I mean why in the world would you NOT implement that? What would be the motivation to let anyone rate a game without knowing if they purchased it (laziness probably)? Doesn't that sort of destroy the whole purpose of a rating system??

Sometimes I think the people behind the lines are really really dumb..

@xreadmore The only issue is people tend to buy games that they like. I always download the demo first to make sure that I like the game. For someone like me, a game I dislike will probably never get a bad rating, and games i enjoy will get good ratings. You end up with game almost only getting good ratings, and so it can be hard to distinguish how good a game is by rating.

A change is needed, but fact is no matter what you do, people will try and find a way to game the system.

@cuiof8d9sa Glad to see the rating system here works as intended. ;)

The rating system on XBL has been pretty broken for a while.

I never understood why anyone who has never even played a game can rate something. It just doesn't make sense.

At least with the Apple App Store they managed to get that thing fixed pretty quickly. Would have hoped Microsoft would have learned from that.

Its not just a "dev say this". The evidence is pretty staggering.

And its not just "Oh my poor game is not as high rating as it was before!". If an XBLIG drops off the top rated (or any of the lists on the main page), sales drop dramatically. Even if a game moves from the first screen (or the top 6ish games), sales will drop because the game's reduced visibility means less people know about it.

While petty complaint to most people, this is a serious issue based on a big flaw in Microsoft's design of their content delivery system.

Why don't they just do away with the ratings system altogether? I can't think of the last time I saw an XBLA game that was ranked less than 3 stars and most are 4 or 5. It's stupid.

@Faceless Troll

these are indies, not xbla.

I'd like to see the system changed to where you can only rate a game once you own it. It's one thing to play a demo and get a fell for a game, but I don't think a rating based on a demo is a good thing.

@theangryheretic

It's a nice idea, but people would be unlikely to buy games they didn't like, so if it was restricted to purchasers only then it's probable that all the ratings would be positive.

they should only allow rating games through the xbox in the marketplace

I rarely trust a game based on how many stars it has on the service provided. If it's anything more than just an impulse buy, I'll check reviews online or at least download the trial before blindly throwing my money away.

I think the reason they can't just start restricting ratings to people who have purchased the game is because they have the same rating system across the entire marketplace. Maybe they can reset the ratings and update the system when we get the new-new new xbox experience.

Maybe people just like the lacrosse games better then Ctuhlu saves the world?

@supersonic124

I highly doubt that.

@Ezio Auditore da Firenze
I dunno, it just seems to me that Boyd is mad that his game isn't as highly rated as the others. Accusing the competition of cheating is a bit low.

@Ezio Auditore da Firenze

ezio, i saw what you posted on the top of this page before it was somehow removed you little weasel. why are you hating anyway? did you get punked in school? or are you just another who shares that small-minded, stereotypical view that all lacrosse players must be jock assfaces? or, are you too tubby to play the game in reality? i'm going with the latter you make believe assassin you!

@supersonic124 You're missing the facts here. Even if the Lacrosse game had the lead by a landslide it doesn't account for the sudden and staggering drop in several other XBLIG games. If the game moved to the top then the others would just move down one spot, not many.

@Liquidfingers

1. The only time I ever got 'punked' in school, was by a redneck, not a jock.
2. Wait, they're not all 'jock assfaces'?
3. Tubby or not, who would want to play lacrosse? I may as well throw on a skirt and take up field hockey.
4. Make believe assassins > liquidy fingered deadpool avatar sporters!

=P

@supersonic124
Psst...Guys, I think Supersonic may be working for the stat manipulators, Quick, Someone declare shenanigans!!

@Ezio Auditore da Firenze

"Tubby or not, who would want to play lacrosse? I may as well throw on a skirt and take up field hockey."

you may not know who jim brown is, but he certainly begs to differ. lacrosse is a very fun sport. i played it in high school, as i'm sure it's become apparent. i don't consider myself a jock or particularly douchy, and not many of my teammates were either.

what did you do when that redneck punked you? did you just take it or perform an assassin leap dagger to the back of the throat like a real man? just wondering...

@Liquidfingers

I wish. =/

Stabbing people in the throat is frowned upon these days, for some strange reason.

I didn't do much of anything, honestly, just took it. I knew if I did reciprocate, his much older and larger brother would've came after me, so....

@Liquidfingers

well you made it pretty obvious that you are a loser

@supersonic124

The evidence would definitely imply there's been some manipulation (though not necessarily related to Lacrosse) - CStW has consistently held the #4 US Top Rated place since its release late last year, until about two weeks ago when the game started to plummet, reaching the #20 position yesterday.

It's statistically really unlikely that this would happen and I believe some developers have done the math and reckon that it'd only be possible if the game had received hundreds of one-star ratings in the past few days. It's also worth noting that this hasn't affected ratings in other countries where CStW still maintains its high position (i.e. if there is manipulation, it's being done with the US ratings only).

@AntiVillian

and you've made it abundantly clear that you eat numerous fat dicks a day.

Manipulating scores? Wah, wah, wah. Ok, let's all give Shoot 1UP 5 stars! :)

just take off ratings entirely.

although i don see the problem here. they said not to rate other games lower. so if CL can get a bunch of people interested enough to vote...how can this not be good for indie games regardless if they own it or not. it brings awareness.

To be fair, the College Lacrosse devs are probably just trying to drum up support for their upcoming Kinect-based date rape simulator.

@baby sea tuna

ha ha...haaaa

couldn't the same (and much, much more) be said of any dev behind a basketball game?

@Liquidfingers

Y U mad, Liquid?

Just because you're part of a pretty disgusting stereotype doesn't mean you have to take it out on all of us...

@Wiizer

lolwut

I'm not american but was that jock thing he said serious? ahahah Here the only people who play lacrosse are... wait. nobody plays lacrosse! it's one of those obscure sports like competitive table tennis or curling. Yino, only skinny nerdy guys who want to be good at a sport without need to compete with anyone fit play it.

This is the case with any system that let's users openly rate things

One issue I see is that on a website like Amazon.com you don't have to buy a product from Amazon to own it. You could have bought it somewhere else and then decided to write an Amazon review for it.

While you might be able to buy an XBLIG in another place, I don't think you should be able to review it on XBL unless you bought it from XBL. It's that simple.

It seems like every other month there is a major news story about people manipulating Amazon.com reviews because the creator of a product did something controversial. I wouldn't be entirely surprised if it was happening here as well.

Lame. More people who can't be trusted.

I can't believe a college lacrosse game has 177,000+ fans on Facebook. I wonder how many of those people actually own the game.

I'm not seeing any fart games at the top of the list, so it can't be all that bad.

Really it's the same throughout the XBL Market place. Unreleased games have 5 star ratings from millions of users as soon as they're put up for various video / theme downloads. I can definitely see how it's more detrimental to the Indie group though. I can't see MS changing the system though. There's nothing specifically stopping anyone else, who thinks they are being unfairly moved in the ratings list, from putting out a call to their fans to rate their game anyway.

I hate when idiots rate games that arent even out yet.

@xbashingx

Amazon customers are notorious for doing this. Drives me crazy too.

@Shadsy

Metacritic is HORRIBLE. Its influence has even reached Joystiq.

I personally would like to prevent indie games from showing up on my console.

@goat biscuit light yourself on fire.

@Jack Kevorkian notice I said, my console, not yours.

A good example? The user rating system on Metacritic, but the sad news for XBLIG is that most buyers don't have a critic score to influence their decisions (at the time of purchase, not including those who do their research ahead of purchase), so they have to base their judgment solely on user scores.

XBL marketplace ratings have been broken since the day they launched. They really need to be restricted to people who have purchased or downloaded content.

@Imperial i rate demos and i'm not buying anything. i also downvote any dlc that overcharges the customer or is abusive. Marvel vs. Capcom, Dragon Age, Tiger Woods, all Konami games, etc.

I will continue to do so until the end of days.

XBL Indie ratings largely work. Look at the Indies up there. Zombies is still probably the best Indie game out there. Ezmuze and Beat Hazard (game created by music on your HDD) are fairly creative / innovative...Love the RadianGames as a whole (also highly rated)...

I actually don't think much of the Lacross games personally...I would understand their rankings going down over time since a new Indie title gets released basically every day...

Amazon (and others) on the other hand, is in much desperate need of a change. You should NOT be allowed to rate things that don't even exist yet. That could be argued to be sales manipulation. It should be illegal. It certainly is annoying regardless...They also lump old DVD reviews in with new blu-ray releases when all I wanna see is if the new transfer is any good - also fairly manipulative....

The IMade a game wit hZombies in it (Oh Yeahhh) is alot of fun on the cheap. All my Live buddies have gotten a copy and love it too (though it would be cool for multi-coop play). only issue- if you are epileptic, needs to warn of the flashing red graphics... Man...

But yeah, I felt XBLA and even "most D/L titles" was full of sh-t when I saw Cabelas Fishing like one of the top five... um... ok.

This happens to all games in good and bad ways on Xbox Live since there's no purchase requirement to rate games.

simple solution: allow only the xbl live gold member to rate games along with purchase requirement.

Really, just Indie games? The entire rating system is rigged.

Cthulhu Saves the World

Xbox Live Indie Games ratings are being manipulated, devs say

Reader Comments (49)

Posted: Mar 30th 2011 2:35PM xreadmore said

Posted: Mar 30th 2011 4:01PM FFKilla said

Posted: Mar 30th 2011 10:56PM daraikenu said

Posted: Mar 30th 2011 2:36PM omgJOHN said

Posted: Mar 30th 2011 2:37PM Alexisonfire said

Posted: Mar 30th 2011 2:39PM Faceless Troll said

Posted: Mar 30th 2011 4:34PM ChomskyKnows said

Posted: Mar 30th 2011 2:42PM theangryheretic said

Posted: Mar 30th 2011 6:03PM DVMGames Richard said

Posted: Mar 30th 2011 2:44PM KiDPANDA said

Posted: Mar 30th 2011 2:44PM Roofie said

Posted: Mar 30th 2011 2:45PM Acosta02 said

Posted: Mar 30th 2011 2:47PM supersonic124 said

Posted: Mar 30th 2011 2:57PM Ezio Auditore da Firenze said

Posted: Mar 30th 2011 3:06PM supersonic124 said

Posted: Mar 30th 2011 3:12PM Liquidfingers said

Posted: Mar 30th 2011 3:23PM Teancum said

Posted: Mar 30th 2011 4:00PM Ezio Auditore da Firenze said

Posted: Mar 30th 2011 4:09PM Vampettrelli said

Posted: Mar 30th 2011 4:27PM Liquidfingers said

Posted: Mar 30th 2011 4:41PM Ezio Auditore da Firenze said

Posted: Mar 30th 2011 5:12PM AntiVillian said

Posted: Mar 30th 2011 6:10PM DVMGames Richard said

Posted: Apr 1st 2011 7:19PM Liquidfingers said

Posted: Mar 30th 2011 2:48PM KungFuChaosNinja said

Posted: Mar 30th 2011 2:50PM Jack Kevorkian said

Posted: Mar 30th 2011 2:52PM baby sea tuna said

Posted: Mar 30th 2011 3:02PM Liquidfingers said

Posted: Mar 30th 2011 3:49PM Wiizer said

Posted: Mar 31st 2011 12:31AM AntiVillian said

Posted: Mar 30th 2011 3:07PM Lerkero said

Posted: Mar 30th 2011 3:08PM VeeDeeVee said

Posted: Mar 30th 2011 3:18PM RMM said

Posted: Mar 30th 2011 3:22PM CaptainProtonX said

Posted: Mar 30th 2011 3:39PM swooded said

Posted: Mar 30th 2011 3:48PM xbashingx said

Posted: Mar 30th 2011 4:01PM baby sea tuna said

Posted: Mar 30th 2011 3:50PM Wiizer said

Posted: Mar 30th 2011 4:15PM goat biscuit said

Posted: Mar 30th 2011 5:22PM Jack Kevorkian said

Posted: Mar 30th 2011 6:01PM goat biscuit said

Posted: Mar 30th 2011 4:32PM lazurus said

Posted: Mar 30th 2011 4:35PM Imperial said

Posted: Mar 30th 2011 5:21PM Jack Kevorkian said

Posted: Mar 30th 2011 4:45PM ChomskyKnows said

Posted: Mar 30th 2011 5:33PM MassDragonEffectAge said

Posted: Mar 30th 2011 5:37PM CrashTunoku said

Posted: Mar 30th 2011 6:03PM huffhuffhuff said

Posted: Mar 30th 2011 8:41PM EvoHelix said

Cthulhu Saves the World Info

Description

MSRP

Release Date

Genre

Developer

Publisher

Rating

Breaking News

WB conjures up Harry Potter for Kinect this fall

Featured Stories

Silver Lining: I Am Alive's unfeeling world

Game Of Thrones and the paradoxes of adaptation

Super Joystiq Podcast 004: 38 Studios meltdown, Gravity Rush, Civilization 5: Gods & Kings, Dragon's Dogma

Engadget

TUAW

Massively

WoW