| Mail |
You might also like: WoW Insider, Massively, and more

Reader Comments (29)

Posted: Mar 11th 2010 7:47PM Coldplay619 said

  • 2 hearts
  • Report
Soooooo....he played lots of Zelda?
Reply

Posted: Mar 11th 2010 11:26PM freaparn said

  • 2 hearts
  • Report
Hey! HEY! He also looked at a lot of Warcraft art books. Don't besmirch the man's efforts!

But seriously, if they can keep putting out solid, entertaining games like Darksiders, I don't care how much inspiration they take from existing franchises. Everyone who creates is standing on the shoulders of those who came before them.
Reply

Posted: Mar 12th 2010 12:19AM TheDarkWayne said

  • 2 hearts
  • Report
it's more like WoW took a look at Joe Madreuria's art books, as in, comic books from the 90s
Reply

Posted: Mar 11th 2010 7:49PM DevilSei said

  • 2 hearts
  • Report
He's saying that the focus testing will create a game that, while having something for everyone, has nothing that anyone will really remember or care about.

Don't really know if I agree with him completely or not, but I can understand why he feels that way.
Reply

Posted: Mar 11th 2010 7:51PM MrAlex said

  • 2 hearts
  • Report
As in, you end up with a generic game that isn't 'the best' at anything.

Like Darksiders (I kid, that's just my opinion of it)
Reply

Posted: Mar 11th 2010 7:52PM Yuccadude said

  • 2 hearts
  • Report
Jack of all trades, master of none.
Reply

Posted: Mar 11th 2010 7:54PM Kif said

  • 3 hearts
  • Report
The reason behind the failing of these focus groups to provide results should be pretty obvious

Oh, look, kids are buying lots of Harry Potter: Kids must like Wizards!
*Cues line-up of shitty wizards books*
Gee, this isn't selling like Harry Potter...
Reply

Posted: Mar 11th 2010 8:06PM Courtney said

  • 3 hearts
  • Report
They funny thing is that every industry has plenty of evidence that this is true, and yet so many projects still rely on focus testing and consumer research to fine tune media. A classic TV example is All in the Family, one of the all time great sitcoms. Early focus testing showed that people hated Archie Bunker, and yet it went on to be a ridiculous success because the creators ignored what the consumer research said.
Reply

Posted: Mar 11th 2010 8:09PM Frellwit said

  • 2 hearts
  • Report
OT: GAH. It's been a half hour, this is ridiculous. Why won't my picture show up? It's a 64x64, 8 KB JPEG. I really don't know what's wrong.
Reply

Posted: Mar 11th 2010 8:24PM Frellwit said

  • 2 hearts
  • Report
Yeah, I've tried several different images, all at 64x64. Been at this for nearly four hours, 30 minutes with the last upload. It's nothing big, but I've yet to encounter someone else without a pic. That being said, I even went as far as saving someone else's pic and uploading it. No dice.
Reply

Posted: Mar 11th 2010 8:39PM Granger said

  • 2 hearts
  • Report
Try duplicating the image and renaming the second one. Then upload back and forth on two DIFFERENT browsers, that always works for me.
Reply

Posted: Mar 11th 2010 9:04PM IslandLife said

  • 2 hearts
  • Report
Frellwit, you're not alone. :(
Reply

Posted: Mar 11th 2010 10:43PM Frellwit said

  • 2 hearts
  • Report
I just tried uploading an image in Firefox, it seems to have worked. I'm just waiting for it to update the posts. *crosses fingers*
Reply

Posted: Mar 12th 2010 12:01AM Frellwit said

  • 2 hearts
  • Report
S-U-C-C-E-S-S, that's the way you spell success!
Reply

Posted: Mar 11th 2010 8:12PM Haggard said

  • 2 hearts
  • Report
I think it can ruin creativity in the game's broader vision for sure. But some games could use a few people to point to one particular aspect of the gameplay that doesn't make sense or is particularly frustrating.
Reply

Posted: Mar 12th 2010 12:33AM seishino said

  • 2.5 hearts
  • Report
As someone who has done player research for the past 5 years on some successful titles, parent is dead on. Usability and player research will let you know where people are misunderstanding things that you thought would be clear. It will tell you where your level design is too difficult (or too easy). And it will bring out all of the ugly stuff in the closet that you should probably fix before shipping. If you don't do an appropriate amount of user research during production, you will wish you had once the reviews start rolling in.

What it *won't* do for you is give you creative ideas or solutions to problems. User research won't tell you if the hero should be an alcoholic father or a widowed boy of seventeen. It won't tell you how much content should be in a game, or even if people realistically are going to buy it (though they'll tell you if they really won't). And, frustratingly enough, it won't give you solutions to the problems that it uncovers.

People who rely upon user research to make creative decisions for games usually come from toothbrush and fizzy water sales, and don't understand what the tool is for. They, understandably, have the instinct to satisfy the wants of their customers. A creative professional, on the other hand, has to shock the customer into wanting something they didn't know existed. It's a form of magic. A magician can't ask the crowd what they want to see, then show it to them. He has to figure out something original that the crowd doesn't expect, and dazzle them with it.
Reply

Posted: Mar 11th 2010 8:22PM Mr Bungo said

  • 2 hearts
  • Report
I work in the film biz, which is another creative industry that's swamped with market research and "wide demographic" thinking (if it can be called that). Relying completely on market data and trends and such will ensure that the project, whether it be film or a game, has most if not all of its quirks and spark removed. It may then be "pleasant" for a larger number of folks, but will it be at all memorable? This isn't to say that focus testing and such is entirely without merit. Plenty of films and games have been improved when test audiences pointed out flaws that the creators were too close to the project to see. But any such feedback needs to be assessed and weighed against the intent of the piece. Slavishly following uninformed or clueless feedback without regard to what you're actually trying to make will doom you to mediocrity, if not worse. If more executives in all the creative industries shared Bilson's outlook, music, games, TV and movies would all be a lot more interesting and fulfilling.
Reply

Posted: Mar 11th 2010 9:22PM onan said

  • 2 hearts
  • Report
THQ may have wanted to focus test the title at least. I own Darksiders and I still forget what it's called.
Reply

Posted: Mar 11th 2010 9:23PM Rocktave said

  • 2 hearts
  • Report
It's unheard of to have an executive say things like this. They're usually too hung up on control, while running a company into the ground with their business-school training, thinking that creating a culture of fear and paranoia among their employees will make them more productive.

It's good to see that at least one has a clue.
Reply

Posted: Mar 11th 2010 9:35PM bm111 said

  • 3 hearts
  • Report
"Inspiration comes from one place and research and focus testing will usually destroy it. It will usually create 'bland' and create something for everyone and really nothing for anyone,"


Oh god finally someone who comes out and says it. Game design seems to revolve around appeasing as many whiners as possible now. The day every developer/publisher adopts this mantra is the day I'll seriously get back into gaming.
Reply

Posted: Mar 12th 2010 12:09AM xnowimcoolx said

  • 2.5 hearts
  • Report
that mantra, and this one:

"If you need to do that kind of research you must not be playing games yourself"

Some games feel like the people who made them don't actually play video games!
Reply

Posted: Mar 11th 2010 10:00PM KillTheApex said

  • 2.5 hearts
  • Report
Exactly. Focus testing is bad, bad, bad. It is the path of those who are creatively bankrupt.
Reply

Posted: Mar 11th 2010 10:36PM Amnesiasoft said

  • 2.5 hearts
  • Report
THQ, if you're trying to become my favorite game publisher, you're so far on the right track that I don't believe you exist.
Reply

Posted: Mar 11th 2010 10:40PM (Unverified) said

  • 1 heart
  • Report
I regret buying Darksiders after reading this.
Reply

Posted: Mar 11th 2010 10:46PM Sage009 said

  • 2.5 hearts
  • Report
I don't get it
Reply

Posted: Mar 11th 2010 11:05PM sigma8 said

  • 2 hearts
  • Report
Well, if games are art, then I'd say he's right. Nobody makes good art by committee.
Reply

Posted: Mar 11th 2010 11:36PM Author X said

  • 2 hearts
  • Report
While I agree that creativity is too easily squashed by trying to please everyone, or even trying to please anyone but the artist, Valve makes some pretty great games using user testing. I recall interesting notes in the developer commentary for Portal about how they improved the usability and the 'learning' section of the game greatly by figuring out how users acted while learning the controls, that they were reluctant to look up, etc. Plus, they mention in the HL2 commentary that when someone heard a poison headcrab, they'd usually freak out, drop everything, and try to find and kill it while ignoring other enemies in the area. They used this knowledge to set traps, using that behavior to distract players and increase tension.

So, as a creative endeavor, a focused, created-driven vision is the way to go. But as an interactive, user-focused experience, the gameplay and design can be improved by listening to your userbase (although, it may just be better to observe their behavior than ask their opinions, :P )
Reply

Posted: Mar 12th 2010 8:21AM Daverator said

  • 2.5 hearts
  • Report
I think there is some confusion over focus testing vs QA testing. Focus testing is basically getting together random nobodies, and finding out what they say they want. If you ever saw the Simpsons episode where they create the cool dog character I think you will see the idea in a nutshell. Draining an ambiguous concept from a group of people might make something they "like" but it says nothing about the actual depth of the creation.

QA testing however is letting people play the game or parts of it to recognize flaws, faults, unintuitive design, confusing levels, unclear messages. That sort of thing, every game ever created needs tons of QA testing (if it gets it or not is another story).

Basically Focus testing would be done long before the creation of a game, where QA testing is usually towards the end of the game's creation.
Reply

Posted: Mar 12th 2010 11:55AM joeboosauce said

  • 2 hearts
  • Report
I can see exactly what he is saying. This is true across all media. Artists and creators are neutered by execs who only want what is most popular and will net the most money. But these focus groups, besides stifling creativity, can and have backfired. You cannot make much media that is excellent for EVERYONE. That is reality. Niche markets are the way to go.
Reply
Sorry, you must be logged in to leave a comment.

Featured Stories

Engadget

TUAW

Massively

WoW