What other types of guerrilla testing have people tried before? This sounds like a fairly witty idea for gathering user interaction information, but I wonder if it's actually functional.
Thanks, that's good to know. I know someone once mentioned using Mechanical Turk and that those sometimes ended up being their most loyal users, but I cannot recall the context. Has anyone used these services and what results did you see?
I've tried Feedback Army and was disappointed w/ the results. Not that it isn't a cool idea, but I think the give and take of a good usability test requires that it be done in person. If you watch usability expert Steve Krug doing a demo test (http://network.businessofsoftware.org/video/steve-krug-on-th...), you'll see what I mean.
I recently saw a blog post where someone wrote about the feedback they received and they weren't happy. The same person then revised their questions and tried a more task oriented approach and they were much happier.
Then again, Feedback Army is just a usability testing tool. The right one for the job. I think it's best when a breadth of ideas are needed quickly or you need some ideas about where to start improving your site.
You should use other methods to do testing that requires a lot of depth from one or two people.
Here are a few more posts where folks have written up their experiences:
kartme posted a good post about changes they made with usability testing. they mentioned uservoice which is on turk if I remember, they mentioned doing a followup post explaining more about what the did during testing.
I like their approach using a variety of tools at different stages of product development to get the job done.
I think something our field may benefit from is a guide that explains how to use the different usability services and at what point in the project cycle they should be used.
Anything run by a 3rd party strikes me as immediately questionable for accurate testing. They necessarily use people more geared towards using applications, as otherwise they wouldn't be there in the first place, getting paid to do something that most people wouldn't touch. They can certainly provide extensively useful insight, so running through a UI-review company is still a good idea, but can there really be a guerrilla testing company? The idea of guerrilla testing is to get totally untainted results, isn't it?
