I’ll say it upfront: I’m a big fan of field & participant observation. I know, I know, getting people in the lab, under controlled conditions, with the latest and greatest observation technology seems “scientific”. Yet, ask yourself how much it changes the very thing you’re trying to observe. I’m not concerned about the user feeling uncomfortable or under pressure. Any good professional should be able make the user feel comfortable in a lab usability or user-centered session. No, what concerns me is that the observation lab is like a giant filter. It keeps out so many things that influence the context in which a device or software is used — the very context that, by definition, determines the usability of your system! Figuratively said, the lab filters out daily life itself: the myriad distractions, the wasp flying in your peripheral vision while you try to focus on the screen, the tools you keep always in your belt but that security did not let you take inside the building to the observation room. I was observing a firefighter once for our emergency management project. He pulled off his gloves and strained them. Out came what looked like a liter of water. Good luck trying to replicate that in the lab.
This is what James Chadley over at UX Mag says on the subject (worth reading in full by the way):
But — you will say — doesn’t it cost more? Not necessarily. But even when it does, ask yourself: doesn’t a failed product launch cost even more? Even if you are still unconvinced, do just one or two field and participant observation sessions in the beginning. They will be enlightening, humbling, eye-opening, and they [tweetherder text=”One or two participant observation sessions will kick you so hard towards the right direction that the inertia will keep you from making mistakes in the lab sessions later”]will kick you so hard towards the right direction that the inertia will keep you from making mistakes in the lab sessions later[/tweetherder]. Trust me on this. Been there, done that.