User experience professionals need to get clear about what’s wrong about focus groups, and what’s so attractive about them. The March (2013) BayCHI monthly program abstract ends with the words, “why you should never, ever hold a focus group.”  Now we’ve got a timely discussion.

Later this week, I’ll be adding additional information about what kinds of useful results you can expect to get from a focus group that uses activities (vs one that follows a script narrowly), how these newer kinds of focus groups are changing the face of civic engagement, what kinds of questions will get useful answers, and how to recruit people to provide meaningful answers.

"Prototyping with Junk" at UC Berkeley

Award for Junk

Did you catch the New York Times word of the year (WOTY) for 2010? It’s junk! The editors of the “On Language” column chose this word as representative of the zeitgeist of the past year. The honor acknowledges the basic meaning of rubbish or trash, debris or detritus, as well as extended meanings: From junk bonds (devalued securities) to junk food (nutritionally empty), to junk shot (stuffing debris and mud into BP’s leaking gusher in the Gulf). Their award called out one euphemistic sense, the male genitalia. The TSA started full body pat-downs in 2010, as an alternative to scanning, and air travelers gave warning, “Don’t touch my junk.”

Next month I’ll be hosting the activity “Prototyping with Junk” at Interaction Design 2011 (IxD11) in Boulder, Colorado. As I’ve previously written, I occasionally get pushback from people who feel junk is not appropriate language for professional settings. Or, perhaps their reaction is that junk — the actual stuff — is not for the workplace. Now I can respond to those objections with the citation of WOTY2010. I’m hip!

JunkFest (2007)

And I’m in good company: Bernie DeKoven, author of Junkyard Games, and funsmith extraordinaire, recently shared a video from 2007 celebrating Junkyard Sports. Here’s an news report from JunkFest in Redondo Beach [warning: narration lacks captions]

Kinetic sculpture with junk

Another wonderful example of how using apparent junk (PVC pipes plus a bunch empty plastic bottles, and other stuff) can turn into something magical: see how kinetic sculptor Theo Jansen simulates animals walking, powered by the wind near the sea.
[warning: narration lacks captions]

Read more at Talking Science: Dance of the Strandbeests, the BBC article about this project.

Jansen’s example shows how prototypes evolve into working “products” or art, depending on your perspective. His process of successive refinements suggest agility: at each juncture, he stops and tests his creatures, from human-propulsion of walking machines to wind-propulsion (which simulates self-propulsion), and all from vernacular materials with clever engineering.

Learning with junk

We used Prototyping with Junk at ACM CHI2004 (in Vienna, Austria), when we challenged the participants in the pre-conference design collaboration to create a product for elders. As I’ve written for interactions magazine special issue on prototyping, this is an opportunity for creativity and fun in a social context. And, it’s yet-another communication tool for your collaborative design kit.

More recently, students at UC Berkeley’s iSchool engaged in Prototyping with Junk as one among many prototyping techniques they experimented with this past fall. You can see still images and a few short movies. Notice all those smiles!

Eager to meet the group in Boulder. I’ll bring one of the several design challenges I’m currently mulling over, …and plenty of junk.

Research methods are divided into various categories. One familiar distinction is the qualitative and quantitative. Sometimes we think of these categories as distinct types of research:  either-or. A research project collects qualitative data (e.g., an interview with questions that invite open-ended responses) or quantitative data (e.g., a survey with multiple-choice, true-false, or scalar responses). Or is it that the analysis uses qualitative methods (affinity diagrams) or quantitative methods (statistical packages)?

Christian Rohrer suggests that qualitative research methods are more likely to involve direct engagement with users, while quantitative are more likely to involve indirect techniques, but that these methods live along a continuum:  more or less, not either-or.

The diagram below maps out Rohrer’s ideas about this continuum.  (Click the image to enlarge.)

Christian Rohrer's Landscape of user research methods

Registration form entries are subject to quantitative analysis

Here’s a case where my colleague and I interrogated a database of responses from roughly 10,000 people using simple quantitative methods. While it’s not data mining with datasets the size that Google or Yahoo! might have, it certainly is a type of data mining – see the upper right corner of the diagram.

Our client’s website offered a weekly newsletter for people involved in affordable housing and community development. For research purposes, we had access to the information from registration forms of prospective subscribers. The form contained at least 4 fields of interest to us, as we confirmed (or revised) our ideas about who was using the original website (Knowledgeplex.org) and its companion site (Dataplace.org) combining geographical data with statistics collected from the US federal government. Besides giving an email address, people who registered were asked to include their job title and the name of their employer, and to select from a list of fixed roles which role(s) fit them best.

We were able to make inferences about users and uses from this large database of qualitative and loosely structured information, in conjunction with other more familiar (qualitative) user research methods (interviews, usability studies). For example, we could show that Researchers (an amalgam of several job titles and roles) were more likely to register through Dataplace, the site with interactive maps and statistics, than at Knowledgeplex, the site with news and policy updates.

We described the work for an audience of peer researchers, at a workshop at CHI 2007. Now some 3 years later, the funding for Knowledgeplex and Dataplace has ended, development has stopped, the sites have been taken down, and evidence of our contributions to this work are even more difficult to document and track. The conference report contains less detail than the report we provided to the client. And yet it gives the flavor of turning qualitative data (job title, organization name) into quantifiable results (Researcher registrations are growing more quickly on one site than the other), confirming that we were indeed headed in the right direction in attracting the audience we hoped to reach for each of these companion sites.

Christian Rohrer’s Landscape of User Research Methods: Slide 20 from the (.pdf) of Rohrer’s presentation to BayCHI.  His blog post about this experience includes a link to the full set of slides: Christian’s Greatest Hits

CHI 2007 Workshop Website:  Imaging the City:  Exploring the Practices & Technologies of Representing the Urban Environment in Human-Computer Interaction

Our contribution:  CHi2007DataPlace-workshop-3 (.pdf download, 336K) (also available from the workshop website)