Archive for the 'research' Category

Paul Krugman’s Rules of Research

From his Nobel talk slides:

picture-1.png

The meaning of the first one was not immediately apparent to me, so I found a longer version of the rules, where Krugman explains: “Pay attention to what intelligent people are saying, even if they do not have your customs or speak your analytical language.”

Applies to many, many things.

Post-Deep Blue Pick-Me-Up

Kevin Kelly has an invigorating post about our the inevitable increase in our ignorance. When I saw the title “The Expansion of Ignorance”, I had a curmudgeonly joy at reading about how stupid we’re allowing ourselves to become. Yesterday, I listened to a series of Open Source interviews with Harold Bloom (while playing my Rogue alt on WoW, no less). He railed against the ’school of resentment’, lamented the celebration of crap books, condemned the loss of memorization, etc. Cocktail Party Physics had a post about how Sarah Palin represents a celebration of dumb and connecting it to bigger, scarier trends:

Despite the Palin-centric focus, this is not meant to be a political post; rather, her candidacy epitomizes one of our most fundamental failings as a nation. I’m talking about the triumph of mediocrity, of settling for “good enough,” in America. No wonder our country is in a shambles, teetering on the edge of economic ruin and losing our historical edge in technological innovation. No wonder we’re lagging so far behind other developed countries in educational testing scores, when we demand so little of even the highest offices of our land.

In a recent post here about how fivethirtyeight.com was uncovering basic 101 weaknesses in long-standing polls, I was surprised to find out how pissy I was about the media’s and larger public’s inability to figure out this problem for themselves.

So I was looking forward to a bilious post from Kevin Kelly, which, now that I think of it, is silly. He’s not the bilious type. What the post is about is how, despite all of the knowledge we’ve acquired, we’re not really getting close to knowing it all or being done with science. He points out that, if every answer raises to more questions, our pursuit of truth is creating more ignorance than knowledge. Chart:

picture-106.jpg

As one who was depressed that Deep Blue’s chess victory over Garry Kasparov was turning things I once saw as art and as having mystery into simple riddles, this was a big pick me up. There’s something very Jean-Luc Picard’s love of discovery and surprise in all this.

Good line from NYT Book Review

“The plural of anecdote is not data”
- from a review of Friedman’s new book

Data and Original Thinking

Nice quotation of Darwin in Glut recommended by @mokindo:

I am a firm believer that without speculation there is no good and original observation

Nothing New: My Top 5 Interactive Experiences

It seems like everything I read or think about interactive eventually, but quickly, zooms into next steps: how can we use this for marketing? how will this help us talk to our customers better?

I’ve almost forgotten the fun stuff that made me think this was an amazing medium, so I put together a list of the top 5 interactive things I’ve experienced over the years. These were more like interactive epiphanies, things that made me think this was a new medium with power. There are millions of little moments I can get all Chris Farley “that was cool” about, but these are ones that showed new possibilities.
Beethoven’s 9th

An educational CD-ROM made in hypercard by Robert Winter. It presents the 9th as the fulcrum to the romantic era musically, culturally, philosophically, and within Beethoven’s career. Using clickable pieces of music, often synched with scores, as well as photos, sketches, and active maps, the CD-ROM explains sonata form, the classical style, and development of themes. It also has an interactive score which allows you to listen to the symphony, while watching the score, all the while displaying comments and which section (development, false cadence, recapitulation, etc.) of the symphony you’re in. (Interesting article about the title and its place in the history of books.)

beethoven9screenshot.jpg

sonataform.gif

Final Fantasy 7

I can’t say it’s the best FF (I have only played a few), but it is the best game I have ever played. The story was one I actually followed, I was genuinely sad when Aerith died (I mean, it’s f*’ed up how bummed I was, I think I gasped), the combat system was clever and required tactics and strategy which I was proud of, and I still remember the characters.

aerith.jpg

Journeyman Project Turbo

This is a strange choice since the game was kind of crap — from that era when interactive stories were getting full of themselves. It was a time travel game, where you have to go and retrieve things from different eras to prevent oh, I don’t know, an exponentially growing rift in the time-space continuum that would destroy this universe and maybe others as well. What was cool, and truly memorable about it, was that one of the time settings was Leonardo’s workshop. You could wander around it at night and it was absolutely gorgeous. It was one of the first games to do sound design with stereo headphones in mind, so the ambient sounds and the music added to the immersion.

Fantasy Baseball Draft

Real-time fantasy baseball drafts are amazing fun. (Real-time as opposed to the turn-based email drafts, which I’m doing this year). Sitting around waiting for the draft to begin and talking to people, watching bots pick players for people who haven’t shown up yet, scrambling to figure out your next pick (or next two picks if you’re at the end of the snake), back-channel chatter. The funnest thing I’ve done on the web. I stayed in a league for three years too many just to experience the fun of that draft.

I actually had to stop at 4, cuz I already had two games and all I could think of were other games. It also highlighted that I haven’t had any mind-blowing experiences in the last three years, which was kind of sad. Flickr comes close, but I was looking for things that I still talked about years later and remembered the epiphanous flash that said, “things are different.”

“Biology gives way to chemistry”, or Number-Crunching Reductionism

Came across a line in Omnivore’s Dilemma that captures some of my frustration with super-crunching and marketing models:

To reduce [a complex agricultural system under discussion in the book] represented the scientific method at its worst. Complex qualities are reduced to simple quantities; biology gives way to chemistry . . . that method can only deal with one or two variables at a time. The problem is that once science has reduced a complex phenomenon to a couple of variables, however important they may be, the natural tendency is to overlook everything else, to assume that what you can measure is all there is, or at least all that really matters. When we mistake what we can know for all there is to know, a healthy appreciation of one’s ignorance in the face of a mystery . . . gives way to the hubris that we can treat nature as a machine.

I love the idea of biology giving way to chemistry: systemic thinking giving way to engineering problems. How often do designers struggle against models of people that focus on two factors to the exclusion of everything else, that reduce people to the actions we want them to take?

Stick a pin in it and it dies.

Steve Jobs: Limits of Customer Research

Two Jobs lines about the limits , or the limiting effects of, customer research:

It took us three years to build the NeXT computer. If we’d given customers what they said they wanted, we’d have built a computer they’d have been happy with a year after we spoke to them - not something they’d want now.

You can’t just ask customers what they want and then try to give that to them. By the time you get it built, they’ll want something new.

Stick a pin in it and it dies

butterflylive.jpg

In my first week at college, I picked up a great line from my micro-econ professor. This was in the 80s, a decade before Freakonomics, but at the beginning of the discipline’s awareness of it mathematicization. The professor said something like this:

You’ll learn theorems and laws in this class, and you’ll pick up some math. But, when you write your papers and take your exams, I want you to always always always tell me: “who is doing what to whom and why”. Economics that doesn’t do that is useless.

I was at a brainstorm a little while ago discussing strategies for a client. Some questions emerged about how we should think about the customer’s buying mode: are we looking to upsell within a store, competing with a brand that’s only available at another? are we trying to generate demand for the activity that requires our product or speak to people already committed to that activity? Pretty standard sets of questions.

What I noticed, though, was a rush to put labels on ideas and capture the dynamic within an existing, perhaps widely known concept (value chain, purchase cycle, influencer strategy). The words were all useful, but they seemed to dampen the energy of the conversation - they didn’t tell us who was doing what to whom (or, more importantly for marketers with whom) or offer theories of why.

I suggested that we should avoid putting conceptual labels on dynamics during a brainstorm. That we should stick to people dynamics — getting inside people’s heads would get us to better ideas. Being inside people’s heads would give us a better handle on whether the idea was good or not. One person, who had been dropping jargon on the conversation suggested he was doing so “to put a pin in the thinking.”

0313-butterfly-decor-l.jpg

Aha! Putting pins in something, like a butterfly, kills the subject! Putting a pin in something makes it static, stops it from its natural movements, makes it less rich.

It doesn’t sound consult-y, but if our marketing models aren’t helping to explain who is doing what to whom (and with whom) and why, then they should be kept out of design discussions.

Grounding Abstract Methods in Design Needs

Two articles, once again from Todd Walker, highlight how research (or research-driven techniques) needs to be (re)-grounded in the needs of design.

The first, Design Meets Research from AIGA,  has a useful survey of leading testing techniques and provides some pros and cons about each of them.  In the middle of the piece is a paragraph that summarizes the key problem most designers have with research:

There is a group of brand consultants and cultural anthropologists alike that believe now that it is not the actual research itself that is the problem. It is rather about how research is often misused, what type of design concepts and stimulus are tested, and how data is analyzed that is most often at fault. When used correctly, research shouldn’t stifle creativity but rather offer designers stronger inspiration and focus.

They remind designers that there’s a critical interpretation phase that comes between research and design.  No one would disagree with that statement, but where it gets tricky is how people define interpretation and who participates in it.  In more than one work environment, interpretation meant a summary of major findings, was conducted by the strategy group or account lead, and somehow straight-lined to design recommendations.  (”Only 49% of respondents viewed element x favorably -> Replace element x or remove it.”)

The article hits some other high points:  know what you’re testing for; remember that testing is ultimately about better understanding a customer (heightening designer empathy with the audience) and not about having customers do design; ethnographic activities are still the best things for designers to do no matter what; research is an art not a science; interpretation is a joint activity between design and research.

The other article, Personas and the Role of Design Documentation has similar themes, but is more focused on personas.   Specifically, it focuses on the way in which most people go through personas as a deliverable that needs to be done, not as a tool with a purpose and communication goal.  Key point for the writer:

Personas are not documents, and they are not the result of a step-by-step method that automagically pops out convenient facsimiles of your users. Personas are actually the designer’s focused act of empathetic imagination, grounded in first-hand user knowledge.

The best part of the article is a distillation of lessons from Alan Cooper’s ‘origin of personas’ story (mythic in its grandeur, but true):

1. Cooper based his persona on a real person he’d actually met, talked with, and observed.
This was essential. He didn’t read about “Kathy” from a market survey, or from a persona document that a previous designer (or a separate “researcher” on a team) had written. He worked from primary experience, rather than re-using a some kind of user description from a different project.

2. Cooper didn’t start with a “method”—or especially not a “methodology”!
His approach was an intuitive act of design. It wasn’t a scientific gathering of requirements and coolly transposing them into a grid of capabilities. It came from the passionate need of a designer to really understand the user—putting on the skin of another person.

3. The persona wasn’t a document. Rather, it was the activity of empathetic role-play.
Cooper was telling himself a story, and embodying that story as he told it. The persona was in the designer, not on paper. If Cooper created a document, it would’ve been a description of the persona, not the persona itself. Most of us, however, tend to think of the document—the paper or slide with the smiling picture and smattering of personal detail—as the persona, as if creating the document is the whole point.

4. Cooper was doing this in his “spare time,” away from the system, away from the cubicle.
His slow computer was serendipitous—it unwittingly gave him the excuse to wander, breathe and ruminate. Hardly the model of corporate efficiency. Getting away from the office and the computer screen were essential to arriving at his design insights. Yet, how often do you see design methods that tell you to get away from the office, walk around outside and talk to yourself?

5. His persona gained clarity by focusing on a particular person—”Kathy”.
I wonder how much more effective our personas would be if we started with a single, actual person as the model, and were rigorous about adding other characteristics—sticking only to things we’d really observed from our users. Starting with a composite, it’s too easy to cherry-pick bits and pieces from them to make a Frankenstein Persona that better fits our preconceptions.

There are, of course, challenges embodied in these lessons.  Grounding a persona in one person could lead to endless ratholes about which one person, and number wonks will immediately jump all over the “method”/”methodology” point.  But the key point is that personas are ways of creating empathy with the user, of getting us (our team and clients and other stakeholders) out of our own heads and into someone else’s, of creating conversations with potential customers and users.