Rapid Prototyping: Why a Rough Version Beats a Perfect Plan

Rapid prototyping has two meanings. In engineering and manufacturing, it is a group of techniques for making a physical part quickly from a computer model, most often by 3D printing, and the first such methods became available in 1987.¹ In everyday design and problem solving, which is the sense this guide covers, rapid prototyping means building a quick, cheap, rough version of an idea, a process or a product, letting people try it, and learning what to change before you commit to the real thing.

In a 2009 Stanford experiment, students who could test and rebuild an egg-drop design while they worked ended up with final designs that survived higher drops than those of students given the same time to build a single version they could not test.² A rough version earns its keep when you build it in five moves:

1. Name the one question it must answer.
2. Pick the lowest fidelity that can answer it, from a sketch up to a physical model.
3. Cap the time each round may take.
4. Make two or three versions rather than one.
5. Say plainly what each version does not test.

Prefer to watch? The video covers the same ground as this article.

VideoRapid Prototyping: Why a Rough Version Beats a Perfect PlanWiserHours

What is rapid prototyping? A cheap version built to learn from

Rapid prototyping, in the design sense, is the habit of turning an idea into something people can see or try, cheaply and soon, learning from how they use it, and then building the next version. The manufacturing sense, fast fabrication of parts, is one way to make a physical prototype, not the whole idea. Wikipedia’s overview lists stereolithography, laser sintering, CNC machining and injection molding among the manufacturing methods, and notes that the term is also used in software engineering.¹

The design sense is broader. Stephanie Houde and Charles Hill, then at Apple Computer, defined a prototype in 1997 as any representation of a design idea, regardless of medium. Their example is a brick: if it stands in for the weight and scale of a future device, it is a prototype of that device.³

Take a team that wants a new way to book meeting rooms. A sign-up sheet taped to two doors for a week is already a prototype: it tests whether people will book at all before anyone buys software. Prototyping is also the fourth stage of design thinking’s five stages; this guide goes deeper on that one stage.

Why a rough version beats a perfect plan

A rough version beats a perfect plan because testing turns up flaws and limits that are hard to foresee from a desk. A 2009 experiment by Steven Dow, Kate Heddleston and Scott Klemmer at Stanford University tested this by letting some people test and rebuild their design and stopping others from testing.

The study

Limited evidence

One egg or a full carton? 28 students and a drop test

Twenty-eight Stanford students each had 25 minutes to design a vessel that would protect a raw egg from a fall, then 15 minutes to build a final version from fresh materials. One group, assigned at random, got a carton of eggs and was encouraged to drop-test along the way; the other had one egg and could not test. The testers’ final vessels survived drops from an average of 6.1 feet, against 3.3 feet for those who built once, and testers new to the exercise did as well as non-testers who had done it before.²

The authors give two explanations, which can both hold: the testers found more flaws and limits and tried more new ideas, while the others could only guess how their design would fare; and the testers got better at building by making the same thing several times. The testers’ confidence also rose over the session, while the non-testers’ did not. The limits are size and setting: one short session, with students working alone on a task with a clear goal, whereas real design problems run for days or months and usually involve teams, as the authors note.²

A popular workshop story points the same way, though it is an anecdote. In a 2010 TED talk, Tom Wujec described the marshmallow challenge, which he credits to Peter Skillman: teams build the tallest tower they can from dry spaghetti, tape and string, with a marshmallow on top. He said recent business school graduates are among the worst at it and kindergarten children do better than most adults. In his telling, the children build one version after another with the marshmallow on top, while business students look for the single right plan and add the marshmallow at the end, when there is no time left to fix a collapse.⁴ His observations come from his own workshops, not a published study, so treat them as a teaching story.

Put the riskiest part on top early, as the children do with the marshmallow. If the room-booking idea depends on people cancelling rooms they no longer need, test that before anyone designs a screen.

Start with the question the prototype must answer

Every prototype should answer one named question. Stanford’s d.school, in its 2010 process guide, tells teams to identify what each prototype tests.⁵ Houde and Hill’s 1997 model sorts the possible questions into three kinds: the role a thing plays in someone’s life, how it looks and feels, and how it would work.³

They call choosing a prototype’s focus the art of identifying the most important open design questions. If a thing would play a new role in people’s lives, the key questions may be about that role; if the role is familiar but offered in a new way, about look and feel; if it rests on a new technique, about how it works, which usually means building something that works.³

the best prototype is one that, in the simplest and most efficient way, makes the possibilities and limitations of a design idea visible and measurable

Lim and colleagues describe prototypes as filters: a good one leaves out everything except the qualities you want to study, without distorting your picture of the whole.⁶ Applied to the room-booking project, three questions call for three different prototypes. Will people book at all, rather than just walking in? A paper sign-up sheet answers that. Can they book in under a minute? Paper screens can show it. Can the calendar system show live free rooms on a tablet? Only a working technical test can.

Write that question on the prototype itself before you build, and judge it only on whether it answered.

Types of rapid prototyping: four levels of fidelity

Prototypes are usually sorted by fidelity, meaning how close they are to the finished thing. Four levels cover most everyday work: a sketch, a paper prototype, a clickable mock-up and a physical model. Higher fidelity usually takes more time and money to make.⁷

1. Sketch: what the idea is for, and whether it makes sense to someone else
2. Paper prototype: whether people can follow the steps, with a person swapping the pages
3. Clickable mock-up: how the flow looks and feels on a real screen, with clicks you can record
4. Physical model: size, weight and scale, using a stand-in for the real object

For the room-booking team, the four levels might be a sketch of a door card, paper booking screens, the same screens linked in a slide tool, and a tablet taped beside a door. Start at the lowest level that can answer your current question, and move up only when that question is settled.

Low-fidelity versus high-fidelity prototypes

Low-fidelity prototypes are rough and quick, like sketches and paper screens; high-fidelity prototypes look and behave much like the finished product. In a 2002 experiment at UC Berkeley, sketched website prototypes uncovered about as many usability problems as polished ones.⁷

Miriam Walker, Leila Takayama and James Landay had 28 people carry out online-banking tasks on website prototypes, either sketched or built in HTML, shown on paper or on screen. Rough and polished versions surfaced similar numbers of problems, of similar severity, and paper versus screen made no clear difference either, although people commented more on the screen versions. The authors suggest that task-based testing keeps people’s attention on the tasks and the structure of a site rather than on visual details.⁷

Myth

People can't give useful feedback on a rough prototype.

Fact

In a 2002 Berkeley test of website prototypes, rough sketches surfaced about as many usability problems as polished versions.

It is one small experiment on websites, though the authors call it consistent with earlier comparisons.⁷ If your question is whether people can find and finish a task, a paper version will often tell you as much as a finished-looking screen.

Time-boxing: how long each round should take

Time-boxing means fixing in advance how long a round of prototyping will take and stopping when the time is up. Stanford’s d.school guide says early prototypes should take minutes and cost cents, and warns against spending so long on any one version that you become emotionally attached to it.⁵

A whole phase takes longer. IDEO.org’s Design Kit, a free online guide to human-centered design, says a rapid-prototyping phase with several rounds of testing can take anywhere between a few days and a few weeks, and that the learning goals narrow with each round.⁸

The reason to cap each round is that time spent makes ideas hard to drop. Dow and colleagues, in a 2010 paper, point to earlier research on the sunk cost effectsunk-cost effect: The pull to keep putting time, money or effort into something because of what has already been spent on it, rather than because of what it will bring from now on.Full entry in the glossary: people sometimes make poor choices to justify money or time they have already invested.⁹ Short rounds can feel uncomfortable, too. In the egg-drop study, some testers said they felt anxious at having to test so early and so often, while several who could not test were dissatisfied at not seeing how their design performed.²

Why to make two or three versions before asking for feedback

Making several rough versions before asking for feedback is one way to widen what you explore. Testing and rebuilding one design did not do that on its own in the 2009 egg-drop study: people in both groups explored a narrow range of designs, and several believed their idea was the only possibility.²

Dow and colleagues tested the alternative in 2010 with novice designers making web banner ads. Those who made several prototypes before getting feedback produced final ads that did better in a real online campaign and in expert ratings, and their prototypes were more varied than those of designers who got feedback after each one. This was a single small study of 33 people, funded by a design thinking research program.⁹ In an earlier 2009 technical report, the same team found no clear difference between the two approaches on an egg-drop task, so the benefit may depend on the task.¹⁰

The authors’ explanation is about investment. Refining one option can lead to fixation on it, and criticism then feels like a rebuke of your only idea; working on several shifts the investment to the process rather than to any one concept.⁹ For the room-booking team, that means three rough versions side by side, such as a paper sheet on the door, a shared online calendar and a booking card with a code, rather than one version polished three times.

When a rough version misleads

A rough version misleads when you ask it a question it cannot answer. Houde and Hill warn that prototypes are not self-explanatory, that looks can deceive, and that being clear about which parts of a prototype match the eventual product, and which do not, is a key part of prototyping well.³

Three traps follow. First, questions about whether something can be built usually need a working system: a paper screen showing live room availability says nothing about whether the calendar software can supply it. Second, audiences read finish as progress, and Houde and Hill note that finished-looking prototypes are often taken to mean the design is nearly done.³ Walker and colleagues add that designers sometimes switch to polished prototypes early because they expect clients to see rough ones as unprofessional. Third, rough and polished versions are not interchangeable for every purpose: in the Berkeley test, the kinds of problems people found differed between the sketched and the polished versions, even though the totals were similar.⁷

Before you show a prototype

• Name the one question it must answer
• Say out loud what it does not test
• Check the question is not about whether it can be built
• Bring more than one version
• Note what people do, not only what they say

Label every prototype with what it tests and what it does not, and tell the people who see it.

Prototype or MVP: what is the difference?

A prototype tests whether a design works for the people who will use it; a minimum viable product, or MVP, tests whether a business idea works with real customers. In Eric Ries’s 2009 definition, an MVP is whichever version of a new product yields a team the greatest validated learning about its customers for the least effort.¹¹

So the two differ in the question they answer, not in how rough they are: whether staff can use the booking screens is a prototype question; whether other companies would pay for the tool is an MVP question. For that kind of test, see how to test a business with a minimum viable product.

The bottom line

A rough version pays off when it is built to answer one clear question quickly. Before your next idea goes further, write that question down, build the lowest-fidelity version that can answer it, and show it to someone the same day.