Does good design really matter?

Because it’s so hard to measure the business value of good design, it naturally gets less attention from managers. It’s easier to watch things like visitors to a website, sales per day, rent and other costs.

Recently we got a chance to see how much it matters when we rolled out a redesign for our Freedback service. Here are some of the changes we made:

  • Reduced the number of steps required to create a form from four to three.

  • Removed list of forms on the left to reduce visual clutter.

  • Added breadcrumb navigation so it’s easier to see where you are.

  • Users now create forms first, then decide what happens when they’re submitted.

 
(Hold your mouse over the image to toggle
between the new and old versions)

Digg users:
It’s possible this mouse-over won’t work for the cache you’re seeing.
Click here to load the original page.


 

Results

To be honest, when we rolled out the new software we hoped to get lots of emails thanking us for the new interface.  Didn’t happen.  We did get a few phone calls, mostly from people reporting bugs, and by the way did we change something?

After squashing a few bugs all was well with the world.  We never did get much customer reaction, so we watched our graphs to see if we could see any change in how people used the software.

Number of forms created

Revenues

 
Thoughts and Conclusions

  • When things are easy to use, people use them more.
  • We didn’t see a dramatic increase in revenue from new users who had never seen the older version, but older accounts coming up for renewal were more likely to upgrade.  This agrees with previous experience from other system upgrades; existing users like to see software improving.
  • Hopefully this will translate into more links, reviews and word of mouth traffic, but that’s harder to correlate to this change because it takes time.

 
Update:

My wife knows much more than I do about graphing data (she’s an astronomer), and was nice enough to plot a graph using the raw data that better showed the results we were seeing.

This graph is showing daily Freedback revenue before and after the change (note the y-axis doesn’t start at zero).

  • The green dots are days.
  • The black line is a four-week moving average.
  • The 13% represents the difference between one month before and after the change. It held true for six weeks before and after as well.
  • Our weekly revenue cycle (lowest on weekends) results in a lot of scatter.

I hope this data is useful to other entrepreneurs wondering if they should invest in design, questions or comments welcome. I’ll keep posting graphs and results from tests we’re running – if you’re interested you can subscribe to our feed.

Advertisements

29 Comments on “Does good design really matter?

  1. While the 13% rise is very cool, and congrats, one crucial data point is missing… daily unique visitors before and after. If your uniques stayed about the same in the two periods, then there’s a much greater weight to the proof of good design being the driver of the form creation and revenue increases.

    But if uniques went up 15% in the same period, then that 13% increase would prove that the old design was more successful (though it’s possible some rise in uniques might be due to people recommending the new design more or the new design providing better SEO benefits).

    Without knowing if there was any change in unique visitors, it’s hard to evaluate the significance of that 13% and how much the new design contributed to it.

    How about it? Let’s see a graph of daily uniques overlaid on the other graphs.

  2. Hey Greg,

    Great point!

    Even more relevant is actual signups (account creations) over that time period since we didn’t make any changes to the external site.

    Signups actually fell slightly over this time period, so it’s not an increase due to that. I’ll add that graph later today.

    – Aaron

  3. I think the interesting thing is that you state significantly more of the revenue increase was driven by users who had seen both versions converting from the free option to a paid option than by newer users who never saw the old interface.

    Because you attribute some of that to “existing users like to see software improving,” it would be interesting to see if the increases hold over a longer term. It seems you’re on an upward trend, but once all the old users have seen that it’s improving, will you see a drop off when all those older users who are likely to be influenced by the upgrade have been influenced? If the increases hold past that point, then it shows that the design and process improvements are having an effect on all users, and the specific improvements are impacting growth.

  4. Definitely. Historically revenues after we make a change, then settle down at a higher point than they were, but not always as high as their peak.

    In general it’s hard to test over long periods because we’re always improving things, so the control is always improving. We’ve thought about splitting out a set of users to be a long-term control, but concluded that maintaining two codebases is a recipe for serious pain!

  5. Hey Aaron –

    I thought the previous version was intuitive, and I love the service, so I would have tolerated a really offensive design (not that you ever had one!).

    But I think the re-design indicates that it’s more accessible to newbies, and that’s a good thing.

  6. Thanks Shawn, glad you like it.

    If you have any suggestions for ways we could keep on improving it please let us know!

  7. Can you really determine if good design was effective simply by customer response? I think most of us working in the field of design realize that most feedback will be negative feedback — people don’t generally go out of their way to tell you that you did your job correctly or you made a product aesthetically pleasing and usable; that is the default expectation.

    Good design isn’t appreciated, that is the bottom line. Good design is expected. Positive design feedback typically comes from designers and negative design feedback typically comes from users.

    Your basis for evaluation is off — determine if you have a successful design is your number of user issues diminishes not if your number of compliments increases. Less users with less issues = success.

  8. Hey Martin,

    Thanks for your comment. I agree, you don’t really know anything based on what customers say to you.

    If you check out the article and graphs you’ll see that our evaluation of success was in how users’ behaviour changed, both in usage and upgrades (revenue) and not so much to do with what they said.

  9. Not to slam anyone but I found the statement “I would have tolerated a really offensive design” as a funny statement and here is why:

    As a designer and a having high standards, if sommeone is Tolerating a design then the design failed just as If you have to give instructions verbally or written the design failed.
    Sure there is some learning with any site to figure it out and utilize it’s capabilities, but here we can see stats to show how effective a GOOD inutitive design can be effective in the education process as well as usability for existing users allowing them to save time and stop “tollerating” the shortcomings of bad design.

    Excellent case study.

  10. Pingback: Turulcsirip - henry copeland

  11. Collin,

    I think part of where that “tolerate a bad design” sentiment comes from is because Freedback helps people accomplish a task. So as long as it does a good job at that, one might tolerate other faults.

    It is a blurry line though – better design should make the task easier, and that’s what we were aiming to to here.

  12. Only partially related, what software did she use to make that graph and does it export to EPS :) I”m tired of the bad interface in SigmaPlat and Matlab is a little obtuse.

  13. Oooh, great question. I have no idea. I think she may have done it in Excel of all places. I’ll find out.

    • Thank you for your love to huzi! We\’ re glad you are interested to feratue our blog post in your newsletter. Please kindly send your information to .Thank you!

  14. Hi Jason,

    I did indeed use Excel to make this plot. Unfortunately, excel does not directly export as an EPS file, but there are some work-arounds for that. One could print to PDF and then pdf2ps and then to eps. but often the resolution of the output isn’t great. If I want a really nice looking plot, I use XMGRACE which is a unix based (with GUI) which is free for the taking: http://plasma-gate.weizmann.ac.il/Grace/. it exports nicely to ps, eps, jpg, png, etc.

  15. Pingback: mcdave.net » links for 2007-05-18

  16. Pingback: PressD 2.1 » Designers: financial proof that good design matters (show your boss)!

  17. Pingback: Design Pilez » Designers: financial proof that good design matters (show your boss)!

  18. Hi Guys,

    I just installed a freedback form on my site. I was over the moon to discover how easy it was to register etc.

    Great job.

    • @HeartburnHomeRemedy I moved the blog to wordpress.com, and I think you got a notification for an old post. Sorry about that.

  19. Pingback: repyacode.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: