Tuesday, April 28, 2009

09NTC Notes From 04/28/09: Multivariate Testing

When you want your buttons pushed: How to optimize your site with multivariate testing

session description:
Ever wonder if your site’s landing pages are too long, donate button is too small, or images are sending just the right message? This session will provide key insights to help you decide whether it’s time for your organization to begin optimizing key action pages with multivariate testing. Stop making decisions by committee or letting the HiPPO (Highest Paid Person’s Opinion) lead the way and start making choices through solid evidence based on your own users’ actual behavior – the results might surprise you! Takeaways: 1. MVT can help you dramatically improve conversion rates. 2. MVT takes the guesswork out of copy, color, location, headline, format, and other key decisions. 3. MVT tests range from fairly simple to highly complex.

-multivariate testing is for when you have many questions about different elements on your page
-ask users in real tim
-generate evidence-driven results based on your users' behavior
-share sample size (get answers more quickly)

Survey of possibilities

-A/B concept testing: better than no testing. problems arise because you have to make assumptions about why users favor something or not

multivariable testing looks at multiple areas on a page at the same time. not just flipping things around, but being able to completely move things around. layout changes, types of content, content itself

content & design are all in the same process

elemental concept testing at Delta
asks: Did we do harm? (with new content)

stop wasting design time, ask does it matter which icon we use? more features or more simple?

offer the beta for people to peruse before launching, may reduce actual changes. you get to understand what matters to your core peeps

the order of copy blocks has substantial impact

moving something like search boxes around the page- does the left-hand side top work best? typically, yes

condense forms by removing non-essential fields (fax #?) always look for opportunities of simplification

sometimes "assurance language" fails (it reminds folks spam exists for example)

images of people tend to have a positive effect when associated with org branding, but some images do not add any value or brand equity to a product.

PHOTOS REALLY REALLY need to be tested.

again, remove distractions (especially on home pages)

some rich media is too much of a distraction, and will carry people off of your site

make a button look like a button. use obvious image metaphors

help people engage, and be very clear about it

interesting & subtle point- understand the interactions- sometimes it's the combination of things on a page that make the difference. if you don't test the items together, then you miss out on good insight

getting started:

1. choose a goal
2. Identify conversion pages
3. select the right tool

lots of different testing tools. google website optimizer is free, starting with free can be really great

case study: ACLU
their goal was grow their list and increase donations, increasing traffic
internal considerations, resourcing, ID highest priorities, IT considerations

design dilemma, when can they use testing to prove out key design concepts
she learned you really have to keep tweaking, changing

No comments:

Post a Comment