A/B Testing

Easy Multivariate Testing with Visual Website Optimizer

August 30, 2010 By Manoj Jasra 2 Comments

Testing is an essential part of online strategies and Visual Website Optimizer is an ideal tool to help you get up and running with AB/Multivariate testing very quickly. I had a chance to chat with Founder, Paras Chopra to get his insight into Visual Website Optimizer.

Manoj:Give us the ‘elevator pitch’ on your technology.

Paras Chopra: Visual Website Optimizer is the world’s easiest A/B and multivariate testing tool. Meant for marketers who want to take charge of conversion rate optimization for their business, the tool eliminating involvement of IT resources by removing all HTML code and integration hassles. Using the tool, a marketer can rapidly test headlines, images, buttons, forms, styling, layout etc. to find out the best converting version of the landing page and/or website. Till now 2000+ enterprises, small businesses and agencies have used the tool to increase sales and conversion rates by as much as 90%.

Manoj:Can you please highlight some of the features which provide the most value?

Paras Chopra: There are three main features in Visual Website Optimizer that provide most value. The first one is the tagless integration where you we require a user to upload a small snippet of code into his/her website only once and after that s/he can create unlimited number of (different kinds of) tests without needing to touch code ever again. So, we don’t require the user to add separate code for separate sections of test or for different conversion goals. The single code snippet works any kind of test that a marketer might want to do. This saves a lot of time, that would have otherwise be spend in integrating every new test into the website.

The second valuable feature is the WYSIWYG editor which lets a marketer create variations using a familiar, MS-Word like interface. S/he is not required to know HTML or JavaScript while creating a test. This really speeds up the tests. So for an example, if you want to do a simple test involving button and headline change. You can select those sections in the visual test designer and then simply change the button and headline in WYSIWYG editor. That’s it and you are done. No HTML; no technical complexities. Here is a screenshot:

Clickmaps and heatmaps is another feature that our customers love. In addition to quantitative conversion rate report, in our tool you also get access to a heatmap which shows visually which parts of your page (or test variation) gets mosts clicks. Our users love this report because it complements the conversion rate data they have and it shows other opportunities on page for optimization. Here is an example:

Manoj:What is your competitive advantage over similar technology?

Paras Chopra: Our competitive advantage is the usability of the tool, our focus on marketers and the price-point for the features we offer. Ours is the first and only tool that makes A/B and multivariate testing a fun and easy job. Fun because the tagless integration allows you to always keep testing your hypothesis without worrying about implementation. So, for example, if you want to do a quick test to finding out if ‘Add to Cart’ works better than ‘Buy Now’ you can create and execute that test in less than 10 minutes. And since reports are realtime you can instantly start getting feedback on your test variations. All these minor features that promote ease-of-use add up to make Visual Website Optimizer the only “marketer-friendly” testing tool out there.

Manoj:Who is your intended target market?

Paras Chopra: Our target market is divided into three segments: a) small/medium sized businesses who do testing in house; b) marketing agencies & consultancies; and c) large enterprises. Our current customer base is a mix of these three segments. Even though our focus has been one SMBs and agencies, we are starting to see great response from enterprises as well. For example, today we have Microsoft, Rackspace, 99Designs, Threadless, AWeber, amongst other enterprise users. In all we have 2000+ users (in first 6 months of launching).

Manoj:Tell us about new features/integrations/improvements that are on your product road map

Paras Chopra: Soon we are going to launch the report segmentation and targeting feature which will allow you to segment test results by different parameters: country, traffic source, search keyword, time of day, etc. So, you will be easily analyze if one of your variations works better for a particular segment (say visitors from UK who searched for “shoes”). That variation may not be overall winner but if it works best for a particular segment, we will then allow the user to setup targeting to always show that particular variation to that particular segment. We believe the segmentation and targeting feature will take our product to the next level because marketers and analysts will have all CRO resources and tools within a single application: testing, analyzing and targeting.

For SMBs and agencies, pricing plans range from $49-$249/month. For enterprises (or other users) with large traffic requirements, we have custom plans where the price typically ranges from $500-$3000 per month.

Filed Under: A/B Testing, multivariate testing, Vendor Spotlight, Web Analytics

Ultimate Multivariate & AB Testing Guide – WhichMVT.com

August 27, 2010 By Manoj Jasra Leave a Comment

If you’re looking for a multivariate/landing page testing solution then I highly recommend you take a look at http://www.whichmvt.com/ (developed by the Conversion Rate Experts). WhichMVT is a complete guide to the most popular multivariate testing platforms available and provides information on each vendors such as: Price, Technology, Features, Support, and Reviews. Here is quick summary on what you can find on a few of the vendors:

Accenture: Setup costs around $33K a year, provides AB/MVT Testing, it can be self hosted, and can be used on email campaigns.
ION Interactive has a monthly price of $995, provides taguchi MVT, can segment traffic and has a platform for making web pages.
OnDialog costs 7.5K/year, doesn’t have the ability set mulitple goals, and customer support by phone available.

Filed Under: A/B Testing, multivariate testing

Top Online Testing Platforms 2010 – Forrester Wave Report

August 23, 2010 By Manoj Jasra Leave a Comment

Forrester recently published their latest wave report on the topic of online testing platforms. In the report, Joseph Stanhope reviewed the top 9 players in the online testing space: Adobe Test & Target, Amadesa Customer Experience Suite, Autonomy Optimost, Google Website Optmizer, Maxymiser Content MVT, SiteSpect, Vertster Conversion Optmization Suite, and Webtrends Optimize.

Although the space is full of strong performers, Adobe Test & Target (formerly Omniture) and Autonmoy Optimost were placed as the leaders: Adobe excels in overall application usability, customer satisfaction, and content support. Autonomy stands out in administration, deployment options, and breadth of testing techniques. Both vendors demonstrate strong integration options with the Web marketing ecosystem, enabling marketers to plug online testing into a broader application framework.

Filed Under: A/B Testing, multivariate testing, Web Analytics

A/B Testing Made Easy with Visual Website Optimizer

May 28, 2010 By Manoj Jasra Leave a Comment

If you have ever done an A/B testing on your website or landing page, you would know how exciting is the prospect of finding out which variation is going to perform best. It is almost like a football game where you can’t tell until the end which team is going to win. But that excitement soon cools off once you actually get to create and implement an A/B test: creating different pages, fiddling with HTML code, integrating JavaScript page tags. And, finally waiting for developers and IT team to implement your simple A/B test. All the technical hassles associated with A/B testing terribly slow down the process and make it unexciting.

Making A/B testing dead-easy, Visual Website Optimizer is perhaps an answer to all these frustrations. The tool loads your website (or landing page) in a visual designer and then you can use a word-like WYSIWYG editor for creating test variations. Then select conversion goals for your test and you are ready to run an A/B test. The integration part is pretty simple too – all you need to do is to upload a small code snippet in your website once and then you can create unlimited number of tests from the tool without touching your website code ever again. This means that once the code goes into your website, you can create and launch tests in less than 5 minutes. Watch a short video below:

In terms of functionality, Visual Website Optimizer has interesting features for tracking and optimization. For example, you can select multiple conversion goals for your A/B and multivariate test. That is, you can measure how your test affects sales, bounce rate, signups – all goals at once. You can run targeted tests only on a particular visitor segment such as new visitors or visitors coming from a particular referral source. The tool also integrates with Google Analytics, Omniture SiteCatalyst and other web analytics platforms so that you can segment test results for deeper insights.

Their pricing plans range from $49/mo to $729/mo which is significantly more affordable than other enterprise testing tools such as SiteSpect, Webtrends Optimize and Omniture Test and Target (costing thousands of dollars per month). Even though it is a recent startup, they claim to have 1000+ users including some well known companies such as Microsoft, Rackspace and 99designs.com. As far as results are concerned, they have a couple of A/B testing case studies on the website where their users have seen 20-90% increase in sales and conversions.

You can sign up for their free 30-day trial to evaluate the tool yourself.

Filed Under: A/B Testing

Love and Marriage: From Testing to Targeting

December 17, 2009 By Matthias Postel Leave a Comment

You can’t have one without the other – or: Whoever considers testing without targeting is already lost

You would think it goes without saying. And still it was presented as one of the future challenges at the dmexco 09 in Cologne, Germany – for instance by Heinz D. Schultz, CEO at mindlab, in his presentation “Crossing the Chasm”: the pooling of Web Analytics and Business Intelligence. Well – do you really have to stress this? Of what use could the best online marketing ever be if the bounce rate of your website rises to astronomic heights? If a visitor to your website does not enter the shopping procedure and does not convert? If an ad addresses a target audience totally different to the one your website appeals to? Testing can provide answers to many questions like these, but as I use to say, testing can only be the first step on the way to targeting.”

Simple and logical as it appears to be – it still does not seem to have caught the attention of many companys in the online world. Whoever calls a testing tool his own is busy testing over and over again and in the end the board of directors, marketing department and design go with their guts. This kept in mind the issue will still be present in the next year –as shown e.g. by the announcement of Kevin Hillstroms presentation, Turning Data into Strategies: How Multichannel Forensics and Online Marketing Simulation leverage the art and science of E-Commerce for April 27th 2010 at the Ecom fair in Berlin.

Meet your customers where they are – thousands of companies in the web refer to this concept. But where is your customer? The knowledge gained from testing can help with this question. Where did he come from? Where does he leave your website? In which way could you address him so that you would get through to him? Whoever searched for the latest bestselling novel in the web and found your website should not be confronted with a kitchenware ad on your landing page. And who – as Heinz D. Schulz described in his presentation – orders trainers, which were returned by most customers for being to small, should be alerted to that during the order process. Your website is mainly used by elderly customers and wearers of glasses? You should keep this in mind when designing the website and be aware, whether the customers after having decided on the product will desperately search for the cart symbol which an aesthetics-oriented design has lightheadedly created just nuances lighter than the background…).

The final goal of every investment in the website should be the amelioration of the sales approach and as a consequence the augmentation of the conversion rate. For this it is indispensible for the Business Intelligence to keep in mind the results gained from testing the website and to turn them into well-directed design and marketing actions. As in every good marriage one has to work on ones communication skills. Until you finally can join in: Love and marriage…

About:
Matthias Postel is an expert in web analytics, testing and targeting. He is founder and managing director of the iCompetence GmbH which is based in Hamburg, Germany. The iCompetence GmbH specializes in digital business optimization with focus on website and conversion optimization.

Filed Under: A/B Testing, multivariate testing, Web Analytics

Webinar: A/B Testing for SEO Landing Pages — How to Avoid Risks

July 11, 2009 By Manoj Jasra Leave a Comment

If you’re interested in learning more about optmizing your landing pages then we recommend you check out Anne Holland’s webinar next Wednesday, July 15 on How to Avoid the Risks of A/B Testing SEO Pages. It’s inspired by a pet peeve of hers where so many marketers think if a page is SEO-ed, then it is human optimized too… usually which couldn’t be further from the truth. Also, so many marketers will A/B or multivariate test their PPC landing pages, but not touch SEO pages because they’re so terrified of what could happen.

Filed Under: A/B Testing

Always Be Testing – Interview with Bryan Eisenberg

November 3, 2008 By Manoj Jasra Leave a Comment

A few months ago Bryan Eisenberg and John Quarto-vonTivadar released an in-depth guide on testing using Google Website Optimizer called, “Always Be Testing.” I had the opportunity to go through the book and thought that anyone could read it and quickly implement a testing framework using Google’s website optimizer.

I also got the chance to interview co-author Bryan Eisenberg to get his thoughts on motivations on writing the book and the benefits readers can get from it.

[Manoj]: Can you start by providing some insight into your motivations for writing the book: “Always Be Testing“

[Bryan]: Google released Website Optimizer a few years ago. It’s evolved and improved greatly in that time, and a lot of people are using it now. But the biggest challenge people face is, “what do I test?” My partner, John Quarto-vonTivadar and I wrote “Always Be Testing” to provide readers with a guide to learning what to test, how to test, and in what order to test.

While many companies are testing, even more companies should be testing who are not currently doing so. Google Website Optimizer represents a painless and cost-effective way to begin testing. Because you can get started in 30 minutes and because it’s free, it cancels out all the usual excuses of time and budget.

In fact it turns those excuses completely around with the challenge “well since it’s simple and costs nothing, why aren’t you testing?” Certainly a forward-looking company has got to figure that their main competitors are testing, and people then begin to see ways to build a culture of testing within their business as well as the economic value of continuous improvement.

[Manoj]: There are numerous organizations who don’t take the time to test their content/landing pages – what does it take to make testing a priority for them?

[Bryan]: Well, for good or ill, the current economic downturn will certainly help focus businesses on the values of doing more with the same (or less) resources. The rising cost of media reach at the same time that traffic numbers are starting to decline is a potent incentive to improve, test, and optimize some more.

And for some people who are looking at all the current layoffs occurring, don’t you think it may cross their mind that “If I can do some testing for free and can improve conversion or company goals, then I look like a star if a decision about layoffs occur and I’m able to show with numbers how I contributed to the top line revenues. “ You can be sure C-level managers understand that argument.

Strange, isn’t it, that a recession is when people finally getting around to getting started testing? Pain can be a motivator, I guess.

Another point, something we’ve seen a lot of this past month: Some clients actively testing are reporting traffic numbers that are flat, even as their conversion rate goes up. Keep that in perspective: in a recession, which after all means a shrinkage of the market, if all your competitors are experiencing lower traffic and lower average sales value but you are reporting flat traffic and flat average sales value, that means you must be gaining market share! In other words, during a recession, being flat on testing might actually mean you’re ahead of the game, compared to your competitors.

[Manoj]: In “Always be Testing” you mention that organizations sometimes forget to understand their customer’s goals (and begin selecting their own goals), how important is understanding your customer’s goals before you start testing?

[Bryan]: Generally speaking, you aren’t going to see large improvements to conversion traffic tests until you start testing for customer goals. There’s only so far you can push your own agenda if it comes at the expense of your customer’s goals.

The reality is that customers completely control their experience online, and at the end of the day if you don’t meet their needs then you aren’t going to be successful. The key to conversion is helping your customers get what they are looking for so that you can get what you are looking for. Testing continuously means you are constantly iterating toward improving conversion – after all if you test and don’t improve then you’d just go back to the original and try a different test – it’s not looking you’re looking to lower your conversion rate! Implicitly, then, testing leads to more customers achieving their goals which will mean more of your own goal being achieved.

[Manoj]: Do you feel Google Website optimizer can handle the majority of an organization’s testing needs whether it’s a small business or a large enterprise?

[Bryan]: Google Website Optimizer is, thankfully, small and robust; but limitations do exist – for example, it doesn’t addresses some of the larger organization’s needs, like segmentation (to mention just one). But the 250+ tests we describe in the book could keep any organization, large or small, busy testing for several years.

Here’s how we look at it: If you’re a small business, then Google Website Optimizer (and, we hope, our book) is right for you.

If you’re a large organization but haven’t achieved a lot of experience testing then Google Website Optimizer and our book is also right for you, because any type of testing is better than doing nothing. But if you’re a large organization and you’ve built a culture of testing with lots of experience doing it, then you almost certainly already have an enterprise-level testing platform installed — in which case, just use it!

The only bad thing you can do testing-wise is not to test.

[Manoj]: What are the most important things that readers will understand after reading “Always Be Testing”

[Bryan]:

They’ll gain the confidence to take the first steps, which is always hard.
They’ll learn a process for understanding how to test, what to test, and in which order to test, from the market thought leader in testing and optimization.
As they improve, they will generate their own testing ideas arising from the unique nature of their business and they will then be able to test and improve the elements of their business that give them a competitive advantage.

Filed Under: A/B Testing, Google