Quantcast

Easy Multivariate Testing with Visual Website Optimizer


Testing is an essential part of online strategies and Visual Website Optimizer is an ideal tool to help you get up and running with AB/Multivariate testing very quickly. I had a chance to chat with Founder, Paras Chopra to get his insight into Visual Website Optimizer.

Manoj:Give us the ‘elevator pitch’ on your technology.

Paras Chopra: Visual Website Optimizer is the world’s easiest A/B and multivariate testing tool. Meant for marketers who want to take charge of conversion rate optimization for their business, the tool eliminating involvement of IT resources by removing all HTML code and integration hassles. Using the tool, a marketer can rapidly test headlines, images, buttons, forms, styling, layout etc. to find out the best converting version of the landing page and/or website. Till now 2000+ enterprises, small businesses and agencies have used the tool to increase sales and conversion rates by as much as 90%.

Manoj:Can you please highlight some of the features which provide the most value?

Paras Chopra: There are three main features in Visual Website Optimizer that provide most value. The first one is the tagless integration where you we require a user to upload a small snippet of code into his/her website only once and after that s/he can create unlimited number of (different kinds of) tests without needing to touch code ever again. So, we don’t require the user to add separate code for separate sections of test or for different conversion goals. The single code snippet works any kind of test that a marketer might want to do. This saves a lot of time, that would have otherwise be spend in integrating every new test into the website.

The second valuable feature is the WYSIWYG editor which lets a marketer create variations using a familiar, MS-Word like interface. S/he is not required to know HTML or JavaScript while creating a test. This really speeds up the tests. So for an example, if you want to do a simple test involving button and headline change. You can select those sections in the visual test designer and then simply change the button and headline in WYSIWYG editor. That’s it and you are done. No HTML; no technical complexities. Here is a screenshot:

Clickmaps and heatmaps is another feature that our customers love. In addition to quantitative conversion rate report, in our tool you also get access to a heatmap which shows visually which parts of your page (or test variation) gets mosts clicks. Our users love this report because it complements the conversion rate data they have and it shows other opportunities on page for optimization. Here is an example:

Manoj:What is your competitive advantage over similar technology?

Paras Chopra: Our competitive advantage is the usability of the tool, our focus on marketers and the price-point for the features we offer. Ours is the first and only tool that makes A/B and multivariate testing a fun and easy job. Fun because the tagless integration allows you to always keep testing your hypothesis without worrying about implementation. So, for example, if you want to do a quick test to finding out if ‘Add to Cart’ works better than ‘Buy Now’ you can create and execute that test in less than 10 minutes. And since reports are realtime you can instantly start getting feedback on your test variations. All these minor features that promote ease-of-use add up to make Visual Website Optimizer the only “marketer-friendly” testing tool out there.


Manoj:Who is your intended target market?

Paras Chopra: Our target market is divided into three segments: a) small/medium sized businesses who do testing in house; b) marketing agencies & consultancies; and c) large enterprises. Our current customer base is a mix of these three segments. Even though our focus has been one SMBs and agencies, we are starting to see great response from enterprises as well. For example, today we have Microsoft, Rackspace, 99Designs, Threadless, AWeber, amongst other enterprise users. In all we have 2000+ users (in first 6 months of launching).

Manoj:Tell us about new features/integrations/improvements that are on your product road map

Paras Chopra: Soon we are going to launch the report segmentation and targeting feature which will allow you to segment test results by different parameters: country, traffic source, search keyword, time of day, etc. So, you will be easily analyze if one of your variations works better for a particular segment (say visitors from UK who searched for “shoes”). That variation may not be overall winner but if it works best for a particular segment, we will then allow the user to setup targeting to always show that particular variation to that particular segment. We believe the segmentation and targeting feature will take our product to the next level because marketers and analysts will have all CRO resources and tools within a single application: testing, analyzing and targeting.

For SMBs and agencies, pricing plans range from $49-$249/month. For enterprises (or other users) with large traffic requirements, we have custom plans where the price typically ranges from $500-$3000 per month.

Ultimate Multivariate & AB Testing Guide – WhichMVT.com

If you’re looking for a multivariate/landing page testing solution then I highly recommend you take a look at http://www.whichmvt.com/ (developed by the Conversion Rate Experts). WhichMVT is a complete guide to the most popular multivariate testing platforms available and provides information on each vendors such as: Price, Technology, Features, Support, and Reviews. Here is quick summary on what you can find on a few of the vendors:
  • Accenture: Setup costs around $33K a year, provides AB/MVT Testing, it can be self hosted, and can be used on email campaigns.
  • ION Interactive has a monthly price of $995, provides taguchi MVT, can segment traffic and has a platform for making web pages.
  • OnDialog costs 7.5K/year, doesn’t have the ability set mulitple goals, and customer support by phone available.


Top Online Testing Platforms 2010 – Forrester Wave Report

Forrester recently published their latest wave report on the topic of online testing platforms. In the report, Joseph Stanhope reviewed the top 9 players in the online testing space: Adobe Test & Target, Amadesa Customer Experience Suite, Autonomy Optimost, Google Website Optmizer, Maxymiser Content MVT, SiteSpect, Vertster Conversion Optmization Suite, and Webtrends Optimize.

Although the space is full of strong performers, Adobe Test & Target (formerly Omniture) and Autonmoy Optimost were placed as the leaders: Adobe excels in overall application usability, customer satisfaction, and content support. Autonomy stands out in administration, deployment options, and breadth of testing techniques. Both vendors demonstrate strong integration options with the Web marketing ecosystem, enabling marketers to plug online testing into a broader application framework.

Love and Marriage: From Testing to Targeting

You can’t have one without the other – or: Whoever considers testing without targeting is already lost

Online Marketing AdvancementYou would think it goes without saying. And still it was presented as one of the future challenges at the dmexco 09 in Cologne, Germany – for instance by Heinz D. Schultz, CEO at mindlab, in his presentation “Crossing the Chasm”: the pooling of Web Analytics and Business Intelligence. Well – do you really have to stress this? Of what use could the best online marketing ever be if the bounce rate of your website rises to astronomic heights? If a visitor to your website does not enter the shopping procedure and does not convert? If an ad addresses a target audience totally different to the one your website appeals to? Testing can provide answers to many questions like these, but as I use to say, testing can only be the first step on the way to targeting.”

Simple and logical as it appears to be – it still does not seem to have caught the attention of many companys in the online world. Whoever calls a testing tool his own is busy testing over and over again and in the end the board of directors, marketing department and design go with their guts. This kept in mind the issue will still be present in the next year –as shown e.g. by the announcement of Kevin Hillstroms presentation, Turning Data into Strategies: How Multichannel Forensics and Online Marketing Simulation leverage the art and science of E-Commerce for April 27th 2010 at the Ecom fair in Berlin.

Meet your customers where they are – thousands of companies in the web refer to this concept. But where is your customer? The knowledge gained from testing can help with this question. Where did he come from? Where does he leave your website? In which way could you address him so that you would get through to him? Whoever searched for the latest bestselling novel in the web and found your website should not be confronted with a kitchenware ad on your landing page. And who – as Heinz D. Schulz described in his presentation – orders trainers, which were returned by most customers for being to small, should be alerted to that during the order process. Your website is mainly used by elderly customers and wearers of glasses? You should keep this in mind when designing the website and be aware, whether the customers after having decided on the product will desperately search for the cart symbol which an aesthetics-oriented design has lightheadedly created just nuances lighter than the background…).

The final goal of every investment in the website should be the amelioration of the sales approach and as a consequence the augmentation of the conversion rate. For this it is indispensible for the Business Intelligence to keep in mind the results gained from testing the website and to turn them into well-directed design and marketing actions. As in every good marriage one has to work on ones communication skills. Until you finally can join in: Love and marriage…

Matthias PostelAbout:
Matthias Postel is an expert in web analytics, testing and targeting. He is founder and managing director of the iCompetence GmbH which is based in Hamburg, Germany. The iCompetence GmbH specializes in digital business optimization with focus on website and conversion optimization.

Engage with Webtrends – 3 Steps to Learning Optimization

Guest Author: Billy Shih, Webtrends

A lot of marketers want to get into testing and optimization, yet it can be intimidating to start. While I don’t think optimization can be learned overnight, it’s not any more challenging than what marketers do already and reinforces many of the skills they already have.

In fact, the better you are at analytics, copywriting or any other useful marketing skill, the better you will do in testing. Right now, I am taking advantage of the analytics prowess of Webtrends to boost the findings from our test results in The Open Campaign. More ways to measure KPI’s means more ways for me to understand my audience. Similarly, if you were good at copywriting, you could write copy for multiple personas and try them all in testing.

With these things in mind, let’s walk through how optimization is done and look at the overlaps it has with what online marketers already do.

Step 1: Evaluate the website for weaknesses – Choosing a page to test

When reviewing your analytics data, one thing you probably already do is look for areas of your site that are underperforming. This also happens to be one of the best ways to choose a page to test.

Look for a page:

  • That used to perform well, but doesn’t anymore
  • That is underperforming compared to similar pages
  • With distinct segments converting differently

Step 2: Redesign a page – Deciding what to test

Anytime you’re looking to do a page redesign or even just make modifications to it, you have to have some reasons behind it. Now that you have a page selected, the first thing to do is to figure out why that page would underperform. You’re not looking for 100% proven facts, you’re just trying to hypothesize why the page isn’t as good as it should be.

Here are some standard questions:

  • Looking at the traffic sources, does the page match up with the mindset of visitors coming from those sources?
  • How can you improve continuity between pages (e.g. using a PPC headline for the page headline)?
  • What is missing from the page? Is there too much information?
  • How well is it organized? Is it organized in a way the audience can digest?

The most common problem I find is that pages are too complicated and need to be refocused on the core goal of the page.

Once you have a few ideas, bring them to your creative team or ask them to contribute some ideas. You should be able to use the same workflow as you do currently. Sometimes I work really closely with creative teams to brainstorm, other times I just tell them exactly what I want. I recommend getting creative input though, as they bring a healthy new perspective and can see some design challenges we marketers may miss.

The only difference compared to the standard design process is that the creative team makes a few comps of different ideas and you don’t have to choose just one—instead, you test them. Similarly, your web development team does basically the same thing, turning the comps into code and then inserting the variations into your optimization platform.

Step 3: Analyze performance – Reviewing the test data

Once the test is up and running, start monitoring your data. Like any new page, it’ll take some time to gather enough data in order to be able to take away some solid information. With a test, it’ll take longer than analyzing a single page’s performance because you are testing many variations.

Traditionally, once a page is redesigned and pushed out live, you would compare how the page performed in a week or month against how the old page performed the week or month before (or the year earlier.) This isn’t very accurate though since outside factors you can’t control may cause changes in its performance, such as seasonality or other marketing campaigns.

With testing, this is avoided and you can compare how your original page performed against the new variations during the same week or month. This makes things easier to understand and a more accurate comparison.

Once you see that the winning variations hold true for a few days, the test can typically be called complete and you should have a shiny, new optimized page. Instead of having a single person or group in your company decide what the best page is, your customers have done it for you!

Optimization doesn’t have to be complicated and lines up closely to what online marketers all over the world are doing every day. Although there is some extra work involved, knowing what your customers want is the most difficult task of all and with optimization you get a powerful tool to resolve that problem.

Omniture Releases New UI for Test&Target

Omniture, Inc. (NASDAQ: OMTR), today announced a new version of Omniture Test&Target that provides marketers with the ability to more visually design, test and execute marketing campaigns directly on their Web sites. Marketers can now save time and implement changes immediately on their site, increasing the speed and ease with which campaign content can be used for A/B testing and targeted campaigns.

“Testing our site is something that we do constantly,” said Matt Sanocki, director of marketing at Design Public, an exterior and interior creative design retailer. “Omniture Test&Target’s new visual capabilities make a tremendous difference because I can get everyone on my online marketing team involved in the process. It is now easier to design and launch new marketing campaigns directly on the site, making the Design Public experience even better for our customers.”

The new version of Omniture Test&Target allows online marketers to:

  • Create Tests Visually – Online marketers can create A/B tests and targeted campaigns directly on the Web pages where and how visitors will experience them.
  • Deliver Tests Faster – New campaigns and marketing ideas can be tested and implemented as they are conceived.
  • Test More Frequently – Because the new interface is easier to learn, teams of online marketers will find it easier to create and collaborate on marketing campaigns, enabling them to run multiple tests more quickly.

“The new campaign creation and testing capabilities in Omniture Test&Target have become an important part of our process for creating and demonstrating value to our clients,” said Nicholas Ward, search technology strategist for Range Online Media, a leading search and interactive marketing agency. “The ability to build and preview tests simultaneously helps us ensure a positive visitor experience during testing and limits the complexity for our team and our clients.”

“Putting the right message in front of the right person at the right time has always been the marketer’s dream,” said Jim Sterne, founder of the eMetrics Marketing Optimization Summit.

Omniture’s Test&Target hands the testing process to marketers and removes the technical and operational steps that have made testing so tedious in the past.”

Omnitures Test&Target: Insight from Chris Duskin

You might have heard of Omniture’s Test&Target product which has the ability to leverage sophisticated behavioral targeting plus Multivariate testing, but you might not completely understand whether it’s right for your business or what type of insight it can provide you. I decided to find out some more information by posing some questions to a Test&Target expert, Omniture’s very own Sr. Director of Product Marketing, Chris Duskin.

[Manoj]: Please provide an overview of the Test&Target framework and how you have integrated Touch Clarity and Offermatica.

[Chris]: The promise and the challenge of online marketing is the ability to meaningfully engage with your customers. One aspect of the promise is knowing which online experience or content is most relevant. And part of the challenge involves tools that give marketers control. TouchClarity and Offermatica combined as Omniture Test&Target give marketers the best tools to deliver relevance quickly and with very little IT involvement.

Testing, whether A/B/n or multivariate, helps marketers know which online experience is best. What we have learned, however, is that what is best for one group of visitors is not necessarily optimal for another. So testing and targeting are two sides of the same coin. Testing naturally leads to targeting – delivering specific content to specific users for even more relevance. Some basic examples of targeting include different experiences for new visitors than for returning visitors or different messaging based on a visitor’s geographic location. And, the most powerful targeting in terms of delivering relevance is one-to-one targeting based on predictive mathematical models. In a lot of cases, there is maximum benefit from letting an automated system determine the best content for the individual visitor at any point in time based on current intent. Omniture Test&Target is a single product providing all these capabilities, all of which are increasingly required by online marketers.

The integration of TouchClarity and Offermatica has certainly been a big effort, but it has been made easier by similarities between the two systems. Key components of the integration include a single tag or container for content delivery as well a single interface for content management, campaign creation, and user management. In coming releases, we’ll also deliver some very compelling, very powerful capabilities that even further build on the power of analytics, testing, and targeting.

[Manoj]: Why should Test&Target be a part of all ongoing landing page/content strategies?

[Chris]: Because more often than not, what we think will work is wrong. We think hard about the problem, look at all the data, consider all the options, leverage all our experience. But in the end, our traffic has the final say. Without testing and without targeting, relevance is left to chance, and the marketer is in a reactive mode. Test&Target lets marketing get into a proactive position and see in real-time what is effective and driving the business’s metrics.

[Manoj]: How much of a technical skill set is required to implement a customized test?

[Chris]: To be effective, marketers need control. In their tests and campaigns, they need to be able to make content, layout, and messaging changes wherever necessary. And they need to be able to do it quickly. So, Test&Target is designed for marketers who aren’t technical. Familiarity with html can be helpful as can the array of Omniture services options available to Test&Target customers.

[Manoj]: I’d like to know more about the specifics on the implementation of div tags and the fact that users have to pre-load their desired content in to the admin interface.

[Chris]: Our tags are javascript-based and allow marketers to use Test&Target’s online admin tool to change content in the tagged area. Implementing our tags is very straightforward and involves only a few lines of code. Once in place, the tag can be used repeatedly of course. The content that goes into those areas is either created in Test&Target or, more commonly, resides in a content management system. We also have features to allow the bulk upload of content that some customers find useful.

[Manoj]: Can you explain how different types of content are rotated based on predefined configuration or content performance?

[Chris]: Test&Target’s behavioral targeting campaigns allow the marketer to specify content for various site locations. Our algorithms then automatically capture and learn from hundreds of a visitor’s behaviors and characteristics to determine which of the content options to serve to the individual. The traffic to most sites is incredibly variable, and so is the intent of an individual. So a system like Test&Target that not automatically reacts to but is made stronger by this variability is incredibly powerful. This is what we mean by learning algorithms.

In terms of predefined configuration, marketers can also set rules that the models consider when determining what content to show. For example, some visitors may not be eligible for a particular offer depending on their geographic location or based on some other characteristic. Our system takes these rules into account. And, what’s also unique about our approach is that it’s effective for anonymous visitors.

[Manoj]: What factors (metrics) are used to help the system target based on behavior?

[Chris]: Success criteria used by our targeting algorithms can range from basic click-through to revenue events depending on the objective of the campaign.

[Manoj]: How can offline data be integrated?

[Chris]: Our customers load offline data into Test&Target using a set of APIs. A lot of companies that have transactions completed offline leverage this capability. For example, we have financial services customers who measure test campaign success by both online loan applications as well as offline loan approval. The loan approvals are loaded into the system using the APIs. The use of vanity or custom toll-free numbers is another technique some customers use to test different messaging or calls-to-action in both the online and offline environments. In these cases, call center activity is loaded into Test&Target for campaign evaluation.

[Manoj]: How can the Test&Target framework be used with email campaigns?

[Chris]: I mentioned earlier that our tag is javascript-based. We also have a non-javascript version used for configuring test content in emails. A leading coffee chain, a national moving company, and a large technology company have all recently tested email content with great success. This technique is also really compelling since it not only results in more effective email campaigns but essentially allows the content of unopened emails in inboxes to be altered to show fresher content. People are also thinking more and more about optimizing entire experiences rather than just pages. What this means is that testing and targeting don’t occur on a single landing page or even solely on the site. Instead, it starts at the first point of contact with a customer such
as in an email, continues across multiple site pages, and lasts for multiple visits. This kind of thinking reflects a real and deep focus on delivering relevance and is at the heart of Test&Target.