Take Politics Out of the Decision Making Process with Testing Part 3



This is part 3 of a multi-part post on website testing.
Part 1 is available here
Part 2 is available here

Alas, we reach the end of this blog series on site testing, but hopefully this is just the beginning of your journey in conversion optimization.

As promised in part 2, this last post will focus on the differentiating features, the reporting functionality, and the summary of tools.

Differentiating Features

Based on my experience I’ve found a lot of A/B testing tools to be very similar, if not downright the same thing. What differentiates one tool from the next often comes down to price alone. My goal here is to ignore price (all three tools were relatively the same price) and focus on the fundamental differences.

Convert Experiments

I was pretty excited to see the types of Google Analytics integrations Convert Experiments offers. All three tools integrated with Google Analytics to push test information into custom variables, but only Convert Experiments allows integration with Google Analytics goals and ecommerce tracking (with the exception of Google Analytics experiments).

Optimizely

I really liked the straight forward approach of Optimizely. If you’re just looking to set up a test and go, Optimizely has a very quick startup. The standout feature of Optimizely is the implementation… 1 line of code, AND you get emailed when the code gets added to a site. This is incredibly helpful.

Additionally, Optimizely integrates with KissMetrics, MixPanel, and SiteCatalyst. It also integrates with click tracking tools like ClickTale.

 

Optimizely-ClickTale

 

Visual Website Optimizer

I felt Visual Website Optimizer had the best overall feature set of all the tools tested. Not only is the targeting incredibly comprehensive and flexible, it has direct integration for “focus group” testing, straight geo-targeting testing and my absolutely favorite… click maps!

Visual Website Optimizer eliminates some of the need for tools like CrazyEgg by incorporating click maps directly into the testing. Additionally, if you only want click maps, you can do that sort of testing alone.

VWO-ClickMap

 

Reporting

I already demonstrated in Post 2 how the different tools report within Google Analytics, but let’s dive into how each tool displays the results.  I’m not providing much commentary on the reporting data and graphs themselves as everyone has a different preference for reporting.

Convert Experiments

I was pretty excited to see the types of Google Analytics integrations Convert Experiments offered. All platforms integrated with Google Analytics to push test information into custom variables, but only Convert Experiments allowed integration with Google Analytics goals and ecommerce tracking (with the exception of Google Analytics experiments.)

Convert-RevenueReport

Optimizely

Report summary is pretty clean, easy to tell which had the highest engagement effect, scrolling to the right shows the different goals.

 Optimizely-ReportSummary

The detailed goal reporting is also pretty clean. It’s easy to tell what’s doing better or worse.

Optimizely-ReportChart Optimizely-ReportData

 

Visual Website Optimizer

Reporting is pretty similar to Optimizely on the summary front, but I found this interface a little more convoluted than Optimizely.

VWO-ReportSummary

I like the detail reporting a bit better in Visual Website Optimizer than the other 2 tools -It’s very clean and right to the point.

VWO-ReportGraph VWO-reportdata

 

Table of Features

 

Google Experiments

Convert Experiments

Visual Website Optimizer

Optimizely

Google Analytics report integration

✔+

Custom Variable Slot

Custom Variable Slot

Custom Variable Slot

Segmentation

 

IP Address exclusion

 ✔ (filters)

✔ (experiment level)

✔ (account level)

✔ (account level)

Import goals from Google Analytics

 

 

Heat/Click maps

 

 

 

Split URL Tests

✔ (redirect)

A/B & Multi Variate Testing

 

Flexible Goals

Google Analytics revenue reporting

 

 

 

Closing Remarks

At the end of the day, the important thing to take away from this blog series is that you have no reason not to test. The tools out there are easy to get the hang of and use and the reporting is fairly straight forward. The real challenge for you is to decide where to start with your tests and what your success measurements are going to be.

So, with that said, go forth and test! Or, Tweet me (@MattAster) with your email and URL and I’ll give you 2 free optimization ideas to test!

Subscribe by Email
About Matt Aster

Matt Aster has more than 10 years of digital marketing experience working with large multinational corporations as well as small independent companies. Matt started out as a database administrator at Goodyear Tire and Rubber Company within the enterprise data warehouse team. He then joined Progressive Insurance Company where he worked on email marketing, web experience, and web analytics. Most recently Matt was Marketing Manager at Steris Corporation managing all aspects of digital marketing including social media, search engine optimization, pay-per-click advertising and email strategy. As a current member of the Inbound Marketing team at Precision Dialogue, Matt creates strategies and optimizes campaigns by using both new and existing digital marketing techniques.

When not writing complete sentences Matt likes to tweet his thoughts and opinions.

Learn more about Matt Aster on Google+
Read Matt's latest articles
Have a question? Contact Matt here

Comments

  1. Hi Matt,

    Thank you for the review. Love to show you the new version of Convert Experiments … super excited, but already pleased with the honest review.

    Dennis

    • Hey Dennis!

      I saw your note, I’ve been cleaning my inbox in the last couple weeks. I’ll shoot you an email, I wouldn’t mind seeing the new version at all :)

      Matt

Speak Your Mind

*