Omnitures Test&Target: Insight from Chris Duskin


You might have heard of Omniture’s Test&Target product which has the ability to leverage sophisticated behavioral targeting plus Multivariate testing, but you might not completely understand whether it’s right for your business or what type of insight it can provide you. I decided to find out some more information by posing some questions to a Test&Target expert, Omniture’s very own Sr. Director of Product Marketing, Chris Duskin.

[Manoj]: Please provide an overview of the Test&Target framework and how you have integrated Touch Clarity and Offermatica.

[Chris]: The promise and the challenge of online marketing is the ability to meaningfully engage with your customers. One aspect of the promise is knowing which online experience or content is most relevant. And part of the challenge involves tools that give marketers control. TouchClarity and Offermatica combined as Omniture Test&Target give marketers the best tools to deliver relevance quickly and with very little IT involvement.

Testing, whether A/B/n or multivariate, helps marketers know which online experience is best. What we have learned, however, is that what is best for one group of visitors is not necessarily optimal for another. So testing and targeting are two sides of the same coin. Testing naturally leads to targeting – delivering specific content to specific users for even more relevance. Some basic examples of targeting include different experiences for new visitors than for returning visitors or different messaging based on a visitor’s geographic location. And, the most powerful targeting in terms of delivering relevance is one-to-one targeting based on predictive mathematical models. In a lot of cases, there is maximum benefit from letting an automated system determine the best content for the individual visitor at any point in time based on current intent. Omniture Test&Target is a single product providing all these capabilities, all of which are increasingly required by online marketers.

The integration of TouchClarity and Offermatica has certainly been a big effort, but it has been made easier by similarities between the two systems. Key components of the integration include a single tag or container for content delivery as well a single interface for content management, campaign creation, and user management. In coming releases, we’ll also deliver some very compelling, very powerful capabilities that even further build on the power of analytics, testing, and targeting.

[Manoj]: Why should Test&Target be a part of all ongoing landing page/content strategies?

[Chris]: Because more often than not, what we think will work is wrong. We think hard about the problem, look at all the data, consider all the options, leverage all our experience. But in the end, our traffic has the final say. Without testing and without targeting, relevance is left to chance, and the marketer is in a reactive mode. Test&Target lets marketing get into a proactive position and see in real-time what is effective and driving the business’s metrics.

[Manoj]: How much of a technical skill set is required to implement a customized test?

[Chris]: To be effective, marketers need control. In their tests and campaigns, they need to be able to make content, layout, and messaging changes wherever necessary. And they need to be able to do it quickly. So, Test&Target is designed for marketers who aren’t technical. Familiarity with html can be helpful as can the array of Omniture services options available to Test&Target customers.

[Manoj]: I’d like to know more about the specifics on the implementation of div tags and the fact that users have to pre-load their desired content in to the admin interface.

[Chris]: Our tags are javascript-based and allow marketers to use Test&Target’s online admin tool to change content in the tagged area. Implementing our tags is very straightforward and involves only a few lines of code. Once in place, the tag can be used repeatedly of course. The content that goes into those areas is either created in Test&Target or, more commonly, resides in a content management system. We also have features to allow the bulk upload of content that some customers find useful.

[Manoj]: Can you explain how different types of content are rotated based on predefined configuration or content performance?

[Chris]: Test&Target’s behavioral targeting campaigns allow the marketer to specify content for various site locations. Our algorithms then automatically capture and learn from hundreds of a visitor’s behaviors and characteristics to determine which of the content options to serve to the individual. The traffic to most sites is incredibly variable, and so is the intent of an individual. So a system like Test&Target that not automatically reacts to but is made stronger by this variability is incredibly powerful. This is what we mean by learning algorithms.

In terms of predefined configuration, marketers can also set rules that the models consider when determining what content to show. For example, some visitors may not be eligible for a particular offer depending on their geographic location or based on some other characteristic. Our system takes these rules into account. And, what’s also unique about our approach is that it’s effective for anonymous visitors.

[Manoj]: What factors (metrics) are used to help the system target based on behavior?

[Chris]: Success criteria used by our targeting algorithms can range from basic click-through to revenue events depending on the objective of the campaign.

[Manoj]: How can offline data be integrated?

[Chris]: Our customers load offline data into Test&Target using a set of APIs. A lot of companies that have transactions completed offline leverage this capability. For example, we have financial services customers who measure test campaign success by both online loan applications as well as offline loan approval. The loan approvals are loaded into the system using the APIs. The use of vanity or custom toll-free numbers is another technique some customers use to test different messaging or calls-to-action in both the online and offline environments. In these cases, call center activity is loaded into Test&Target for campaign evaluation.

[Manoj]: How can the Test&Target framework be used with email campaigns?


[Chris]: I mentioned earlier that our tag is javascript-based. We also have a non-javascript version used for configuring test content in emails. A leading coffee chain, a national moving company, and a large technology company have all recently tested email content with great success. This technique is also really compelling since it not only results in more effective email campaigns but essentially allows the content of unopened emails in inboxes to be altered to show fresher content. People are also thinking more and more about optimizing entire experiences rather than just pages. What this means is that testing and targeting don’t occur on a single landing page or even solely on the site. Instead, it starts at the first point of contact with a customer such
as in an email, continues across multiple site pages, and lasts for multiple visits. This kind of thinking reflects a real and deep focus on delivering relevance and is at the heart of Test&Target.

SES NY: Widemile Launches New Multivariate Optimization Platform

Widemile Inc., recently announced the beta availability of its third-generation optimization platform, a browser-based application which helps online marketers maximize campaign performance by testing and optimizing landing pages and other web content for higher conversion rates. Widemile also announced thirteen (13) industry partners who will use the new platform on behalf of their clients. Partners participating in the initial platform rollout include Ascentium, Avenue A Razorfish, Brand Digital, Closed Loop Marketing, DDB in Seattle, Palazzo Intercreative, POP, Portent Interactive, Red Bricks Media, SolutionSet, Stratigent, TMP Directional Marketing, and ZeroDash1.

Late last week I had a great chat with the CEO of WideMile, Robert Bergquist regarding the new optimization platform:

[Manoj]: Please start with an overview of the optimization platform including the main benefits

[Robert Bergquist]: Widemile is introducing its third-generation optimization platform, a browser-based application which makes every advertising dollar work harder and become more productive by increasing conversion rates, overall campaign performance, and return on ad spend. For interactive agencies, benefits include: (1) Extension of their services offerings to include optimization and realize incremental revenue; (2) Enhanced performance of their client campaigns; (3) Inclusion of process and methodology – critical to client success; (4) Cost-effective licensing and business model that scales with their business activity; and (5) Eliminating demands on IT staff for implementation and support.

[Manoj]:
Who is the target market for your platform?

[Robert Bergquist]: Interactive agencies, SEM and SEO firms, and other marketing service organizations serving online advertisers.

[Manoj]:
How does the Widemile platform integrate with a person’s web property?

[Robert Bergquist]: A small encrypted JavaScript tag is attached to the page to be optimized, typically residing on the advertiser’s domain. This tag provides the instructions for the calls to the Widemile platform for tracking and rotation of content elements, which are overlaid onto the page in a matter of microseconds. Each user sees one specific content combination, which is consistent upon subsequent visits. The testing program is invisible from a user perspective. Importantly, SEO quality scores and activities will not be impacted.


[Manoj]: What kind of support is available with your product?

[Robert Bergquist]: Partners will receive a combination of online documentation and reference materials, phone and email support, training, and access to Widemile creative, optimization, and implementation resources.

[Manoj]: How were the media firms you partnered with approached?

[Robert Bergquist]: Widemile identified a broad cross-section of different types of agencies – large and small, media or creative-focused, SEM specialists, traditional, and broad-based interactive. We then targeted the leading firms within these subsets to start a dialog on our new platform and engagement model, and how it can help their clients. The response was overwhelmingly positive and exceeded our expectations.

[Manoj]: What is the approx. cost for your service?

[Robert Bergquist]: Widemile’s goal with pricing is to reduce barriers to optimization. Widemile has taken the approach for agencies to easily get started on Widemile’s Optimization platform. Platform license fees scale with amount of optimization activity, which is highly aligned with agency and client success.

The new Widemile optimization solution allows agencies to help their clients maximize campaign performance and return on ad spend. Widemile has honed its advanced multivariate testing and other proprietary analytic techniques, which are embedded in the application to scientifically determine the best performing set of advertising content — whether on a landing page, microsite or other web page. Marketers are able to simultaneously test a variety of offers, text, images, and other key variables, quickly and efficiently.

“Widemile’s third-generation software-as-a-service (SaaS) multivariate optimization system was specifically designed using open software and systems to meet enterprise standards for security, stability and performance,” said Dean Kimball, Widemile co-founder and CTO. “Developed with partners in mind, the Widemile optimization system contains a wide range of testing, reporting and client management capabilities within an easy-to-use browser-based application, and provides a level of performance and interactivity that has previously only been possible with desktop applications.”

Optimization can be challenging to execute, requiring both expertise and technology for client success. In addition to integrated “help” wizards within the application, Widemile provides partners with training and support, including proven best practices. The Widemile solution is designed to integrate with and complement partners’ other applications and services to maximize results for their clients.

Interview: Widemile CEO – Robert Bergquist

A little while back I got the chance to speak with Robert Bergquist, CEO of widemile (specializing in multivariate testing and optimization – based in Seattle). Widemile is recognized as one of the leaders in multivariate testing consultation and analysis therefore it was very interesting to get Robert’s insight on the company’s background and latest case studies.

Who are you and what is your background?

I love helping companies with great innovations find success in the marketplace. I’ve tried to make a difference in this regard in several roles, not just as CEO, over the years. (See my bio below for the boring details.) The consistent thread has been spotting great ideas and talent – doing my best to help both succeed. With my background helping companies in media, broadcasting, software and advertising, the Widemile value proposition (higher returns from online ad spend!) was a natural fit. I started as an investor and advisor to the company. Eventually, I was asked to join full time and take the reins. I haven’t looked back since, nor had as much fun. I’m not sure how it fits within this narrative, but other “real life” jobs I’ve held in the distant past are tire builder and commercial Alaska salmon fisherman.

In his 25-year career running and counseling companies, Robert has served as CEO and President, as well as in other roles, to a variety online marketing and advertising, broadcasting and other technology-driven businesses. Prior to his current position, Robert ran an online collaboration applications provider which was acquired by Hewlett-Packard. He has held positions with Bankers Trust Company (now Deutsche Bank) and worked as a corporate attorney and partner advising companies on finance issues and corporate transactions. He has also provided consulting services to businesses, including Microsoft and venture funds. Robert is active in the Seattle technology community, including serving on Washington Technology Alliance committees and as past Chair and President of the MIT Enterprise Forum of the Northwest. He holds a B.A. from Colorado College and the London School of Economics and a J.D. from Northwestern University School of Law. Robert serves on the Seattle Aquarium Society Board of Directors and is an advisor to Red Feather Development.

What is Widemile all about (i.e.: services, offerings)?

Widemile helps companies achieve dramatically better results from their online advertising – meaning higher customer conversion rates and more revenue. We have helped many companies, including The Weather Channel Interactive (http://www.weather.com/), Concur (http://www.concur.com/), SmartSheet.com (http://www.smartsheet.com/), Webex (http://www.webex.com/), Microsoft (http://www.microsoft.com/), Miva (http://www.miva.com/), and Telenav (http://www.telenav.com/), to name a few.We focus on delivering superior results to our clients by leveraging our own advanced multivariate testing and predictive analytics technology, along with an array of professional services, which includes full conversion marketing consulting, “conversion-focused” original creative development and MEC-certified Optimization Analyst project management. Our testing and optimization solutions can be applied to improve the performance of landing pages, microsites, sales funnels, registration pages or other key web pages with a visitor conversion goal – even if it’s just clicking through to the next page. All sources of traffic benefit, including paid and organic search, banner display ads, email, affiliate and direct.

What makes Widemile unique – What’s Widemile’s competitive advantage?

Again, we focus on results. We do what it takes to maximize our clients’ return on ad spend by testing, tuning and optimizing their high-value web pages to maximize customer conversion rates. We have found that great testing and analytics technology is just part of the success equation. Great conversion results also require expert marketing analysis (you have to understand the traffic, for instance), outstanding original creative and copy to communicate the value proposition in an effective way, and expert test design, set-up and interpretation. A simple testing tool alone will not deliver superior results. Our competitive advantage is superior results delivered using our proven testing methodology, original creative development and technology platform.

Can you talk a little bit about the level of knowledge surrounding A/B and Multivariate testing for SEM companies?

Most good SEM companies understand the importance of testing for superior results. However, they apply testing methodologies, such as A/B split testing, to the front-end the sales funnel to improve key word performance and optimize text ads. Some have begun to partner with web page and content testing companies like Widemile, Offermatica and Optimost in order to offer true multivariate testing of web pages as well. We almost always see an increase in the key word budget following a landing page optimization project where the traffic source is paid search – when returns on ad spend increase, most marketers will increase the spend.

At Emetrics you presented a multivariate testing case study for the Weather Channel – can you give some insight on what Widemile did and the results you noticed?

Widemile worked with The Weather Channel Interactive to optimize website customer acquisition for Notify!, a core subscription product that provides consumers with highly relevant and timely severe weather information. Launched in late 2002 as The Weather Channel’s first paid subscription product , Notify! provides consumers with highly relevant and timely severe weather information. The Weather Channel wanted to capitalize on the immense interest in weather information and the traffic provided by weather.com to capture new subscribers for the Notify! service. Initially, Notify!’s access to the massive traffic from the weather.com home page was leading to new subscribers, but conversion rates could be improved further. The landing pages had to quickly and clearly convey the unique selling proposition of the Notify! to weather.com consumers – sometimes in 8 seconds or less. The Weather Channel had done some optimization testing previously, but found the linear/sequential testing approach had limitations – namely timeliness and the identification of the most relevant factors that influence consumer behavior and conversion.

Working with Widemile, The Weather Channel was able to identify traffic sources and consumer segments to understand key user demographics and conversion behavior. Widemile devised an action plan which refined the main value proposition, developed a brand new page based upon best practices, and then designed and executed the multivariate test on new design. Six page elements (factors) were tested, with up to 4 versions of each factor presented randomly to site visitors. For example, 4 versions of the main image were tested, as well as 4 versions of the “hero shot” image on the left hand side of the page. The image of Jim Cantore, on-air talent for The Weather Channel, was tested against other images of snowy or stormy weather conditions to assess which worked better. Widemile tested the equivalent of over 1000 possible webpage combinations for The Weather Channel, including Flash® animated image versions.

The results far exceeded its goals
and also delivered some unusual findings. The goal was to improve conversions by 30% over the previous Notify! landing page – any slight increase in conversion rates equaled a big increase in revenue due to the high levels of traffic. In fact, actual results improved results by more than 225% over the original control version. And, contrary to commonly held industry wisdom, the Flash version outperformed its static counterparts by a large margin.

Do you have any additional case studies that people can look at?

See the Smartsheet.com case study. Webinar available at http://www.widemile.com/company/case_studies.html

How Smartsheet.com maximized results and leveraged SEM investments through multivariate testing

Challenge: Smartsheet is an online provider of collaborative task management software and services. Using SEM extensively, Smartsheet.com successfully employs numerous keyword and text ad campaigns through the major search engines. Smartsheet.com, however, felt performance could improve further.

Solution: Smartsheet partnered with Widemile to strengthen the linkage between SEM and landing pages. Using multivariate testing and other optimization techniques, Widemile helped them to increase conversions and performance of the entire campaign. The top 5 Smartsheet.com landing pages, representing more than 50% of pay-per-click (PPC) traffic, were targeted for optimization. Widemile employed its proprietary multivariate testing technology and predictive analytics, original creative development and professional services to accelerate this process for optimal results. Widemile tested the equivalent of over 1000 possible page combinations during the 3-week optimization process.

ROI: The best-performing templates improved conversion rates by an average of 64% over the initial control versions. This translates into a dramatic improvement of 115% in conversion efficiency – capturing over twice the number of conversions for the same level of overall SEM investment!