Latest Jobs

UK and US FlagsDigital Marketing, Social Media, Web Design, Development, Analytics and Security.

Measure and Management Tools

Figure with wrenchOur guide to useful tools by type and price.

Advertise with Us

blue dart on targetTake your products and services to our highly engaged audience.

Video Walkthroughs

Internet Marketing WordsStep by step on screen visual expert guides.

Hot Topics
Internet Law
Salary Reports
Mobile
Website Optimization
Hot Topics
Social Media
Facebook
Twitter
YouTube
Google+
Hot Topics
Digital Marketing
Analytics Tools
Google Analytics
Internet Business
Search our Site Content
Loading

Broad Match is Dead. Long Live Broad Match



When launching a new paid search account, restructuring an existing account, or creating a new campaign, limiting where your ads are shown is essential. Showing your ads to the right target audience and carefully selecting when your ads are shown will lower what you pay per click and allow you to allocate spend to new campaigns.

You simply cannot trust Google (pro tip: Google is in it for the money) or any other search engine. Google defines broad match as “any relevant variations of your keywords, including synonyms, singular and plural forms, possible misspellings, stemmings, related searches, and other relevant variations.” Google’s expansive definition creates the potential for a lot of unwanted impressions. Why give search engines the opportunity to select what keywords you target. At Obility we have seen egregious examples of Google displaying ads for keywords not remotely relevant to the target broad keyword.

The example below shows the search term report for a client targeting client brand checks broad match. When targeting keywords with the client brand name, advertisers can reasonably expect Google to only show ads related to the client brand. However, when viewing the search term report, we found not only our ads being shown for competitor terms but also for incredibly broad terms (e.g. business checks, order checks, print checks, harland clarke). Google basically chooses to display your ad for anything remotely relevant to the broad keyword you are targeting.

SEARCH TERM

Gain Control with Modified Broad Keywords

Conversely, you cannot limit your account to exact and phrase match either. This simply leaves too much opportunity on the table.

Modified broad match (mod broad in PPC lingo) should be used liberally. Mod broad allows for keyword discovery while reigning in Google overreach. Modified broad (where advertisers identify one or more keywords that much be in the search query with a plus sign e.g. +modified broad must include the term modified) give advertisers a powerful tool to target long tail keywords while maintaining significant control of what keywords are targeted.

Use mod broad and phrase match keywords for discovery: identifying new keywords that perform well. Exact match keywords should be used to apply stringent bid control. While modified broad and phrase give advertisers an opportunity to reach a larger audience, exact match keywords allow better control of what an advertiser is willing to pay for target terms.

In addition to mod broad keyword targeting, advertisers should also frequently use search term reports (STRs) provided by the AdWords UI. Obility recommends running account STRs bi-weekly. Run both ad group and keyword level STRs to provide information on both new keywords and negative keywords.

Tighten Targeting

While broad matching keywords can be damaging to account performance, negatively targeting broad match terms can significantly increase clickthrough rate and improve account performance. Use negative broad terms whenever you are not going to block a potentially relevant term (e.g. job and jobs should often be added as negative broad match terms). Use both campaign and ad group negative broad match terms. While campaign negative keywords are typically used to limit where your ads are shown, ad group negative keywords are best used to prevent your target keywords from competing against each other (e.g. negatively target software in a general lead management ad group so that its keywords don’t compete with a lead management software ad group).

In addition to negative keyword targeting, controlling your account settings are essential. Google being Google defaults campaign settings to target misspellings and near match for exact match keywords. This means that if you target lead score, Google could potentially show ads for leading scorer. At Obility, we recommend turning off misspelling targeting for all campaigns other than Brand and Competitor.

When Broad Makes Sense

As with all blanket statements of trends being dead – think the constant refrain of SEO is dead – use of broad match targeting in paid search can make sense when used sparingly. The trouble with only targeting mod broad, phrase, & exact match keywords is that they are expensive. Advertisers willing to pay more for these terms has ratcheted up cost per click (CPC) leading to many instances where targeting broad keywords makes sense.

For example, we have a client offering email marketing as part of its platform. Email marketing terms are incredibly competitive where advertisers are paying $40-50 per click. Our client was paying over $35 per click for 5th position for many general email marketing terms when targeting with exact match. However when targeting broad match best email marketing software, our client would pay less than half the cost.

email marketing

The table above shows what our client was paying per click to appear for searches for email marketing (search term impression) using two different methods: one, we targeted best email marketing software broad match; the other we targeted email marketing exact match. As you can see, targeting exact match is considerably more expensive and led to essentially the same position. While targeting broad match relinquishes some control, Google tends to award that recklessness with cheaper CPC’s.

A Mixed Strategy is Best

Our recommended strategy is a paid search account that prioritizes top terms (i.e. high bids for top exact and phrase match terms) but also includes lower bids for broad match terms that are too expensive to target with phrase or exact match. Not surprisingly, this is a difficult balance to attain and requires quite a bit of experimentation.

Takeaways:

  1. Keep broad match targeting to a minimum
  2. Use modified broad targeting liberally
  3. Search term reports should be run regularly
  4. Copious use of negative broad terms
  5. Campaign settings should limit targeting of misspellings
  6. Broad keywords have their place
Subscribe by Email

Why we can’t just be SEOs: A reply to Rand Fishkin



Image of a globe with the title SEO representing SEO practisesLast week I saw this interesting whiteboard Friday which talked about ‘Why We Can’t Just Be SEOs Anymore’ by Rand. Though he has raised some valid points, like ‘perception of SEO is hard to change’, I have to disagree with him overall.  Sorry Rand, you are missing the complete picture. 

 

SEO is not bigger than SEO

SEO is all about generating relevant organic traffic to the website through search engines. That’s it. SEO is not about email, CRO, UX, Social Media, Branding, PR, Reputation Management, Coding, Advertising, Customer Service …

You may argue that there are 200+ ranking signals so I need total control on everything which influences SEO. While it is true that they are 200+ ranking signals, don’t forget the 80/20 rule i.e. 80% of your output comes from 20% of the input. If you have worked in the SEO industry long enough, you already know what that 20% is that will generate 80% of the SEO results.

That 20% consists of basic on-page optimization, keyword research, content development and above all link building. We can tweak brand signals, social signals, authorship, Page Rank, markups and other weak ranking signals all day long but they won’t generate any considerable amount of organic traffic on our website. What really drives traffic is that 20% I am talking about.

If we talk about the real world (which could be very different from the blogging world) there could be unlimited ranking signals. For a start, your client is a very strong ranking signal for you. Without his support and cooperation you can’t make any change on his website. No amount of SEO is going to help, if the client is not responsive to your needs and demands. Poor product, bad reputation, poor customer service all are sort of ranking signals which are beyond our control. 

Just because something may impact your SEO so you must develop expertise in it or take total control of it is a wrong mindset. Here is why. When someone works as a marketing generalist who knows little bit of everything (well sorry but this is what specialists really think about him) he is eager to give suggestions to specialists (like CRO consultant, PR consultant, Community Managers etc) on how they can do their job better.

Since he is not a specialist, his suggestions may not be well received or align well with the recommendations of specialists. This creates disruption in digital strategies and work environment. So instead of creating synergy the marketing generalist could inevitably end up creating stress and chaos. 

 

The Perception of SEO is hard to change so we should change the job title

There are lots of job titles out there which have earned a bad name over time like car salesmen, lawyers, brokers, cold callers etc. There is hardly any industry left where you won’t find people who give the whole industry a bad name. Even in a profession as noble as Health Care you can find doctors who steal kidneys and are involved in the illegal organ trade. So have they changed their job titles? No.

So this argument of bad perception of SEO is not strong enough to drop the SEO title. 

 

We are selling ourselves short by not calling ourselves a SEO

That is true. When we say we do email, CRO, UX, Social Media, Branding, PR, Reputation Management, Coding, Advertising, Customer Service….all in the name of SEO or whatever titles we prefer, say ‘Inbound Marketer’, we are not only spreading ourselves too thin but are abnormally raising the expectations of our clients. With so many marketing channels being leveraged they will definitely earn tons of money. Won’t they?

We are also holding ourselves responsible for client’s sales ability as we are now in charge of CRO, UX, branding, customer service, PR etc. All of these are full time jobs in themselves. And if we have to do all of these jobs and practically run the entire business then a monthly retainer can never be profitable, partnership in client’s business and profit will be. While we can make SEO as big as we like, do as much volunteer work as we want, don’t expect the clients to handover their business to us just because we practice every marketing tactic in the book. At the end of the day even business development managers (who are specially hired to develop businesses) are handed over a monthly pay slip instead of partnership in business and profits, let alone SEO. 

 

Subscribe by Email

Take Politics Out of the Decision Making Process with Testing Part 3



This is part 3 of a multi-part post on website testing.
Part 1 is available here
Part 2 is available here

Alas, we reach the end of this blog series on site testing, but hopefully this is just the beginning of your journey in conversion optimization.

As promised in part 2, this last post will focus on the differentiating features, the reporting functionality, and the summary of tools.

Differentiating Features

Based on my experience I’ve found a lot of A/B testing tools to be very similar, if not downright the same thing. What differentiates one tool from the next often comes down to price alone. My goal here is to ignore price (all three tools were relatively the same price) and focus on the fundamental differences.

Convert Experiments

I was pretty excited to see the types of Google Analytics integrations Convert Experiments offers. All three tools integrated with Google Analytics to push test information into custom variables, but only Convert Experiments allows integration with Google Analytics goals and ecommerce tracking (with the exception of Google Analytics experiments).

Optimizely

I really liked the straight forward approach of Optimizely. If you’re just looking to set up a test and go, Optimizely has a very quick startup. The standout feature of Optimizely is the implementation… 1 line of code, AND you get emailed when the code gets added to a site. This is incredibly helpful.

Additionally, Optimizely integrates with KissMetrics, MixPanel, and SiteCatalyst. It also integrates with click tracking tools like ClickTale.

 

Optimizely-ClickTale

 

Visual Website Optimizer

I felt Visual Website Optimizer had the best overall feature set of all the tools tested. Not only is the targeting incredibly comprehensive and flexible, it has direct integration for “focus group” testing, straight geo-targeting testing and my absolutely favorite… click maps!

Visual Website Optimizer eliminates some of the need for tools like CrazyEgg by incorporating click maps directly into the testing. Additionally, if you only want click maps, you can do that sort of testing alone.

VWO-ClickMap

 

Reporting

I already demonstrated in Post 2 how the different tools report within Google Analytics, but let’s dive into how each tool displays the results.  I’m not providing much commentary on the reporting data and graphs themselves as everyone has a different preference for reporting.

Convert Experiments

I was pretty excited to see the types of Google Analytics integrations Convert Experiments offered. All platforms integrated with Google Analytics to push test information into custom variables, but only Convert Experiments allowed integration with Google Analytics goals and ecommerce tracking (with the exception of Google Analytics experiments.)

Convert-RevenueReport

Optimizely

Report summary is pretty clean, easy to tell which had the highest engagement effect, scrolling to the right shows the different goals.

 Optimizely-ReportSummary

The detailed goal reporting is also pretty clean. It’s easy to tell what’s doing better or worse.

Optimizely-ReportChart Optimizely-ReportData

 

Visual Website Optimizer

Reporting is pretty similar to Optimizely on the summary front, but I found this interface a little more convoluted than Optimizely.

VWO-ReportSummary

I like the detail reporting a bit better in Visual Website Optimizer than the other 2 tools -It’s very clean and right to the point.

VWO-ReportGraph VWO-reportdata

 

Table of Features

 

Google Experiments

Convert Experiments

Visual Website Optimizer

Optimizely

Google Analytics report integration

✔+

Custom Variable Slot

Custom Variable Slot

Custom Variable Slot

Segmentation

 

IP Address exclusion

 ✔ (filters)

✔ (experiment level)

✔ (account level)

✔ (account level)

Import goals from Google Analytics

 

 

Heat/Click maps

 

 

 

Split URL Tests

✔ (redirect)

A/B & Multi Variate Testing

 

Flexible Goals

Google Analytics revenue reporting

 

 

 

Closing Remarks

At the end of the day, the important thing to take away from this blog series is that you have no reason not to test. The tools out there are easy to get the hang of and use and the reporting is fairly straight forward. The real challenge for you is to decide where to start with your tests and what your success measurements are going to be.

So, with that said, go forth and test! Or, Tweet me (@MattAster) with your email and URL and I’ll give you 2 free optimization ideas to test!

Subscribe by Email

Customer Cyber Security Concerns



gavelsmall

Business customers are increasingly raising questions about the security provisions that suppliers have in place to protect customer data that is supplied to them via the Internet or stored online in the cloud. Customers are increasingly insisting on having onerous rights of audit in agreements with their suppliers to enable the customer to monitor and check the supplier’s compliance with such security provisions.

For example, under the UK’s Data Protection Act (DPA) customers (data controllers) are required to take appropriate technical and organisational measures to prevent the:

  • unauthorised or unlawful processing of personal data; and
  • accidental loss, destruction or damage to personal data.

In order to comply with these duties and avoid substantial fines, customers need to ensure that their suppliers have adequate security measures in place to prevent data protection breaches from occurring.

Due Diligence and Auditing

When dealing with government departments or customers who process financial data, customers often want to carry out due diligence on a supplier’s security systems at the pre-contractual stage. This will include ensuring that the customer has the right to check the supplier’s security measures and the supplier’s on-going compliance with the security provisions set out in any terms and conditions during the term of the agreement.

Information Security Officer

Both suppliers and customers should consider appointing an information security officer to assess their cyber risks. Once appointed the information security officer will be able to deal with any security issues when they arise, at the pre or post-contractual stage. Without an information security officer many organisations will lack sufficient knowledge or understanding of actual cyber security risks. Customers will be unable to carry out a proper due diligence and suppliers will be unable to respond adequately to customer queries. This will result in the customer and supplier spending unnecessary time on the negotiation of the supply agreement.

Hackers are increasingly accessing online data (in particular online payment details) and using new methods to do so. An information security officer once appointed could monitor and detect such problems, keep up to date on the latest security countermeasures, deal with queries (not just in the contracting process but also from concerned data subjects – customers) and report to management on a regular basis.

Notification and Response to Cyber Breaches

Both suppliers and customers need to have obligations to inform each other of security breaches (such as hacking) in order for both parties to deal with the issue in a timely manner. If there is a slow rate of detection the potential for the scope of the data breach (and fines) increases. Also, once a party has been notified of the breach, the incident needs to be quickly contained to limit any further potential damage.

Passwords

One of the most common causes of security breaches is the use of inappropriate passwords such as the use of “password”. Both the supplier and the customer should have adequate systems in place to monitor and prevent the use of such passwords. Systems and procedures should also be in place for the regular changing of passwords to minimise the risks of a security breach via misuse/use of unsuitable passwords.

Summary

These are just some of the general security issues that customers and suppliers should consider when entering into an agreement which involves the transfer of personal data to a supplier. There are many other issues which also need to be taken into account depending on the business sector in which the customer operates and the types of data that the supplier is processing.

image courtesy of Sam Howzit

Subscribe by Email

Take Politics Out of the Decision Making Process with Testing Part 2



This is part 2 of a multi-part post on website testing, part 1 is available here

Part 3 is going to include reporting features and final summaries.

In my continuing journey to open eyes to A/B and multivariate testing, I’m going to show you 2 more tools this week. On the docket: Visual Website Optimizer , Convert Experiments, and Optimizely

For purposes of this post, we are going to ignore Split URL tests. In my humble opinion, if you want to run split URL testing you only need Google Analytics Experiments. For this comparison I am going to focus on multivariate testing

The easiest way to do this comparison is to break this up into 3 sections.

  1. Creating test variations
  2. Setting up the test (implementation, targeting, etc.)
  3. Results and reporting (I’ll cover this in Post 3)

Creating Test Variations

Both tools are WYSIWYG (what you see is what you get). Simply put in the URL you want to test and it spits back the page with every element clickable, allowing you to edit each.   You’ll notice the editing elements are fairly similar between the tools. Each platform lets you choose to “Track Clicks” to determine goal success and every platform lets you edit the element.

In all 3 platforms you are able to edit the HTML directly, perhaps to add custom JavaScript or CSS styling. In Optimizely you have an advantage of adding custom JavaScript directly (instead of editing the HTML first).

 

Visual Website Optimizer

Convert Experiments

Optimizely

 VWO-ElementOptions Convert-ElementOptions  Optimizely-ElementOptions

 

On our homepage, I wanted to test the effects of pausing our auto-rotating banner. Our site sits on a DotNetNuke CMS and the only way to turn off the auto-rotator was to insert a line of JavaScript telling the normal rotating banner to load as PAUSED. 

Since the JavaScript had to be applied at the page level, I could not utilize the Pause button highlighted in the screenshot below.

 PD-Banner

For this, I needed to edit the javascript of the page. The 3 platforms each have this option. For Optimizely I had to select an element and go to parent until I hit the Body, then I could edit the HTML (I felt this was very messy as it required me touching the base of the site for a very simple test.)

Visual Website Optimizer and Convert Experiments allow you to add custom JavaScript to the specific variations without needing to touch the overall site code. *Note, I verified with Convert Experiments and global is being renamed to avoid any confusion, it DOES apply only to the variation.

Visual Website Optimizer

Convert Experiments

Optimizely

 VWO-CustomJavascript1 VWO-CustomJavascript2  Convert-AddingJavascriptToPage  Optimizely-CustomJavascript

 

Test Setup                                                

Implementation

I’ll start with the part most people are going to hate (those that aren’t developers): implementing the code needed to actually execute the test. All 3 platforms have a similar process, add some JavaScript to the head of your website, and move on.

  • Convert Experiments does execute asynchronously.
  • Visual Website Optimizer has an option for async or standard
  • Optimizely’s code is NOT async, but they do have an option for asynch if you contact them… however, the synchronous option was the easiest install with only 1 line.

All 3 platforms give a nifty “check code” after you are instructed to implement it. Optimizely even sent an email to me to let me know the code was implemented. This would be incredibly helpful if you work in an organization or an agency where someone else has to implement the code.

Targeting

All 3 platforms have targeting customization some way or the other; look at the table below to get a sense of how each is structured. I’ll note Optimizely DOES offer geo-targeting, but only at the Platinum level (my testing was done at the Gold level).

I found Optimizely’s targeting a little lacking, compared to the options available in the other 2 tools. But theoretically the Custom JS condition could add these functions, although it would require some serious JavaScript knowledge.

 

Visual Website Optimizer

Convert Experiments

Optimizely

VWO-TargetingOptions 

 

 Convert-TargetingOptions  Optimizely-TargetingOptions

 

Traffic Allocation

I mentioned in part 1, if you have a particularly sensitive test to run, choosing what percentage of your visitors to test is a key element to stakeholder buy-in. All 3 platforms have a traffic allocator.

Visual Website Optimizer

 VWO-TrafficAllocation1

VWO-TrafficAllocation3

 

And they have a nifty tool to help you estimate how long a test will need to run at a specific allocation. This tool will be great when you’re writing up your testing schedule.

 VWO-TrafficAllocation2

Convert Experiments

Nothing fancy here, just straight to the point.

 Convert-TrafficAllocation

Optimizely

Similar to Visual Website Optimizer, you’re able to specify allocation among variations.

 

 Optimizely-TrafficAllocation

Analytics Integration

All three tools have native Google Analytics integration. This is done via custom variables. .

The main difference between the three tools in this area is that Optimizely is the only one that lists what’s in the variation. This is incredibly beneficial for future reporting when you want to look back to remember what a test was.

 

Example, both Convert Experiments and Visual Website Optimizer

VWO-Convert-CustomVariable

Optimizely shows what each combination was. Granted, it’s truncated text, but at least it gives a sense.

Optimizely-CustomVariables
But that’s all the reporting I’m going to show you now! Part 3 will have the final summary and all the reporting goodness.

Stay tuned!

Subscribe by Email

What is Software as a Service – “SaaS”?



gavelsmall

SaaS is the common abbreviation used for “software as a service”. You may know this under another name, for example ASP services (application service provider), software on demand or software subscription services. These names all refer to the same thing – software being made available to users via the Internet.

Unlike standard software, with SaaS you are not given a copy of the software – on a cd or installed on your computer. Instead you rent access to software. No permanent access or ownership in the software is given to you. You are simply given the right to access the software in return for paying a subscription fee. When you stop paying the fee, you no longer have the right to access the software and will no longer be able to use it.

What is a SaaS Agreement?

A SaaS agreement is the name given to the agreement between a SaaS supplier and a SaaS customer which sets out the terms under which SaaS software may be used and accessed. This will usually include a service level agreement (SLA). SaaS agreements are also sometimes referred to as subscription agreements, subscription licences or ASP agreements.

Differences between a SaaS Agreement and a Standard Software Licence

A SaaS agreement differs from a standard software licence in that:

  • the SaaS customer will not usually receive a physical or installed copy of the software on their computer
  • no ownership in the SaaS software will be transferred to the SaaS customer
  • the customer’s right to use the SaaS software will end upon termination of the SaaS agreement.

In a SaaS agreement the customer is simply “renting” access to the SaaS software for the duration of the SaaS agreement.

Is a Service Level Agreement (SLA) a Software Licence?

No. A service level agreement (SLA) sets out the SaaS services being provided in addition to the right to use the SaaS software. These will usually include:

  • hosting of the SaaS software on the SaaS Supplier’s server
  • hosting of the SaaS customer’s data on the supplier’s server
  • provision of customer support services in relation to use of the SaaS software; and
  • provision of software maintenance services to update and maintain the correct functioning of the SaaS software.

image courtesy of Sam Howzit

Subscribe by Email

Take Politics Out of the Decision Making Process with Testing



This is part 1 of a multi-part post on A/B testing, stay tuned for future updates

I saw this Dilbert comic on Facebook and it really got me thinking about a question you could ask the same person 5 times and get 5 different answers. All too often we’re asked to pull up metrics for someone only to see the final report exclude all metrics because someone had a “gut” decision, or they got overwritten higher in the organization.

 

DashboardMetrics

 

Here’s how I imagine a conversation going with a Manager at most Fortune companies.

Matt Aster: How do you make a decision about checkout page order?
Joe Smith: I have one of my analysts look at the data and then we make a decision.

3 Months Later

MA: The data said you should ask CC/Exp before Name and Address… why didn’t you do that?
JS: We mocked that up and the Marketing Director didn’t like it
MA: Did you show the Director the data to support your decision?
JS: Yes, but bottom-line she decided it wasn’t something with which she felt comfortable.

This isn’t even an imaginary conversation, I’ve HAD this conversation with folks. The bottom line is simple, if you’re going to ask for, and analyze, data then use the data. Often times we’re asked to pull metrics for someone and often times we do that without question. The issue is when you’re tasked with managing or improving an existing website/process/flow/email and politics get in the way.

How do you completely remove politics from decision making, or at least remove some of it? Let the data speak for itself by running A/B tests. If you run an A/B test and show that revenue increases X% on version B, even if someone is uncomfortable with it because it goes against their best instincts, they’ll be hard pressed to fight it when the revenue increases.

 

Before I begin:  You’re going to see “percent of visitors to test” very frequently in the tools I’m evaluating. This is where you’re going to be able to convince executives it’s ok to test. At my previous workplaces we had varying degrees, some tests (low risk, low politics) we’d run 100% of users through, other tests (where particularly ornery leaders would have a strong opinion) we’d run at 5% of users. Keep in mind, the smaller the % of users tested, the longer the test will need to be run to gain a good confidence.

Testing Platforms:

Google Analytics  Experiments

Formerly Website Optimizer, and formerly incredibly useful, Experiments allows for straight A/B testing where each version has its own distinct URL.

  1. Install Google Analytics 
  2. Design your experiment, keeping in mind you must use separate URLs, so if you’re designing an experiment that is during checkout you may have issues.
  3. Implement the code changes, the URL is less important, but write it down
  4. Configure the experiment in Google (see images below)
  5. Install the experiment code (unfortunately you cannot use Google TagManager due to the code for Experiments needing to be in the head)

If you’d like an in-depth how-to you can follow the instructions here.

Google Analytics Experiments Review

Let’s start off with the positives.

  • Experiments is fully integrated with Google Analytics, so if you’re familiar with Google Analytics you’re going to find navigating the Experiments section very similar.
  • When you begin having data you might notice you didn’t include a goal you were hoping for. You can easily see the conversion experiment’s impact on Site Usage, Goal (all site goals), and eCommerce. 

 

Politics-ChooseDifferentMetrics

 

Now the bad

  • Here’s the big one… no multi-variate testing. Your only option is straight A/B testing with different URLs. This is a far departure from their original website optimizer stuff.
  • There is no test segmenting. Your choice is only the % of visitors you show the test to. In my next post I will be reviewing 2 tools that both allow segmentation testing (only show this to new visitors, or only show this to people in ohio). True, you can create an advanced segment to view the data by segment, but what if you only want to show it to specific people. This is particularly important for tests like dynamic content insertion from organic search results. 
  • Once you define a goal for the experiment you can’t change it. I mentioned above you can look at the experiment’s impact on everything, but Google will still declare a winner based on the goal you established when you setup the experiment.
  • You can’t share the variations with people outside of Google Analytics. The screenshot below is the final page in the experiment setup, if you click the link for “Preview All” you’d expect to be taken to a URL that you could share. Instead you’re presented with a modal window that allows you to click between the options. (To be fair, if you wanted to show someone the versions before you implement the Google code you could just share the URLs, but once the code is published you cannot share the original 100% of the time)

Politics-VariationPreviews

 

  • Lastly, you can’t pause experiments, you can only stop them and copy. You could remove the code from your original page, but that’s cheating.

 Here’s what a final report looks like taken from Google’s help files. In terms of reporting, Google does a great job making this clean. When it comes to integration and finding everything in one place, this is great.

GoogleExperiments-resultsReport

How to read the report

The graph at the top is your different variations and their conversion rate. You can adjust what metrics you’re looking at by changing the drop down, I wouldn’t recommend utilizing the “VS.” option since you’ll end up with a lot of lines.  The key column to look at is “Compare to Original.” In this test Variation 2 beat the original by 7% (not 7 points) and consequently Variation 1 and 3 both did worse than the original. Depending on the conversion goal, these would likely be the variations you’d delete. (I say depending on the conversion goal because there is further analysis needed, you would want to double check the other goal sets and ecommerce before you make a final decision.) Last note is the “Probability of Outperforming Original,” in this case Variation 2 only has a 66.11% chance to beat the original, I like to see 90′s before I call a test definitive. For this test I would continue to run Original and Variation 2 alone for another few weeks to solidify the winner.

Stay tuned for the Part 2 where I’ll cover 2 more popular tools for testing!

Subscribe by Email