Debugging Google Analytics code (II): A Tutorial Video on Fiddler’s Inspector and AutoResponder functions

February 7, 2012 By Lukas Oldenburg Leave a Comment

Fiddler is a tool that many web developers are common with. It is my favourite tool for debugging existing tracking code and testing new code. This video shows two useful functions for this purpose: Inspectors and the AutoResponder.

Fiddler is a free Web debugging tool. You can download it at fiddler2.com. Fiddler logs all HTTP traffic between your computer and the internet. That means you can view just about everything that is exchanged in the background while you’re browsing the web.

What makes Fiddler so great

Apart from the functions shown in the video below, I love Fiddler because of at least one more reason: It is not dependent on any browser. Tools like Firebug, Chrome’s Web Developer Tools or the Chrome Extension “Google Analytics Debugger” (see the first article on how to use these tools) all just work within one or the other browser. But Web Analytics code is usually JavaScript code. And JavaScript works differently from browser to browser. That does not mean that Fiddler should be seen an alternative to Firebug. Both tools have their purposes and go very well together.

The following video shows how to use Fiddler for Web Analytics purposes. You will learn:

How to test existing code with Fiddler’s “Inspectors” feature. The use case will be the test whether a click on an application button really triggers the desired Virtual Pageview call
How to try out new or different tracking code without having to upload anything to your server (no IT needed!).

Filed Under: Errorlytics, Google Analytics, Web Analytics, Web Analytics Implementation, web analytics tools

Basics of Debugging Google Analytics Code: GA Chrome Debugger and other tools

January 20, 2012 By Lukas Oldenburg 4 Comments

**7th Feb 2012 – you can now watch the second instalment A Tutorial Video on Fiddler’s Inspector and AutoResponder functions**

Is that Virtual Pageview really being sent to Google Analytics when someone submits my registration form? Does the Event Tracking Call for outbound links get to GA in time before the visitor has left my website? Why do I not get any data for clicks on my teaser links? Questions that can drive a Web Analyst mad. Not if you know how to debug.

When I started using Web Analytics tools, there always was this huge black box: I’d put the tracking code on each and every page, and from then on, I usually waited and hoped the code would work as intended. Often, it didn’t, but I wasn’t able to see that until hours or sometimes a day later when the reports poured in to Google Analytics. Sometimes, it seemed to work, but only in some browsers. And so on. The more complex the code got (like when using Event Tracking for Flash applications), the harder it was to find out whether the code actually did what I was hoping it did. Luckily, that time is over.

Why is tracking code debugging so tedious?

Time lag: It usually takes several minutes to hours for the data to pour into the reports (even real-time analytics is never really real-time)
It was me, wasn’t it? You sometimes wonder whether the data in that report was caused by you or someone else that accidentally did the same thing on your website
No access to IT resources: Web Analysts may know some JavaScript, but we mostly are not IT people, so if you are testing web analytics code you sometimes have to wait for your IT guy to upload every little code change. An absolute killer for continuous improvement!
Testing on a test system isn’t the same as on a live system: There might be other scripts that are being loaded on the live system, the test system might be on another subdomain etc… Things that can be crucial for your code to function.

You can certainly name several more reasons why debugging the old-fashioned way is a very frustrating and time-consuming exercise.

Common Debugging Tools

There are some good tools out there to switch on the light in the tracking code’s black box:

Fiddler2
My absolute favorite (see next post on how to use this HTTP traffic monitoring tool). Fiddler2 makes it easy to view every request to Google Analytics (and any other javascript-based Web Analytics tool). You can even try out changes to your tracking code on your live system before releasing them to everyone, the tool is independent from browsers – and it is free!

Web Analytics Solution Profiler (WASP)
A well-known Firefox add-on by Stéphane Hamel that now belongs to iPerceptions. It recognizes a wide array of Web Analytics Tools’ tracking codes and shows you if your tracking code is successfully being executed (or not), crawls your site to find untagged pages, and so on.

A drawback is that it only works in Firefox, although it allows you to fake another user agent (like “iPad” or “Internet Explorer” which can be cool.

The basic version is free, but to be honest, it doesn’t help me that much for my use cases. Maybe I am too stupid to use it the right way, my use cases aren’t the ones the tool is made for (like using several web analytics solutions on one website), or I should try the paid versions.
So feel free to share your experience with WASP here in the comments.

Charles Debugger
A tool similar to Fiddler2, see this blog post on how to use it for Web Analytics Code Debugging.

Firebug/Chrome Developer Console:
A tool that makes any JavaScript Debugging so much easier, so it is a must-have for Web Analysts, too. And if you use Chrome, you don’t even have to install anything, not even a browser add-on (see this video by a Google Chrome Developer with 12 tricks to get the most out of the Developer Tools). That helps when you have to debug on someone else’s computer for example.

The function I use most of the time apart from the Console (where errors are being logged) is the Network Tab. It can tell us if the tracking beacon has been sent to Google Analytics successfully. To find out, look for the __utm.gif request. If it displays a “200 OK” status code (see the green light in the screen shot), you know that Google Analytics has received the current Pageview or Event. You can take a look what is inside that request in the “Headers” tab (Cardinal Path’s Kent Clark’s marvelous “Cheat Sheet” helps interpreting the values).

Chrome GA Debugger / ga_debug.js
Google’s recommended debugging tool for Google Analytics is Chrome’s Add-On “GA Debugger”. It is basically a form of using the “ga_debug.js” script without having to alter your page’s code at all (if you use ga_debug.js, you will have to change ga.js into /u/ga_debug.js on every page you want to debug). Chrome GA Debugger is a nice and easy-to-use tool that logs every Pageview and Event that you send to Google Analytics in your Chrome Developer Console (right-click on any part of the page => “Inspect Element” → go to tab “Console”):

Chrome GA Debugger shows you in an easy-to-read format what is being sent to Google Analytics without having to understand or inspect cookie variables or the Network Tab of your Console. It gives you hints like:

Does my visit have the correct source/medium/campaign?
Are there pages that accidentally override those sources?
Are there pages where conflicting JavaScript or other reasons hinder the Tracking Code from being executed?

I am not using Chrome GA Debugger much anymore though because:

It does not help with most cases of Event or Virtual Pageview Debugging. Events or Virtual Pageviews are most often tied to a click on a link (if you want to track the clicks on outbound links for example). If you click on a link though, you usually get to a new page, in which case Chrome’s Console is being cleared – and the Event Tracking call with it. So before you can take a look at what is being logged, it is gone.
It is Chrome-only. What works on Chrome doesn’t necessarily work on Firefox or Internet Explorer.
It breaks down sometimes, especially if you click quickly from page to page. And once it has broken down, nothing is being logged anymore even if you click slowly again to ensuing pages. That freaked me out because it always made me wonder whether my code wasn’t working well. You can easily restart the tool though by reloading the current page, but, even so, that is one factor that caused me to rarely use Chrome GA Debugger.

So much for a brief overview on the matter and a deeper look into Chrome GA Debugger. In my next article, I am going to show you how to effectively use Fiddler2 on some real-life examples.

Filed Under: Errorlytics, Google Analytics, Google Chrome, Web Analytics, web analytics tools

Predictive Analytics World Conference – 15% Discount code for our Readers

January 13, 2012 By Angela Wilkinson Leave a Comment

Want to optimise your business performance by predicatively scoring customers, establishing new opportunities and seeing real examples of how predictive modelling is employed?

Predictive Analytics World are in

San Francisco March 4-10, 2012 (San Francisco Marriott Marquis)

with a program that promises to be the biggest cross-vendor predictive analytics event; covering today’s commercial deployment of predictive analytics across industries and software vendors. There will be over 40 vendor-neutral sessions, split into 2 tracks: i) All Audiences and ii) Expert/Practitioner.

There is an incredible range of sectors covered; banking, retail, high technology, non-profit and social gaming. Conference attendees will see how predictive analytics is applied by Alberta Motor Association, Altos Research, Broadspire, CA General Underwriters, CompassLabs, HP, Interclick, Kelley Blue Book, LinkedIn, Major League Baseball, NFL, MLB, and the NBA, Pfizer, Sisters of Mercy Health Systems, Social Media Research Foundation, University of Phoenix, USPS Office of Inspector General, Volunteers of America, Wells Fargo, Wikipedia, Yahoo!, YMCA, ZZAlpha, an IT support firm, a sales workforce, real-world examples in financial services emergency response, and true-to-life anecdotes based on miscellaneous enterprise successes, plus insights from projects for Anheuser-Busch, Dept. Homeland Security, and US Postal Service Office of Inspector General.

Keynote addresses will include:

Anne Robinson, Director of Supply Chain Strategy & Analytics, Verizon Wireless,
Piyanka Jain, CEO, Aryng.com, former PayPal Business Analytics Head, and
PAW Program Chair Eric Siegel,
plus special sessions from industry heavy-weights Andreas Weigend, former Chief Scientist at Amazon.com and John Elder, CEO & Founder, Elder Research.

Web Analytics World Readers gain a 15% Discount when booking 2 Day and Combo passes. Register now using the code WAWBP12

And there’s more…

Co-located with PAW San Francisco there is Text Analytics World running from March 6-7, 2012. Featuring over 20 sessions with case studies from leading enterprises in e-commerce, financial services, government, healthcare, high technology, insurance, retail, and social media; this conference will deliver case studies, expertise and resources to leverage unstructured data for business impact and deploy the latest text analytics technology.

TAW San Francisco’s agenda covers hot topics and advanced methods such as blackbox trading, customer service and call centers, decision support, document discovery, document filtering, financial indicators from social media, government applications, insurance applications, knowledge discovery, marketing and branding, product launching, sentiment analysis, social data, social media applications, test analytics software, topic discovery, voice of the customer, and other innovative applications that benefit organizations in new and creative ways.

Text Analytics World Registration

This post provides Web Analytics World with an individual discount code for use with our Readers and the opportunity to earn a free pass to the PAW or TAW conferences which we intend to give-away to a WAW mailing list member.

Filed Under: Web Analytics

Tracking On-Site Campaigns with Google Analytics, Part III: 4 pros and cons of Internal URL Referral Parameters

December 20, 2011 By Lukas Oldenburg Leave a Comment

Facebook used to have them, Amazon still has them – parameters in the URL that show through which on-site campaign link the visitor has reached the current page. But unless you really have a thoroughly thought-through system in place, you should think twice about using them.

Internal Referral Parameters are not Campaign Parameters

First of all, let me clarify what I mean by “Internal Referral Parameters” (IRPs): Those are not to be confused with (external) Campaign Parameters (for Google Analytics, read “utm_campaign” and the like). As stated in the first article of this series, you should never use Campaign Parameters in the links of your on-site campaign because they will override the external source of the visitor.

To give you an example for an IRP: If you visit Amazon.com’s homepage and click on “Books” in the left-hand navigation, you get to a page with the URL:

http://www.amazon.com/books-used-books-textbooks/b/ref=sa_menu_bo8?ie=UTF8&node=283155

See the “ref=sa_menu_bo8”? That is the IRP. It helps Amazon.com determine which links (=on-site campaigns) on its homepage, category pages, and so on are the most effective.

Another website that uses IRPs is YouTube. There, you often encounter URLs like

http://www.youtube.com/watch?v=xxxxxxxx&feature=related

Here, the “feature” parameter is telling YouTube which link you used to get to the current video (in that case, I clicked on a “related video” on the right-hand side.

We stopped using them, and so did Facebook
My company used internal referral parameters for some months, but we stopped using them for the reasons I will explain in the full article. So has Facebook, which had used them extensively until a couple of months ago. They now seem to have dropped almost all URL parameters. Apart from the challenges those can cause for web analysts, they had other negative side effects: It was not easy, for example, to get a clean or “canonical” URL of your profile or fanpage because there always was “parameter clutter” attached to it, and every page existed in the form of countless URLs. You’d even see websites linking to the “wrong” (non-canonical) version of their fanpage.

In the full article, I will look at four pros and cons of Internal Referral Parameters. Although they are easily combinable with conversion metrics and make some basic reporting easier, there are hefty issues like SEO problems, cluttered advanced reports and more work up-front if you want to avoid these problems.

Filed Under: Google Analytics, SEO, Web Analytics, web analytics tools, Website Optimization

How To ‘Excel’ At Search Marketing

December 16, 2011 By David Fothergill 6 Comments

I love Excel. There you go, I’ve said it. It’s not even a love/hate type thing – I genuinely find that my life is a better place because of this wonderful, agile, willing gem of a piece of software.

Excel Mug

Gushing, maybe, but in the 7 years I’ve worked in search engine marketing (SEM), I have almost certainly used Excel every day for one purpose or another. So what I really wanted to do in over the course of a couple of posts was share some of the key formulas that are most useful in this line of work and then expand these into examples of how I would use them on a daily basis to analyse and optimise campaigns.

Formulas With Specific Benefits For SEM

LEN(text)

What does it do?

Counts the number of characters used in a given piece of text

Why is this useful for SEM?

Whether writing advert copy for PPC or meta descriptions/titles etc for SEO getting a visual aid within Excel can really help you make the most of these characters available in these situations (especially combined with conditional formatting): AdWords Using Len Formula

VLOOKUP()

What does it do?

Uses a given text or numeric input to reference related information, grabbing it out of a table or list.

Why is this useful for SEM?

There are endless instances when you will be given data from two sources which need matched up. As an example, it could be your web analytics tool data and customer details from your offline sales team. This is probably the formula that’s given me the most satisfaction, and is probably one of the most useful single formulas in the toolbox:

2 Semi-Useful Data Sets:

Using VLOOKUP

One Even More Useful Data Set:

Tables without vlookup

SUMIF(), SUMIFS()

What does it do?

Allows you to specify conditions on which to add data from a specific range or array of data. For example, turning it spoken word logic you could have:

“Sum values from the specified column IF they match criteria 1 AND they match criteria 2”

Why is this useful for SEM?

With so many sources, mediums and keywords we will often find ourselves with raw data sets which are no use to man nor beast and are left thinking “If only there was a quick way to tidy this up into some meaningful order”. There are often many ways to tidy up data, such as creatimg subtotals, but I find SUMIFS to be the quickest and most flexible way:

Example of using the SUMIFS function

The Real Magic

Now, all these formulas are well and good, but the real power comes in learning to combine them in the right way, at the right time, to get actionable insights. I’ll go into this further with some walkthroughs of sheets which I regularly use for SEM analysis, but here’s a quick example which illustrates what I mean:

My ‘Keyword Category Potential’ Analysis Sheet:

For this report, we take data from our Google Analytics account, regarding current traffic levels and a success metrics, such as the ecommerce conversion rate, and match it against traffic estimate data pulled from the Google Keyword Tool.

The key requirement of actionable data is that there is significant context created by the chosen metrics. This is achieved by combining the click and traffic estimate data to get a rough ‘share’ of potential traffic. Pulling this into the table below is done on the fly by typing a keyword into the ‘Category’ column, and relying on the SUMIFS in to pull the relevant pieces of information from other sheets which use VLOOKUPs (amongst other formulas) to tidy up raw output from the tools used: Example Table

The final piece of the puzzle is to make it even easier to get the actions to take away from the analysis. I like to do this by populating a 2-axis scatter chart. A quick glance at the below tells me that there are a few keyword areas that fit the bill of having both a high potential to get more traffic, and a higher than average conversion rate:

Chart showing category metrics across 2-axis

Quickly and easily we can take from this that there are 3 categories of good converting keywords which have potential to drive more traffic

—————–

Note: all data sets have been arbitrarily created for the purpose of the post, so no client information is included whatsoever

Filed Under: landing page optimization, SEM, SEM Training, Tips, Web Analytics, Web metrics, Website Optimization

Tracking on-site campaigns with Google Analytics: Part 2 – The pros and cons of Event Tracking

November 17, 2011 By Lukas Oldenburg Leave a Comment

Welcome to part 2 of this series on how to track on-site campaigns, for example, teasers on your homepage. After taking a deeper look at In-Page Analytics in part 1, we are now going to dive into the world of Event Tracking and Virtual Pageviews and offer you a script that will track all the clicks on your homepage by default.

One of the best things about In-Page Analytics was that you don’t need to add any code to your page or alter your links. With Event Tracking or Virtual Pageviews though, there is no way around some additional coding.

Event Tracking or Virtual Pageviews?
So what’s the difference between Event Tracking and Virtual Pageviews again? To make a long story short: With Virtual Pageviews, you can track Pageviews for “virtual” URLs (you can decide their name), even though there is no real pageview. For example, anytime someone clicks a link on a teaser, you can tell Google Analytics to record a virtual pageview like “/teaser-click-to-url-x”.

Almost the same happens with Event Tracking. To stick with our example, when somebody clicks on a teaser link, you can tell Google Analytics to track this as an Event. The major difference is that Event Tracking doesn’t create fictional Pageviews and thus doesn’t inflate your Pageview-based data. Imagine that every click on every teaser on your homepage was tracked as an additional Pageview. That would skew your data a lot because it also has a major impact on other data like Bounce and Exit Rate, Pageviews per Visit etc…

I prefer Event Tracking
That is the main reason why I do not use Virtual Pageviews for this purpose. Be aware though that Event Tracking can also skew your bounce rate and Time on Site/Page metrics, for example when you use it to track outbound links (usually bounces). Luckily, Google Analytics has recently published a way to work around this with “Non-Interaction Events”.

In the full article, you will find advice on how to properly implement Event Tracking for on-site campaigns, including a javascript that automatically tracks all the teaser links on your homepage or category pages, and we will analyze 7 advantages and 5 disadvantages of Event Tracking as a means for tracking on-site campaigns.

Filed Under: analytics, Google Analytics, Web Analytics, Web Analytics Implementation, web analytics tools, Website Optimization

Google Secures User Data: Mixed Messages, Mixed Feelings

October 31, 2011 By David Fothergill 2 Comments

Google recently announced changes which will see them redirect all logged in users to https://www.google.com. The reason given for this change given by Google is that it will make search data more secure for it’s users, ensuring private data remains that way.

From a web analytics point of view the main impact is that no keyword referral data will be passed for organic searches in the above circumstances. Although Google played down the impact, stating that this would affect less than 10% of data, there was much uproar from the whole SEM community (I don’t think I’ve ever seen so many comments on an GA blog post or reaction blog posts), with much of the ire being directed towards what has been dubbed a highly hypocritical move: not applying the same user protection to keyword data for users clicking on AdWords ads. If user data protection is priority number one, then Google should have the courage to stick to it’s principles, especially when money is involved.

The reason for doing this is obvious – a great selling point of advertising through AdWords is the ability to accurately measure ROI and optimize performance at keyword level. To take this away would surely lead to a huge drop in revenue from the cornerstone of Google’s financial strength.

This lack of consistency is at the root of the frustration for the SEO community: by not committing fully to the cause which they give as the reason for the change, it clearly shows that Google is merely paying lip service to need to to show they are serious privacy issues. The move has all the hallmarks of a compromise; finding a middle ground to keep everyone within the Google team happy.

I specifically choose to say ‘reason’ rather than ‘defence’ in previous paragraph – in the big picture this is a positive move as you cannot deny that any move towards increasing the security of data around the web is hugely important. For anyone outside the SEM industry, it would be hard to find any negatives to this change. Ill feeling at this point from a minority voice is surely collateral damage that was expected and accepted before the changes was publicly announced.

With my web analytics hat back in place, I have to draw the conclusion that this change merely pays lip service to the idea that they are prioritizing the protection of user data.

This is a real shame, but I get the impression that no amount of outcry will change the stance Google has taken.

Change happens all the time and at the end of the day the best thing to do is to accept it, adapt and continue to do the best job possible with what data we do have. As Stephen Hawking wisely said “Intelligence is the ability to adapt to change.”.

Filed Under: analytics, Google Adwords, Google Analytics, SEM, Web Analytics