The good old Referrer started his life with a birth defect: It was misspelled in the original HTTP Protocol specifications (“Referer”). So to this day, developers are forced to misspell it in server-side programming languages. For Digital Analysts, even if you could spell it correctly, you should avoid it when measuring your online campaigns.
A client recently asked me to investigate a problem: The reports of the sites where his company had bought ads showed many more clicks than his Analytics tool (Google Analytics) showed visits from these referring sites. The differences were glaring and far more than the usual plus/minus ten percent you often get due, mostly, to the well-documented fact that clicks are not visits (see for instance this help article for Google Analytics and AdWord data).
Twice as many clicks as visits
In the client’s case, the clicks roughly doubled the visits. To give you an example, the fashion search engine “Stylight” had wrought 4,199 (non-organic) clicks according to Stylight’s reports. But there were only 1,794 visits in Google Analytics. An exceptional case? No. Another client had a similar problem with newsletter campaigns, a third one with his Facebook Ads.
The “Referring Sites” report list far from all of your referring sites
The reason: All these clients relied on the “Referring Sites” report in Google Analytics. The underlying thought: If, in this report, we apparently see all the visits that originated from a link on other websites, shouldn’t we also get all the visits that originate from links in the ads on those other sites?
The answer: First, you get far from “all the visits that originated from a link on another website” in the “Referring Sites” report. Second, and for the same reason, relying on this report to measure the impact of campaigns is extremely unreliable – and a no-go for paid campaigns where you can determine what the links to your site look like.
There are just too many monsters out there on the internet that don’t like typos and devour referrers. They thus bloat your “Direct Traffic”.
So it is Google Analytics’ fault, right?
The question our project manager then had was: “So Google Analytics must be wrongly configured. What can we do about it so Google Analytics captures the referrer?”
But it is not Google Analytics’ fault! No Webtrends, no Webtrekk, no SiteCatalyst will be able to capture the referrer if the browser throws it away. Why? Because the Web Analytics tool gets the referrer from the browser, and if the browser doesn’t provide it, there is no way to get it.
The only way around this: Tag your campaign links. More on that further down.
How do I know if my referrer gets ditched? This bookmarklet helps
Referrer Amnesia is very common. So you may want to know whether you are suffering from it yourself. For this purpose, you can install this little bookmarklet into your browser. That way, you can easily see the current HTTP Referrer on every page by clicking on it. No need to open the developer’s console or install a plugin. Beware though: The HTTP Referrer is not necessarily the Referring Site you see in Google Analytics! It is just the referring page as the browser records it. The code for the bookmarklet is:
To install the bookmarklet into Firefox or Chrome, just right-click on the “Bookmarks” bar and select “Add page” or “New bookmark”. Then paste the code below. To add it in Internet Explorer, you just create a bookmark for a random site, then right-click on the bookmark and select “Properties”. Overwrite the original URL with the code above. In Safari for iOS (iPhone, iPad), you do it like in Internet Explorer (first create a random bookmark, then edit and overwrite it with the code above).
So with this bookmarklet (or your favorite debugging tool), you can more easily detect referrer problems.
The only sensible way to deal with Referrer Amnesia
But you should not care too much about the referrer in the first place. For anything that is business-critical – and measuring the success of paid campaigns is business-critical! -, tag your off-site campaign links with URL parameters. In the case of Google Analytics, those are the utm_source, utm_medium and utm_campaign parameters you append to the link; the first two parameters are very mandatory, the latter is just rather mandatory. By the way, Google Analytics will soon also allow a simple campaign ID parameter if that is your preferred choice.
Reasons for Referrer Amnesia
To close this off, let’s look at some of the most common reasons why the Referrer gets marooned:
- Links from https to http sites will never send along a referrer. That’s why Facebook ad links or Xing links to your http site will not have a referrer. So if you don’t tag your links, there is absolutely no way to see how much traffic or conversions your Facebook ads generate (see my article on how to solve the “https to http problem”)!
- IE also loses the referrer when the link is in a Flash application.
- The user views the link in a mobile app and uses an option like “open link in Safari”.
- In iOS 6, if you use Safari’s integrated search field up on the right side and then click through to the search result, there will be no referrer at all! That is, if the ensuing page is on “http”. That way, you also don’t get any search engine keywords either. Your web analytics tool will not even know the visit came through a search engine (even worse than “keyword not provided”). That is because Safari seems to use Google’s Encrypted Search by default. You can also use Encrypted Search in your desktop browser. To make things even more complicated, you WILL get the referrer AND the keyword if your site is on https!
- So if your site runs on http,
- you don’t get the keyword nor the referrer if visitors came through Google’s Encrypted Search or iOS’s Safari’s Google Search bar.
- you don’t get the keyword, but the referring domain (google.xy) if visitors came through Google’s SSL search (logged into Google account, the well-known “keyword not provided”).
- If your site runs on https,
- you will get the keyword and the referrer if visitors came through Google’s Encrypted Search or iOS’s Safari’s Google Search bar.
- If visitors use Google’s SSL search, there is no difference to what happens if your site is on http
- So if your site runs on http,
- Links that lead to pages with non-301 redirects (but you can configure other redirects in a way that they send the referrer along, I think).
- Browser privacy-enhancing add-ons like “No-Referer” (Firefox).
- Visits through links that are in a non-web page document (PDF, DOC etc.).
- The visit came via a link carrying the “rel=’noreferrer’” attribute.
Have you experienced Referrer troubles?
This is far from a complete list. So, to finish off, let me ask you: which referrer problems have you come across? How have you solved them? Which major referrer problem is missing in this list?