The Twitter and LinkedIn algorithms both thought it would be a good idea for me to read Gary Angel’s latest post the other day. I’m glad they recommended it since it digs into some really interesting considerations around attribution models.
Giving Credit Where Credit is Due.
We are always attributing some outcome to some action in our analyses – at the most basic level when we report on how many visits came from each traffic source – we are attributing visits to a traffic source.
Attribution models help us understand how marketing channels work together to produce an engaged audience and impact business outcomes. For instance, I would not have visited Gary’s blog if I didn’t get a notification from my social media services that he had posted something new. Twitter and LinkedIn worked together to not just get me to read Gary’s post, but to read it over and over to inform my own POV on attribution models. Though I clicked on the Twitter link first, I don’t think I would have spent as much time with the post and the topic if LinkedIn hadn’t reminded me of it, and when I returned to the post to reference it for my own post, I typed in the URL directly. Obviously, I know that Twitter and LinkedIn both played an important role in generating a conversion for Gary’s blog, but can Gary know that? What kinds of modeling strategies would best help Gary see how those services work together to generate traffic to his blog?
The Model Is Not The Thing Being Modeled.
One issue with any kind of data modeling (whether it’s explicit as with attribution models or implicit as with A/B testing) is that it is, by definition, inexact. Despite this, there is a tendency when working with models to expect them to spit out a single number that answers everything and is the number.
Paul Krugman, an Op-Ed columnist for the NY Times and an Economics Professor at Princeton University, says everything is a model and models will be wrong sometimes. The challenge – and this applies to analytics as well – is figuring out why. If we can identify the weaknesses in our models then we can improve them.
In analytics, the results of our models are being used to justify strategic decisions and play a key role in deciding where to allocate resources (money, people, and time). This is why it’s important to understand not only what our models are telling us, but also what they are leaving out. And not fall in love with one approach or one number, but rather use all the tools available to generate the fullest picture we can.
Important Considerations For Attribution Models Today.
Gary’s post goes through the evolution of attribution modeling and then lays out a strategy for improving the validity of algorithmic attribution modeling (the best option available for attribution modeling today):
- Attribution considerations begin before traffic reaches the destination i.e.
- Where media is being purchased: If media is purchased on a website with a shared audience with the brand website we can assume that this audience would be likely to visit and convert on the brand website at the same rate regardless of if the campaign was running or not. Thus, the propensity of an audience to convert should be the baseline for the model and the attribution model should be measuring the lift the campaign traffic sources had on top of that.
- Where in the consideration funnel the audience is when they are arriving at a site via a marketing channel: Branded searches come later in the consideration funnel and thus should be evaluated differently from traffic sources, like generic searches that don’t require brand or product familiarity regardless of whether or not it is first touch, last touch, etc.
Audiences may have visited and converted at the same rate if they got to the site via any means and no campaign traffic source should get credit for this audience even if they clicked through from one of those traffic sources. There are only some categories of sites where people arrive without any prompting (Sites like the semphonic.com blog, ebay.com or the nytimes.com come to mind.). Many attribution models will benefit from having an additional consideration for propensity of audience to arrive at the site and convert anyway – though this may be more applicable for some online business models than others (Would anyone have visited the Nature’s Recipe for Moments website if there hadn’t been owned, earned and purchased links driving traffic there? I don’t think so.).
If You Build It, They Will Not Come.
In most cases, attribution models must be used to understand how marketing channels work together to drive qualified traffic to a site in the first place – particularly a new site. (If only I could ask everyone reading this post if they found it through a link in their Twitter feed, a link in their Facebook feed, or through a Google search? Or was it a combination of those?) And then analysed with regards to how traffic sources work together to boost conversions of those who wouldn’t have arrived and/or converted otherwise. (Segmenting out regularly returning visitors from the model, for instance.)
Getting A Full Picture.
Gary’s post also links to a summary of an eBay SEM study indicating that eBay is not benefiting from buying branded terms on Google. This was determined based on essentially a massive A/B testing program in paid search (not via algorithmic attribution modeling). A best practice is to use A/B testing to validate findings from data models i.e. using a data model to further understanding and generate test ideas. So I like to think that some other analysis triggered eBay to conduct this test and that A/B testing was not done in isolation.
While A/B testing may be seen as more “accurate” or more definitive than attribution modeling, it is still modeling and comes with its own assumptions. In the case of eBay, any decision to move away from buying branded terms would expose them to some risks that cannot easily be discovered through modeling. For instance, they will be left vulnerable to competitors who can bid on those terms and rank #1 (and divert traffic). Also, the study admittedly did not take into account other long term effects of not bidding on brand terms. Furthermore, not all campaigns (or even all campaign components) have direct response conversion as their objective.
Deriving Actions From Insights.
Even though I only clicked on the link to Gary’s post via Twitter, the LinkedIn ad reinforced its relevance to my day-to-day work. In practice, neither an attribution model nor an A/B test can be used by Gary to really know how LinkedIn’s listing influenced my actions. (The attribution model will have LinkedIn as a blind spot due to my clicking the Twitter link to visit his site. An A/B test is not practical because LinkedIn does not allow you to target status updates to representative control and test segments.) But he can (and should) still use attribution models and A/B testing to understand what other factors influence engagement on his blog in aggregate.
Attribution models and A/B tests are the best tools we have to describe, find patterns, and make predictions in how traffic sources work together to drive audience engagement and business outcomes. We need to use all our tools together, continually seeking to improve our models, to gain further understanding of online behaviors.
Our data models help us understand how audiences react to the mix of content and experiences online. We can combine eBay’s study and Gary’s insights to inform expectations we have around performance lifts from branded search vs generic search and Tweets vs LinkedIn article recommendations, and then use our models to piece it all together into actionable information.
What do you think? How did you get here?