De-mystifying Quality Score
Quality Score is one of the most important parts of the AdWords auction, and a key to paid search success. The trouble is that as a metric it comes with a lot of disinformation. I’m going to take a look at how we can better understand what it is, how it affects our campaigns, and what we can just ignore.
Quality Score is a multiplier in the auction.
This is the simple bit. Google run the auction based on a metric they call Ad Rank. In fact when Google talk about the auction nowadays they don’t even mention quality score, they discuss ad rank and the factors that are involved. Their chief economist Hal Varian recently released a very good video explaining the concept of ad rank and the consequences of its presence in the auction.
Short version: your bid is multiplied by your quality score to determine your ad rank. The ad rank of every ad determines the position on the page. If your ad is clicked, you pay just enough for your ad rank to beat that of the ad in the position below yours.
Using Quality Score maximises Google’s revenue.
Google’s search revenues derive from two things: the revenue per search and the number of searches performed.
The AdWords team don’t have too much control over the latter. Obviously in the long run as Google gets better (or more prevalent, e.g. smartphone growth) the number of searches will increase, but in the short or medium term they need to maximise their revenue per search.
You can break revenue per search into two factors:
- Expected revenue from an ad click
- Likelihood of getting an ad click
We remember from our GCSE maths that if you have two linear functions, one rising and one falling, the highest product will arise when they’re both in the middle of their range (99 x 1 < 50 x 50, even though they both sum to 100): i.e. Google doesn’t maximise their revenue per search by prioritising cost per click over probability of getting a click (predicted clickthrough rate), or by prioritising predicted clickthrough rate over cost per click. They need both.
Predicted clickthrough rate is super important.
Google don’t know which ad a user will click on, but they do know that it’s skewed towards the ads at the top of the page. The average clickthrough rate uplift for a non-brand keyword when it moves from the right hand side of the results to the banner positions is 14x. It is very true ads in better positions on the page attract the most clicks. So Google’s revenue per search is greater when ads that make them more money (that combination of cost per click and predicted clickthrough rate is high) get into those top positions.
So what Google really needs is to know the predicted clickthrough rate for each ad. Their best signal for this is the historical clickthrough rate for that ad.
This is complex to calculate. The AdWords interface makes it seem simple: navigate to an ad, choose a time period, look at the CTR.
Google need more granularity than that though. They need to know which search queries those previous ad impressions came from. They need to know which positions they were in. So Google end up looking at the historical clickthrough rate for an ad when it showed in the same position on the same search query. Which is a much smaller data set.
Quality score adds other factors to historical clickthrough rate.
Historical clickthrough rate is the best predictor Google have for future clickthrough rate, but there’s no need for them to ignore the other data at their disposal. This is where Quality Score (QS) comes into play.
The below factors are all extra pieces of information that Google can use to help assess your predicted clickthrough rate, in addition to your historical clickthrough rate:
Search query presence in ad text. This is a simple one. Although on an individual search it might not be a predictor for which advertiser has the better ad and experience, across the millions of advertisers on Google it can be. By using this as an extra piece of information they can easily algorithmically make an educated guess about whether your ad will be good or bad.
Search query presence on the landing page. Similarly to above, it’s a sign Google can use to help them give extra weight to better campaigns.
Page loading speed. A poor loading speed will mean a poor experience, and over time a lower user satisfaction will reduce ad clicks. So this can be used by Google as a penalty to get you to improve your page load speed.
- Mobile friendliness. If Google’s results on mobiles are littered with ads that lead to mobile unfriendly pages then Google’s mobile search engine will be used less. So if a user is on a mobile device, then how mobile optimised your landing page is will make a good factor for Google to prioritise your ad. This includes things like fixed width pages, flash presence, or images that don’t scale.
Search queries are not keywords. That matters.
I’ve been quite careful so far not to use the term “keyword”.
You choose and bid on keywords in your account, but keywords do not map exactly to the queries that users type into Google. They can’t, since Google estimate that 20% of all searches are all-time unique, i.e. nearly a billion searches per day have never been searched for on Google before, in their trillions of previous searches. That’s a big deal, and it implies that we cannot predict every search query that will be useful to us.
By providing different match types, Google help us to alleviate that problem.
The trouble is that we know QS is being applied per search query and ad combination, which we can’t see within AdWords. Instead we see QS per keyword. A keyword covers several search queries and several ads, so it is at best a broad brush, at worst it can be misleading.
QS has to be reassessed every single search.
Since the historical performance of an ad on each search changes every time that search is performed, QS has to update on every single search.
Since QS is applied at a much more granular level than it can be displayed, and is re-calculated so often, we end up with a situation where it’s impossible for Google to give us much insight into QS as it’s being applied in the auction. Instead what we get is almost a summary.
QS is reported by AdWords as a mark out of ten. A lot of talk has gone into what these marks mean, but what’s clear is that these do not represent QS directly. Instead these tell you how many criteria for a good QS you have achieved.
For example, if a clickthrough rate above 10% in the banner gets you a QS of 7, then an ad containing the keyword might bump that up to 8. It’s not meaningful in the sense of what’s happening in the auction, and a change from 5 to 10 doesn’t imply a doubling of ad rank. Instead it just tells you as an AdWords user that you have achieved more of the factors Google want, at a broad level.
We can monitor trends of these metrics, but we can’t do much with them.
By default Google doesn’t track QS over time. If I look at a keyword’s metrics for six months ago, the QS value will show the current score. That makes it very tough to see how well you’re doing on that front.
By utilising third party or proprietary technologies (including our own Sonar campaign analysis and reporting tool) we can download that data from Google on a daily basis and track it over time.
There is almost no circumstance in which we would want to use that directly, or compare keyword against keyword. We don’t trust the scores enough to tell us anything meaningful about the keyword or the search queries, and performance metrics like Return on Investment or Cost per Acquisition are usually more important.
What we can do is track the QS figures over time. By weighting the average for an ad group or campaign by the number of searches (impressions divided by impression share) we can look at an overall figure for a segment of our account. By monitoring those on a time series basis we can judge whether actions that we’ve taken specifically to improve QS have actually done so.
Unfortunately it’s rarely worthwhile. Since the primary factor in ad rank is our historical clickthrough rate, the main way for us to impact QS is to write ads that attract a better clickthrough rate themselves. This comes with two complications:
Not all ads should be optimised for clickthrough rate. We could easily write ads that attract a lot of clicks by making crazy promises. “Free One Hour Delivery!” or “90% Off IPads!” spring to mind. But we wouldn’t sell anything. An ad has to do the job of filtering out poor prospects as well as attracting good ones. So optimising an ad for clickthrough rate isn’t always ideal.
- When we do optimise ads for clickthrough rate, the effect this has on traffic volumes is direct. Forget improving ad rank, we get more clicks even if our position doesn’t change, by virtue of having a better ad. That’s a truism that makes it hard to judge the effect of the QS changes.
QS is super important, and super unhelpful.
The summary of this is that we never learn much from looking at quality scores in AdWords. For something that is so impactful on everything we do, we can’t really do much about it or act on what it tells us.
Luckily, that doesn’t matter. Since the factors that go into determining quality score are factors that broadly suggest good quality ads, we have a double-win.
By making our ads better suited to our campaign (more compelling, tightly targeted to search queries, relevant) we make our campaign better at the same time as improving QS. The actions Google encourages us to take to help improve their profit generally make our campaigns perform better regardless of the QS bonus. When we do get that bonus it’s hard to track, but very welcome. Moving up the page can have an enormous impact on our traffic volumes and is something we always want.
If you take away one piece of information from this article, I’d like you to remember this: we obsess over quality score, but we probably won’t talk about it much. It’s too important, but it takes care of itself.