How are Facebook Relevance Scores calculated?
Relevance scores are generated from several performance indicators, not too dissimilar to a Google quality score. Facebook still haven’t been too clear on everything that goes into coming up with the metric and there is limited visibility in ads manager, but they have shared some details:
- Ad quality: make sure the ad complies with Facebook’s ad policies1
- Post-click experience: is the landing page pleasant/relevant?
- In-feed experience: is the ad engaging or does it make people want to click through?
- Personal experience: do people feel it’s relevant to them?
Once you’ve served around 500 impressions, these factors, along with levels of positive and negative feedback, are taken into consideration and a score will be generated. Positive feedback is primarily based on levels of user engagement and negativity: when people are reporting or hiding your ad from their timeline (also known as X-outs). Then, on a scale of 1-10 it gives you an idea of how relevant your ads are. This score will be updated as more insight is gathered throughout the campaign lifetime.
A score of 1 indicates that your ad needs some work! Feedback has deemed it as substantially less relevant than your competitors. A score around 5 means your ads are similar in relevance to those of your competitors. Obviously we are all aiming for 10, to know that our ads are getting more positive feedback than others being served to the same audience, and as a result are significantly more relevant.
How does my ad’s relevance impact its performance?
To look at how it impacts performance, we should first look at how Facebook comes up with your total bid:
Total Bid = Bid x Estimated Action Rate + User Value
Ultimately, Facebook wants to make the user-experience as good as possible across their platforms. If the user experience is poor, then people will stop wanting to use Facebook (and it will hurt Facebook’s business model. Facebook’s system “is designed to show the right content to the right people” so the higher your relevance, the better! If you don’t show relevant content, you’ll end up paying more to show that content.
We’ve seen this day to day; ads with lower relevance scores are more expensive to serve. Take the example below with data from one of our clients:
As you can see, there is a moderate negative relationship between relevance scores for each ad and CPM: as relevance scores go up, CPM goes down. With creative changes made by us and the client from September to October 2018, changing our ad formats from carousel to collection ads, average relevance scores increased by 9%. This helped our client in the auction and consequently CPMs fell by 21% in the same period.
Carousel vs Collection ads. Which do you think has a better user experience?
Because it helps with delivery, you’ll likely also see your other performance metrics improve. AdEspresso analysed over 100,000 of their ads and found that Cost per Click (CPC) was strongly related to relevance score2. Their ads with relevance scores of 10 had CTRs 12.6 times higher and CPCs almost 97% lower ($0.04 vs $0.95. When analysing the relationship between CPC and relevance score in our client example, we found that the correlation was even stronger at -0.7, with at least 56% of the variation in CPC coming from relevance score (R2 = 0.57)
That’s cool and all, but what can I do with it?
Aside from ad delivery, relevance score it also a useful tool in optimisation. A decreasing relevance score in a live campaign might be an indicator of creative fatigue or that you’re in need of an audience refresh. In the early stages, relevance score can also be used as an indicator of expected creative performance – helpful when considering your testing strategy.
Well, how can I improve my ad relevance score?
As mentioned, you want to make the user experience 10/10. First, make sure that your ad is relevant to the audience. My Nan, for example, loves music. However, there’s not much of a chance that she’ll be clicking on an ad for a Metallica concert. Adjust your targeting so that you hit the right people and your relevance score will shoot up. If that doesn’t work, there are a couple of recommendations3:
- Tell a visual story: grab people’s attention with engaging videos and stills.
- Don’t use controversial or provocative imagery (unless completely relevant).
- Tweak and test your creative slightly to fit the audience – you may see that younger demographics respond differently than older demographics. What’s causing that?
- Make your creative relevant to real-life moments (See our case study on syncing our ads with the weather or check out Sasha’s blog on social sync).
- Refresh your creative: after a while people may get bored of seeing your ad.
While understanding relevance scores has real benefits for advertisers, it’s important to keep this metric in perspective. Relevance scores should not be used as the primary indicator of an ad’s performance - relevance score is an output, not an input. As has long been the case on Facebook, the most crucial factor for success is bidding based on the business goal you hope to meet with an ad.
In other words, Facebook aggregate the factors and convert them into the 1-10 scale to give you a general idea of your ad's relative relevance. When your ad enters an auction, the 1-10 number is not part of the "total value" calculation that determines which ad gets shown. Because of this (and the fact that the number is relative), we recommend not focusing too much on raising your relevance score. Raising it is not directly tied to improved performance. Instead, we recommend using it to get a sense of your ad's relevance, and then focusing on improving your targeting and creative.