A common misconception in the industry is that incrementality only applies to verticals where customers visit the stores frequently (think QSR, gas stations, home DIY improvement, etc.). However, this could not be further from the truth.
Incrementality Is NOT Simply Pre/Post Exposure Testing
Let’s start with understanding what incrementality is NOT.
Many people mistakenly believe that measuring the effect of ad exposure on visitation can be calculated accurately by applying a pre/post-test. In other words, by looking at the number of visits a given person was making before the exposure and comparing that to the number of visits that same person made after exposure, you assume any difference in visits can be attributed to the advertising campaign.
Pre/post testing has a number of methodological issues, but for a moment let’s only consider the utility of a pre/post-test in determining the effectiveness of ad exposure in verticals with low customer visit frequency (like auto-dealerships, tourist destinations, etc). Since most people do not go on holidays multiple times a month, or to the dealership enough times in a month, comparing visiting behavior before and after exposure might create a situation where we have nothing to compare behavior with! An approach that used pre/post testing would not be useful in these cases.
Cuebiq’s Incrementality Methodology Goes Deeper
In a nutshell, Cuebiq’s incrementality methodology works around this problem as follows: for every device in the exposed group we look at all the brands they visited in the 12 weeks prior to the campaign, and build a visit profile—for which is simply all the brands a person likes to visit. So, for every device that was exposed to the campaign, we have a series of brands that describe that person, creating a “persona”. Personas can vary widely from person to person. For example, think about the difference between people who only visit fast-food restaurants and bars (Persona A) and people who only visit parks and gyms (Persona B).
We then repeat this process to create personas for every device in both the control and exposed groups. From there, we can identify similar personas (Personas A for example) that exist in both groups. That is, we end up with devices in both the control and exposed group that are very similar to each other in terms of what brands they like (and by extension, their interests), except that one set of personas is exposed, and the other ones are not. The only difference between them is the advertising exposure.
See where this is going? By using machine learning we can predict visits to the POI of interest for Personas A in the exposed group, then Personas A in the control group, and link their visit history to their current visiting behavior. We then use sophisticated machine learning to create a mirror version of exposed Persona A that behaves as if they had never been exposed. This goes beyond a mere look-alike logic and even standard control-group methodologies: you’re comparing exposed devices to themselves, unexposed!
The difference in visiting behavior between Persona A that was exposed and their mirror version had they not been exposed gives you the increase in predicted visits that happened because of the exposure – that is, the incremental visits!
Putting it All Together For Low-Frequency Verticals
Now let’s go back to our use case of verticals with very low visit frequency, like auto-dealerships and hotels. Remember: the problem with these verticals is that very few customers repeatedly visit an auto-dealership or a hotel throughout the duration of a campaign.
With our methodology, repeated visit frequency by the same device doesn’t matter! Why? Remember that these personas are composed of many different devices. As long as there is at least ONE customer within that persona that visited a location at least ONCE (in both control and exposed groups), we can establish an average baseline of visitation for both control and exposed groups. Then, we use the same mirror technique to understand the average baseline of visits for the exposed group had they not been exposed.
See how this has nothing to do with how many times a specific person visited repeatedly? It’s all about having at least one device in each persona with a visit so we can establish baselines of visitation for both exposed and control groups, measure the difference, and determine incremental visits.
An Auto-Dealership Example
Let’s look at an example that simplifies our methodology for the purpose of illustration: an auto-dealership is running a campaign to promote people to visit their brick-and-mortar stores. You don’t usually visit these stores. However, all we need for our methodology to work is for you to visit sometime during the campaign.
- First, we create a Persona for you: “Persona A”
- Then, we find devices that look like you (Persona A) in our exposed group
- We also find devices that look like you (Persona A) in our control group
Imagine there are 100 devices that look like you in the exposed group, and 100 devices that look like you in the control group. Even though you only visited once during the campaign, we are able to see that, for example:
- Your look-alikes in the exposed group visited 5 times (so 5/100 or .05 visits)
- Your look-alikes in the control group visited only 2 times (so 2/100 or .02 visits).
We then apply machine learning to create the mirror version of the exposed group (like we mentioned above), to calculate the incremental effect of ad exposure on devices in Persona A. In this case, the calculation would be something similar to .05 – .02 = .03 incremental visits – or, we can attribute 0.3 incremental visits to the auto-dealership to the ad exposed to devices in Persona A.
If we add all the incremental visits for all personas in the exposed groups across the whole campaign, we get the total number of incremental visits for all of the exposed groups!
So, as long as there are people in both the control and exposed groups that look like you (in terms of which brands you like to visit), we are perfectly able to determine incrementality by roughly following the logic we described above, even for verticals with very low visit frequency such as hotels, auto-dealerships, etc.
Pretty powerful isn’t it?
