A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook
This is a very dense, 30+ page study from professors at Northwestern. They were given access to campaign data from a number of FB campaigns. I appreciate that very few of the subscribers of this newsletter have the time to read through this entire thing so I'll share my big takeaways here.
In the study, they compared a true conversion lift methodology (using a real test and control group) to other common approaches to measurement, including exposure based models. They found that a number of these other approaches grossly over-stated the efficacy of the campaigns measured.
They also have a really interesting finding on to the use of "PSAs" (like Red Cross ads) as control ads to measure against test campaigns. Their big problem with this approach is that the advertising platforms don't always uniformly deliver the PSAs and test ads. This led to instances where the PSA campaigns' conversion rates led to the incorrect conclusion that the actual campaigns lowered conversion rates.NORTHWESTERN.EDU