“Buyer Beware: Agency Reports Can Show Practically Any Number or Result”
This one could piss off a lot of people, and I’m completely okay with it.
It will mostly only upset the agencies that operate like scumbags.
Over my 20 year career, working with considerable organizations, I have often been the conduit between the brand and their marketing agency. Sometimes as a part of my full-time, in-house job. Sometimes as an independent third party, hired to audit and evaluate digital and marketing agencies – sometimes hire, sometimes fire, sometimes go to market with RFPs.
Nonetheless, the point of this video is to serve as a “Buyer Beware” to businesses working with agencies – specifically when it comes to performance reporting.
I can make almost any report show almost any outcome.
What do I mean by that?
I mean – it is possible to change the lens from which you view the data, to skew the numbers, to make bad look good, and average look great.
Here is an example of what I mean, that you can look out for in your own reporting.
A client of mine went to market with a new product.
They tasked their agency with a KPI of new visitors.
“Increase the percentage of new visitors to the website.”
If you go into a tool like Google Analytics, they will delineate new versus returning visitors to your website.
How is a returning visitor differentiated from a new visitor?
Naturally, they had to have been on the website at some point in the past. The crux of the issue is how far in the past?
That is usually determined by cookie settings. A cookie, or a pixel, is a tiny piece of code that lives on your site that speaks to your ad platforms and your analytics tools.
Cookie settings can expire after 24 hours, or 30 days, or can even be set to what’s called, “lifetime”, meaning they will never expire until the person on the other end signs out of their device entirely, drops it into the ocean, gets a new phone or laptop – you get the idea.
I have worked with brands that had their cookie set to lifetime, so the very first campaign they ever ran was still getting “credit” in their reports for purchases and conversions that happened 2 years later.
Not exactly insightful information.
Back to my client and their scummy agency.
When tasked with finding a new audience segment, and casting a wider net online to attract new visitors to the website for the new client product – the agency changed the cookie settings from 30 days, to 7 days.
This means that if someone visited the website and came back in 7 or more days, they were now flagged as a new visitor instead of a returning visitor (as they would have been previously under the 30-day expiration).
The client saw a 99% lift in new visitors. 99%!
They were a little curious so they brought me in.
I was way past curious. I was well into skeptical, so I looked under the hood.
Within minutes, I discovered the cookie setting had been changed.
The agency skewed the lens in which we view the data.
The data itself had barely changed at all.
With the settings at 30 days, the agency barely moved the needle at all in attracting new visitors to the website.
This happens more often than you think, sometimes out of malicious intent, sometimes out of ignorance.
The lens you view the data through is often referred to as your attribution model. How you attribute a conversion or a sale to any of the 100 inroads to your website via a dozen different social platforms, multiple search engines, any number of paid marketing campaigns you may be running, email, mobile app, etc.
Attribution models are broken down in much greater detail in an attribution module I have created and that I will link to from this video.
But it is important to know, the same data can produce wildly different results, by changing the lens you view it through.
Be aware and be diligent. Know the right questions to ask.
If you aren’t sure – reach out to me. I’m happy to help.
Cheers.
