Volume 178: Data Is An Agenda.

Data Is An Agenda.

tl;dr: Nothing is neutral.

Whenever teenagers ask me for advice on what to study at college, I always respond that irrespective of their major; they should take at least some statistics classes because data literacy is a critical skill in the contemporary world.

The reason is simple. The further removed you are from being able to analyze and understand quantitative data yourself, even if you’re a designer, writer, or otherwise creative person, the more likely someone else will misuse such data to influence you. Sometimes, this will be outright fraudulent, but mostly, it’ll be massaged just enough to fit an agenda.

Anyone remember the peddlers of purpose? Yeah, just four years ago, they claimed things like “consumers are 4-6 times more likely to purchase, protect, and champion purpose-driven companies.” Clearly, either consumer behavior changed massively over the past four years (unlikely), or the data was bogus and driven by an agenda (bingo), which in this case was to sell purpose-driven brand consulting, PR, and purpose-vertising.

Then there was the metaverse. Anyone remember that? Well, just this year, McKinsey used its fearsome powers of data analytics to predict that $4-$5trn worth of value will be created in the metaverse by 2030. Do any of us still think that? Nope, good. Why make such a prediction? Why McKinsey’s tech and consulting $$ agenda, of course.

Today’s equivalent of purpose or the metaverse is generative AI. Clearly, it’s a technological breakthrough that will transform many aspects of the economy, but it’s also true that we simply don’t yet know enough about what it will be good at to make definitive calls, even though there’s a lot of data purporting to tell us we can.

For example, my head was spinning the other week when I read on the one hand that McDonald’s is canceling its entire investment in generative AI-based drive-through ordering due to it being completely unfit for purpose while, at the exact same time, a synthetic data startup launched, claiming to offer high efficacy customer insights for B2B corporations, especially solving the problem of extremely hard to sample segments like CFOs.

So, we’re left with something of a conundrum.

On the one hand, McDonald’s, a massive global corporation with infinite resources, cannot make generative AI work for something as basic as closed-loop burger ordering. Yet, we’re expected to believe that a startup with extremely limited resources can provide highly accurate insights into the complex open system of extremely difficult-to-sample audiences like CFOs.

Enter data. And how to embed it with an agenda.

Evidenza claims that its synthetic data is either 89% or 95% as accurate as traditional market research, depending on which article you read. I have no reason to doubt this. However, while this may, on the surface, appear to be pretty amazing, it also acts as an excellent example of how you can use data to hide a multitude of sins.

To put this in perspective, I jokingly made the comparison on LinkedIn that cats have 90% of the same DNA as humans, but I wouldn’t ask a cat what a CFO thinks.

In other words, it's entirely possible for synthetic data to be 90+% as accurate as a research study (which in turn will have its own error percentage, so goodness knows how they landed on their figures) while also being useless depending on where the differences fall and what the nature of these errors are. And since we’re highly unlikely to run concurrent research alongside the synthetic data to find out where exactly the differences reside, we’re never going to have that visceral “I ordered 6, but I got 200 chicken nuggets” moment that tells us something has gone catastrophically wrong. And let’s not forget that what gets euphemistically labeled “hallucination” (catastrophically wrong) stems from the basic architecture of a generative AI system, which is basically an advanced form of probabilistic guessing, which means you simply can’t get rid of them. Not yet, perhaps not ever.

In other words, if we rely on black box systems, thinking the error percentage looks pretty good while having no idea what that error percentage covers or how the system even works, then we owe it to ourselves to understand just how big the risk potentially is. (It's not that I’m saying not to do it; there will likely be many times when the cost/benefit of synthetic research makes a whole lot of sense. It’s just really important to understand not just the margin for error but also the nature and scale of the risk you may be taking on. A topic Evidenza appears to be singularly silent on).

As an aside, while I know it looks like I’m picking on Evidenza, it’s more that it’s a perfect and timely example to work from. In terms of the business, I think the Evidenza pitch is super smart. The problem of high-value, impossible-to-sample audiences in B2B is absolutely real, so it’s tackling a potentially valuable niche. The true genius here, though, might well be that because these audiences are almost impossible to sample, Evidenza is unlikely to ever face its “bacon on an ice-cream cone” moment because we lack any way to spot when it’s making catastrophic mistakes. Well, not until it’s too late, anyway.

So, if data has an agenda, this neatly brings me to those who will happily use and misuse data to influence you.

Mark Ritson is an academic, strident talking head, consultant, creator of marketing courses, and now, paid pitch-man for Evidenza.

To understand the nature of agendas and influence, it’s worth following his own particular AI journey. In his weekly Marketing Week column a year ago, Mr. Ritson said the following:

“ChatGPT is a toy. A fucking toy. Put it down…While we play with meaningless toys and then waste even more time proclaiming their importance, the basic foundations of marketing are nowhere to be seen.”

Such commentary from him was pretty common until recently, and then suddenly, on June 13th of this year, he wrote a breathless advertorial for Evidenza titled, “Synthetic data is as good as real – next comes synthetic strategy.” However, within it, there’s a single blink-and-you-miss-it sentence that highlights exactly why he feels the need to lick these particular founders from head to toe while being famously curmudgeonly toward everyone else:

I’m a very minor shareholder in Evidenza so I need to declare my interest before I go on.

And that was it. Personally, I’m a bit shocked that Marketing Week has such lax editorial standards that it would allow a column like his to be used to so obviously shill for a company he has a financial stake in, but then again, when did any marketing trade rag demonstrate anything remotely approaching an ethic?

Anyway, what’s not in question is Mr. Ritson’s work ethic. He’s so into being the pitchman for Evidenza that he even goes into the comments of LinkedIn posts by marketers highlighting their own experience with synthetic data to scatter his trademark snide commentary like dog turds on a pristine lawn. Oh, good gosh, these marketing influencers are just the best. They make LinkedIn such a great place to hang out. Sigh.

I pick on Mr. Ritson, not for the sake of it, but because he’s a public figure who has very successfully built his own brand as a marketing thought leader, and his shift from skeptic to breathless believer is a wonderful case study in what happens as corporations wield their $$ to bring influencer credibility to a space where there’s doubt. (For the record, I think the Evidenza founders were exceptionally smart in getting Mr. Ritson on board, and I’m sure he had many other offers from competing AI firms. But that’s a different topic).

No, the reason for highlighting this example is that when it comes to data, you have to be very careful to understand not just what the data itself is saying but also why it’s being said, who is saying it, and what their financial incentives for saying it are. Because it is entirely possible, if not likely at times, that your economic incentives and theirs will not be well aligned.

And look. Ultimately, everyone has an agenda, and increasingly, the way data and statistics are wielded in the public square means that data has an agenda, too.

Forewarned is fore-armed. Go take some statistics classes.