Volume 176: Oops, AI Did it Again.

Oops, AI Did it Again.

tl;dr: Blitzscaling, AI, and it’s failures.

It’s a famous military truism that, in times of peace, we spend an inordinate amount of time, energy, and resources preparing to fight the last war we were in.

For a similar reason, if we are to understand today’s all-or-nothing grab for AI riches, we must first understand the previous battles that have been waged in Silicon Valley.

Specifically, we must look to Reed Hoffman and his vision of “blitzscaling," which he used to turn LinkedIn into the dominant professionally focused social platform while simultaneously being a terrible product.

The basic gist of blitzscaling is that you raise a lot of money very quickly, which crowds other capital out of the market, reducing your likely future competition. Then you spend this money very quickly on two things: 1. Developing a product, any product, no matter how bad, that you then 2. scale by spending heavily on marketing. The idea is that by having access to vastly more capital than your competition, you can scale product development and marketing faster than anyone else can catch up, leaving you as the sole winner, no matter how crappy the product.

The most interesting thing about blitzscaling isn’t just how cynical it is but how antithetical it is to the mythology of Silicon Valley. The Valley's preferred history is the literal opposite of the blitzcaling mentality: That the best ideas, most effectively realized, will win. The story is that VCs and founders represent a market that efficiently matches ideas with capital, enabling a smorgasbord of entrepreneurial bets, from which the market sorts the wheat from the chaff, and only those businesses forged amid the white-hot heat of competition ultimately win, marking them as highly resilient in the process. However, if you’re blitzscaling, it’s all about grabbing as much capital as possible, crowding out the proto-competition before it can emerge, and then using that capital as quickly as possible to achieve a scale that’s too great to be replicated. Avoiding the whole ‘white hot heat of competition’ phase in the process.

Now, why does this have anything to do with AI?

Well, for a couple of reasons, but mostly because the path of Generative AI commercialization looks a lot like blitzscaling right now. And, something we know about blitzscaled startups is that they make errors, lots and lots of them, which they then like to obfuscate by firing off round after round of PR propaganda chaff and flare as distractions. Sound familiar?

Remember, Reed Hoffman famously stated, “If you’re not embarrassed by your first product, you launched too late.” So, since GenAI looks to be heavily blitzscaled, and the Silicon Valley intelligentsia has already picked the winners, let’s examine ten of its rather significant failings to date (in no particular order).

1. Generative AI requires breathtaking amounts of energy.
   At current growth rates, it’s predicted that by 2030, US-based AI data centers will consume 323 terawatts of energy yearly, equivalent to seven New York Cities. That’s literally insane.

   Unless something radical happens with renewable energy, expect AI to be a major climate bogeyman for the foreseeable future. Why is it so inefficient? Mostly because OpenAI led the way by blitzscaling a brute force model that seems to be working best right now. Bigger models trained on more data seem to be smarter. But bigger models trained on more data also seem to be unbelievably energy-intensive.

2. Generative AI trained on shitposts produces…shitposts.
   Yeah, we’ve all been laughing at the delusional responses from the Google AI search summaries, from claiming it’s nutritious to eat a small pebble daily to telling you to put glue on your pizza and such. But what we should really be asking ourselves is whether AIs trained on the largest repositories of shitposting in history, like Reddit, could ever produce anything but shitposts. It’s a bit like the Silicon Valley equivalent of Wall Street bankers telling us that if we bundle a whole bunch of subprime mortgages, they’ll automagically go from being junk-rated to AAA-rated. And, of course, nothing bad happened there either, right?

   Being serious for a second, since the blitzscaling model of AI has an appetite for data that’s almost as rapacious as its appetite for energy, these models are now filled to the brim with the worst of humanity as well as its best. Since we know that, left to its own devices, Silicon Valley will do literally nothing to take responsibility for anything, expect that without robust regulation, the worst of humanity will pop up over and over again in the form of casual racism, hate, shitpost nonsense, and bias in the most incongruous of situations.

3. Generative AI is a security nightmare.
   Security researchers have recently highlighted a challenge: code-assistant AIs like Github Copilot tend to hallucinate things like code repositories that don’t actually exist. To cut a long story short, a hacker can easily exploit a hallucinated code repository by filling it with malware. And, before you know it, your whole business is toast.

4. Prompt hacking is a thing.
   Various reports have now been published on the potential for ‘prompt hacking,’ which is essentially the practice of persuading the LLM to disregard its built-in safeguards. The success rate varies from 30% in one example (sorry, I can no longer find the link) to 7-8% in this study. Either way, even the low end of 7% success represents a significant vulnerability that definitely won’t have corporate CIOs sleeping well at night.

5. Generative AI leaks like a sieve.
   Samsung famously fell victim to employees inadvertently leaking highly sensitive corporate information by sharing it with ChatGPT. While the Samsung response - disabling ChatGPT access and forbidding its use - made sense, it’s also a bit like fiddling while Rome burns. Of course, employees will use it if it makes their lives easier. Unfortunately, the primary fix being recommended is to make it impossible for the LLM to reveal sensitive information…the only problem is that, as mentioned above, prompt hackers have shown that it’s pretty easy to route around such safeguards.

6. Hallucinations are a feature, not a bug.
   The way current generative AI models work is that it’s a massive probabilistic engine. This means it isn’t really intelligent as such; it just has vast amounts of data to draw from, to which it applies advanced mathematics to make a probabilistic guess based on your prompt. Most of the time, this guessing is scarily accurate, but sometimes it goes way off base. (Especially if it’s been trained in shitposts, btw). To use an innocent example, when I ask ChatGPT about myself, it says I’ve worked for Landor, Interbrand, and Siegel+Gale. Three brand shops that I’ve never worked for but easily could have, so it’s pretty good as a guess while also being completely, factually, wrong.

   As a result, it’s important to understand that generative AI is a lot more like really advanced autocorrect on your phone than anything else. And, in the same way that autocorrect sometimes comes up with the most bizarre of suggestions, so does AI. What matters here is that you can’t just turn this feature on and off; it’s an inherent part of how the technology works, so we should be very wary indeed of anyone telling us the hallucination problem has been solved. Much more likely, a whole new category of drudge-worker gets created, where people are being hired solely to check and fix AI hallucinations.

7. Generative AI is abusive to humans.
   Thinking about drudge-work for a second, one of the tech industry's dirty little secrets is that much of what we believe has been automated has, in fact, been outsourced to low-paid workers in developing countries, commonly known by the name given them by Amazon, “Mechanical Turks.” In order to develop the massive Generative AI models now in play, thousands of such workers have been hired to label and clean data prior to ingestion by the AI. It’s repetitive, boring, often emotionally disturbing work that pays very poorly.

   While we may not think about it much while using tools such as ChatGPT, the low-paid Kenyan workers it relies upon certainly do, having recently written an open letter to President Biden highlighting that:

   “US Big Tech companies are systemically abusing and exploiting African workers. In Kenya, these US companies are undermining the local labor laws, the country’s justice system and violating international labor standards. Our working conditions amount to modern day slavery.”

8. Generative AI has a copyright problem.
   After years of publishers being systematically screwed over by tech firms, most notably Google, it should come as little or no surprise that they now view AI firms scraping the internet for training data as copyright theft of breathtaking scope and scale. And while different publishers are taking different stances, a slew of copyright infringement lawsuits could, foreseeably, place a major crimp on the sector.

   I’m certainly not going to predict the outcome of such suits, but if OpenAI, Microsoft, Google, etc., are held liable (and lawyers with a lot more expertise than I think there’s a strong chance they will), then the costs will be very significant indeed.

   As an aside, I feel very squeamish about AI systems trained on copyrighted material competing directly with the creators of said copyrighted material at a lower cost. That’s anticompetitive theft.

9. Generative AI works cannot be copyrighted.

   If you work in the creative services arena, especially in the area of branding, you’ll be intimately familiar with the legal protection that copyright provides for things like brand assets, logos, taglines, etc.

   One of the most interesting outcomes of an early generative AI court case is that AI-generated works cannot be copyright-protected. To be so, the work must have been authored by a human. Clearly, it’s going to be fairly easy to route around this by ‘augmenting’ humans with AI rather than replacing them, but don’t expect the tech firms to stand for this for long. Instead, look for them to push extremely hard to change the law to enable the copyright of AI works, and removal of humans completely.

   However, in the meantime, this will be one arena where an AI will require a human overseer for the foreseeable future.

10. Generative AI is a liability nightmare.
    Air Canada offers a cautionary tale. It used a generative AI chatbot to replace customer service operators - a common early-stage use case for the technology. During a customer service exchange, the chatbot hallucinated a corporate policy that had not previously existed and communicated this to a consumer. Afterward, when asked to live up to the promise expressly stated in this policy, Air Canada tried to weasel out by claiming it wasn’t responsible for chatbot misrepresentations.

    So, they were taken to court, where they promptly lost the case.

    This is a big deal. Most recent technology developments, such as social media and adtech, have benefitted from a complete lack of legislation designed to curb its worst excesses on society. With generative AI, tech just walked straight into some very mature arenas of law, like copyright and consumer protection, where the ‘move fast and break things’ mentality might just run straight into a wall of ‘massive legal liability.’

Now, don’t get me wrong. Generative AI is clearly a technology with significant upside potential. Clearly, corporations have to pursue their own experiments with it, either because they view it as something they can exploit to their own advantage or, more likely, because they have FOMO that if they don’t keep up, they’ll be terminally left behind by more aggressive competitors.

However, we also need to acknowledge that the vast majority of what we’re seeing right now is supply-driven blitzscaling rather than demand-driven value-creation. Just look at the recently announced “AI PC” that screengrabs your screen every few seconds so you can find whatever you might be looking for months later. Ummm, here’s the thing. While it will ‘ignore’ DRM-protected works, like, say, the latest Avengers movie, what it won’t do is ‘ignore’ things like images of passports, passwords it may have captured, social security numbers…in other words, this is yet another example of a blitzscaled tech product that brings very little utility adding vast, new, sources of vulnerability to our digital lives. Oh great, just what we all need. Sigh.

So, when I talk shit about AI, it isn’t that I’m a Luddite who believes that human endeavor is better. Actually, I think we’ll see people doing some super interesting and unexpected things with it. No, what I find concerning is the overt use of an error-strewn, move fast and break things, blitzscaling playbook by firms like OpenAI, which increasingly appears to be long on ambition and incredibly short on ethics.

I find this concerning, and you should, too, because blitzscaling tends to bring with it a range of societal costs. At the low harm end, with LinkedIn, there was never an opportunity for a better product to win, so a bad product won instead. While at the highly harmful end, Meta has proven to be effectively immune to anything resembling responsibility to anyone but shareholders, with Google not far behind.

Personally, I’d advocate for much stricter regulation and stiffer penalties earlier that can then slowly be loosened over time as we learn more, rather than what we do have, which is a screw-the-consequences mad rush for scale and billionaire status, where our modern feudal landlords force us to use AI services no matter how bad they are because there will be no other choice.