SBN

Grappling with the Unpredictable Second-Order Effects of LLM

Yogi Berra was (among other things) an incredible quote machine, and he’s often credited with the following gem: “It’s tough to make predictions, especially about the future.” Yes indeed. Seeing how the world will unfold is incredibly difficult because of how convoluted second-order effects can become. So, with the legendary Mr. Berra providing me with some air cover in the form of a caveat emptor, I would like to talk about why it’s so darn hard to predict the ways in which Large Language Models (LLMs), like #ChatGPT , and #GenerativeAI in general will impact us.

The challenge of prediction, of course, is that it is hard to see all the consequences of an event. While it’s often easy (or, at least, easier) to see the immediate impact, looking further downstream to see what are called second-order effects can be much more difficult. When you look around for examples of this, they’re pretty plentiful. An intervention, made with the best intentions, can turn out to be disastrous.

Unintended Consequences

Poor Australia. Back in the 30’s, the Greyback cane beetle was laying waste to sugar cane crops, and the Aussies needed an answer. Fortunately, it was a case of “nature to the rescue,” or so people thought. Cane beetles can be eaten by Cane Toads. Let’s introduce these Cane Toads into the ecosystem , as they’ve been successfully used in other places (most notably Hawaii) as a way to control a pest. Sounds simple, right? Not so much.

Fast forward to today, and Cane Toads have become an invasive nuisance, with their range expanding southward across the continent. Why? Well, you can say that the whole idea was poorly thought out. In particular, Cane Toads prefer wet conditions, and the sugar fields in Australia are drier than they would like. They also multiply like crazy. Ah, the joy of hindsight. Alternately, you can point out the fact that biological systems aren’t simple . They’re complex, and the downstream consequences of a change aren’t always obvious.

It’s not just biology. The poster child for this, of course, is the Industrial Revolution.

From Farm to Factory

So, we’ve all read about the Industrial revolution, and it’s near and dear to my heart, as I grew up a bike ride away from Ironbridge Gorge, hailed by many as ground zero for where it all began. Of course, some would (not incorrectly) argue that the real disruptors were the changes to the textile industry, which rapidly evolved from manual to (mostly) automated. Couple this with the adoption of the factory system and the use of steam power to drive it, and modern society as we know it began to take shape.

All this sounds great, but we’re thinking first-order. Stop a moment and think through some of the second-order effects you might expect. Okay, don’t cheat… ready? Okay, now read ahead!

Despite all the good associated with better efficiencies, negative second-order effects from the Industrial Revolution are still in play today. Among some of them are the following:

  • The rapid growth of large cities and associated poor living conditions. As workers moved from an agrarian society to an industrial one, there was a tendency for these displaced workers to be clustered together in terrible living conditions. The cities didn’t grow in a planned or thought out fashion, and this led to health issues and pollution. That, of course, led to illnesses.
  • Child labor. Many saw the use of steam as the “muscles” of factory work to mean that children could be the perfect workers for the new factory. After all, they could now quickly be put to work for long hours with little training and without requiring significant skills or strength (unlike a farm, where work was limited by physical strength). This view directly impacted the path of education in the Western world, and still affects public education in the United States.
  • The creation of internal hierarchies, as new management positions were required to manage the influx of workers. This fundamentally led to some level of stratification of wealth which still exists to this day.

I could go on, but the point I want to make is the set of connections that go from “Hey, I have a better way to create cloth” to the outbreaks of diseases in the cities is not so obvious. As an aside, if you want a really heady discussion about nutrition during this time (something I included in my first draft and then deleted after more research) I refer you to this gem . It is a fascinating article that questions some of my previously held assumptions and shows the complexity, even with the benefit of hindsight, of figuring out cause and effect.

Back to AIs…

Turning our attention to the impact of generative AIs in the world, I think the first order effects are easy to see. However, many of the second order effects are truly opaque to me. There are so many ways to go here, let’s just pick one example (the simplest one I could think of… that’s a worry) and see where that leads. Let’s talk about the democratization of content creation.

On the surface, this sounds great, right? First-order effect, any office worker or computer user will be able to create meaningful and intelligent-sounding content about any particular topic using a tool like ChatGPT. Research work that would have taken hours to support a blog post can now be short-circuited, and a day compressed, quite literally, to a handful of minutes. The result: cheap, easy, and fast content creation for all. Who wouldn’t want that?

But There’s A Catch, Of Course

Well, before we do our happy dance, we now know it’s going to be important to ask what second-order effects we might see with this. The first “predictable surprise” that comes to mind is that we’ll greatly devalue content. If all content is trivial to generate, why work long hours on truly original work? How much is it worth? That shift will have consequences in the market and on individual contributors. Making it personal, I learned a lot rolling around in research for this post. I’ve made that knowledge part of me, and it will impact my thought processes and decisions from today onward. Had I just used an AI to create this article, not so much. It would have saved me hours, but I wouldn’t have gained the level of comprehension I have now. My one data point here is not much, but add that up through my learning arc, and the intellectual outcome would be devastating to me.

Another “predictable surprise” is that it’s about to be trivial to create misinformation at scale. Want to run a bot army to flood the world (or a social network) with your point of view? Easy. The last few years have been bad enough with the Russian Troll Farms but you might imagine a second-order effect where things are much worse. When much of the population gets its news from social media, the implications could be wide ranging. (I prefer “catastrophic” here, but I don’t want to be accused of hyperbole–to remind myself of the opening of this post, predictions are hard!) In the event of this all happening, what then? Do we see a privacy/anti-AI backlash? Do we see society tear itself apart over an AI-written fever dream designed to incubate conspiracies? The downstream consequences are hard to predict.

The Cynefin Framework as a Tool

Navigating these murky waters is really hard, and so a way to structure your thoughts and approach the problem may help. I’ve used the Cynefin framework before, and found it very useful.

The Cynefin Framework is based on viewing the world in 5 basic “states”: Clear, Complicated, Complex, Chaotic, and Confused. Figure 1 provides a useful diagram of the concept.

Figure 1: The Cynefin Framework as redrawn by [email protected] to incorporate recent framework changes. [1]

Most disruptions push us to the Chaotic state where cause and effect aren’t clear. As the world becomes better managed and adapts, we cycle through states in a clockwise manner until things move to the Clear domain again.

We also can pour on constraints to move us from Chaotic to Clear more quickly. Imagine a law banning use of AI (not something I see, but go with it). In such a world, assuming the law was broadly enforceable, we have reduced degrees of freedom in order to simplify the problem.

As a business manager, you need to be aware of where a particular problem sits in this framework, as there are well-known recipes for how to operate within each region.  The big takeaway: the way you view the world and manage within it needs to vary depending on the domain in which you find yourself. In the framework creators’ words,

Using the Cynefin framework can help executives sense which context they are in so that they can not only make better decisions but also avoid the problems that arise when their preferred management style causes them to make mistakes … Leaders who understand that the world is often irrational and unpredictable will find the Cynefin framework particularly useful.

I am not going to do this work justice, but fortunately, the creators of the framework have provided a thorough explanation online for free at HBR . I recommend taking the time to read through their full explanation and then come back here for some closing thoughts.

Closing Thoughts

“The world is complicated” is an understatement, especially right now. We’re about to see the widespread adoption of a technology that could be a game changer. From a first-order perspective, if this turns out not to be a “disruption” event, at a minimum we will experience significant change in several important aspects of modern living. As we’ve seen, the second-order effects could be even more impactful, as the entire system will flex and change in response.

As leaders, we need to actively engage with those changes, not ignore them. The genie is very much out of the bottle, and no amount of pushing will stuff it back in. Instead, we need to intelligently change our management approach to fit the conditions for which we are managing. As the Cynefin Framework states, in a chaotic context, “a leader’s immediate job is not to discover patterns but to staunch the bleeding.” As such, we must focus on responding and driving order so we may build a stable point from which to view the problem. In addition, our management style needs to change, becoming less top-down and reactive and more OODA-like, until, finally, we can understand, predict, and tame the system.

No matter how far we try to see ahead, “for now, we see through a glass darkly” and truly cannot predict the complex second-order effects of technology like LLM and generative AI. Our management styles need to reflect that.

 

This article originally appeared in a slightly different form on the author’s LinkedIn

References

1. By [email protected] – Own work – a re-drawing of the prior artwork found here (https://commons.wikimedia.org/wiki/File:Cynefin_as_of_1st_June_2014.png) that incorporates more recent changes, such as renaming “Simple” to “Clear”, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=123271932

The post Grappling with the Unpredictable Second-Order Effects of LLM appeared first on Praetorian.

*** This is a Security Bloggers Network syndicated blog from Blog - Praetorian authored by emmaline. Read the original post at: https://www.praetorian.com/blog/second-order-effects-of-llm/