The AI vibe shift, f. Von Neumann

VN’s 1948 take gives us (some) language for the changes happening

[click to view transcript]

The era of agentic AI is a good moment, I think, to become a little bit obsessed with John Von Neumann. Von Neumann was this incredibly prolific mathematician whose work led to game theory, quantum physics, artificial intelligence, and many other things.

Today I’m pulling from a lecture he gave in 1948, published in 1951 under the name The General and Logical Theory of Automata. This was in the early days of so-called “computing machines”, but in this lecture Von Neumann uses some framing that I actually think we need for today.

He’s deep diving in this lecture into “computing automata”, which in plain language we could describe as the system, rules, and processes that shape how a computing machine works. So he’s looking at how computing machines do what they do—what drives and governs them? And he defines two classes of these ‘automata’, two classes of that ‘how’ behind the machine.

The first type is what he calls digital. This is the type of computing machine we’re all using today. Its base language is digits: it represents things in digits, does operations with digits, and then outputs a translation of digits.

The second is what he calls analogy. This is where a computing machine does an operation via a physical object (for example taking in two currents) and then eventually maps that operation to a meaning (for example the sum of the two currents) and an output (for example, a current equal to that sum). What matters is here is 1. that analogy machines are indirect ways of producing a meaning (here, the sum), and 2. because noise is an unavoidable part of their functioning, they’re also less predictable and precise.

There is a long tradition of scientific understanding translating to lay person understanding of the world around us, and vice versa. We’re accustomed, today, to the digital paradigm of controlled and precise representation leading to fairly predictable and direct outputs. That’s how we see a lot of the world. But I think that paradigm is changing, and here’s why I think that.

As Von Neumann says in this lecture, human beings are a mix of digital and analogy. Our central nervous systems do things like transmit impulses between neurons, which can be seen as equivalent to binary digits, but also do complex analogy operations like shaping blood stream composition in response to stimulus. Which means our bodies and brains are both precise and imprecise, both direct and noisy. Yet for a long time, we’ve seen ourselves basically as digital. And it could have been easy to go on seeing ourselves this way.

But then generative AI happened. Which is also both analogy and digital, with all the noise and thus chaos that comes with it.

I think part of why this ongoing AI moment in time feels so big is not just the Gartner hype cycle, not just that we may lose our jobs, restructure our economy, face extinction, etc… but also that our fundamental paradigm for understanding ourselves and the world as digital automata is changing. We’re being confronted with the imprecision and noise of analogy automata, which I think we understand on a visceral level because we ourselves are that too. Which means suddenly the ordered world we believed in feels not so ordered, and we’re left trying to navigate what I can only call a paradigmatic vibe shift.

Von Neumann wanted to wrestle the nervous system to the ground with formal mathematics. If he had lived long enough, maybe he would have. But at the time, in response to a question about whether we can fully understand and map out the organizing principles that govern us, he responded, “There may be problems which cannot be formulated with our present logical techniques.”

Well, we haven’t yet figured out problems Von Neumann flagged in 1948: problems with AI’s management of internal errors, problems with AI’s management of noise and reliability. And btw, the vibe shift actually extends well past AI: scientists today are currently questioning the validity of space-time and trying to look into the noisy, imprecise parts of quantum physics to find a different paradigm. As always, I’ll drop specific sources below.

So look, I think how we as smart people understand the world right now has problems which, in Von Neumann’s words, cannot be formulated with our present logical techniques.

The paradigmatic vibe shift, this change in the understanding we have of the world, is happening, and it’s happening mathematically, scientifically, and in our day-to-day lives. We’re realizing that things we kind of can’t explain are emerging from our brains and the brains of people around us. My friends who are working on cutting edge AI are finding that things are emerging from their AI that they kind of can’t explain. And we have this emerging paradigm—from digital to analogy, in Von Neumann’s terms—that lets us see and feel that. But we don’t have quite the right language or logical techniques to talk about it. Not yet. Thanks for listening.

Sources: https://www.taylorfrancis.com/chapters/edit/10.4324/9781315130569-15/general-logical-theory-automata-john-von-neumann (PDF available via Google Scholar) and https://www.quantamagazine.org/the-unraveling-of-space-time-20240925/