AGI, elevators, and other thoughts I didn’t mean to publish

Note: I accidentally published this before it was ready! I decided to tweak the typos, remove a couple terrible fragments and otherwise leave it as-is.

Ideas have always been cheap; it's the execution that's the multiplier. In the AI world we're hurtling toward, which is driving down the cost of execution, I guess that means ideas are more devalued than ever?

Cheaper still than an idea is an observation. It's the Xerox of an idea run through a filter—artifacts and fidelity sometimes lost, but perhaps a bit of unique character introduced.

Everyone seems to have their own AI thoughts these days. Fresh off 4.5 years leading engineering efforts around a SaaS product through the birth of this LLM era we're all living in now, I thought I'd offer a scattershot collection of my own half-baked ones:

Abstractions on Abstractions

Computing is the history of abstractions. We've gone from punching holes in cards to perform calculations to talking to pocket devices in our mother tongue, asking what the weather will be—all in less than a century.

The advent of computer languages to create instructions led to software as we know it. From peeking and poking at individual memory addresses to writing scripts that imitate language to (now, with LLMs) literally using language as we're hard-wired to learn it from birth—conversing with machines.

What happens when the next "hello world" program is just: "ask the computer to write a hello world program"? Sure, behind the scenes it may literally write an entire operating system from scratch in some virtualized container, all in service of simply spitting out the words Hello World—but it doesn't matter.

I think there's an inevitable discomfort living through this kind of progress. You see what's lost. You see the depths of understanding falling short of your own. But there's progress on the other side of that that will outweigh anything lost. Feelings that end up bordering on nostalgia cannot slow this train.

And that's okay.

Probabilistic Elevators

At some point the interface to everything becomes a machine you talk to. In the same way microcontrollers have found their way into all manner of things. They're fast, cheap, and out of control. Wait until we have highly capable LLMs on chips. In that world, why create a unique electronic component for a button when you can manufacture a magical component that you kindly ask to behave like a button? Shrunken to microscopic sizes, how close does this start to resemble magical nanotechnologies capable of anything?

Imagine a grid of elevator buttons, floors 1–10. Each programmed with a simple prompt: "Send the elevator to floor X."

Now, if you're thinking about the probabilistic nature of elevators, you might be thinking: "That is insane. I would like a deterministic elevator, please."

But the truth is: if it's cheaper and easier, it will win.

And the other truth is: we're already living in a probabilistic world. This future is simply embracing it.

Also, Probabilistic Elevators would be a great band name. Please steal it.

AGI

The psychology of AGI, SGI, and trusting our robot overlords fascinates me. Science fiction has always been aspirational, but you have to remember that it's ultimately fiction. Real life always lands differently.

You achieve AGI when you're willing to trust the machine blindly. I think that's a product of comfort over time more than capability. At the end of the day, it's a marketing term for a highly capable version of this tool that hasn't quite landed yet. But I can squint and see it. We're talking about tools, not machines with will. We always have been.

It says more about us when we decide to feel comfortable making that leap than about the capabilities of the thing. I have a personal feeling that AGI is also only perceivable to the extent you've given the brain-in-a-jar tools to act on its scattered thoughts. I can't really conceive or understand an intelligence in the abstract, independent of tools that help me recognize that level of cognizance—writing, speaking, actions.

There’s a whole era of study dedicated to pretty much the opposite of my intuition that goes back to the 1950s called symbolic AI. I trust those people are much smarter than me—that my intuition is missing the point and that there’s something there—but I do suspect my ability to relate to this kind of intelligence is diminished absent relatable tools. A brain-in-a-jar absent the tool of language to talk to me is just a brain-in-a-jar, no matter what's going on in there. It feels more like philosophy than engineering something in the real world. At least today.

It’s like anthropomorphizing Excel or a bash script, to some extent. Or maybe asking Deep Blue or Watson to operate a forklift.

Anyway, debating AGI strikes me as a distraction. A fun distraction, but not a particularly useful one beyond feeding the marketing hype. Discussing the inevitable capabilities of these tools—and the agency we might feel comfortable granting them—is not.

AGI happens when you're ready.

Published on Saturday, June 7th 2025. Read this post in Markdown or plain-text.