The Crooked Timber of AI

The philosophical perils of conflating discoveries and inventions, and how to overcome them

A famous line from Immanuel Kant has inspired a good deal of interesting philosophy, particularly from Isaiah Berlin:

“Out of the crooked timber of humanity, no straight thing was ever made.”

The Crooked Timber of Humanity is the title of an essential collection of Berlin’s writings (thanks to Mick Costigan for gifting me my copy several years ago) and Crooked Timber is also the name of a fine group blog featuring Henry Farrell, among others.

In this essay, continuing the broad line of argument I introduced in Inventing New Nature, I want to argue that like us, the species on whose history of thought it has been trained, AI too is crooked timber, of which no straight thing will ever be made.

More so, in fact, because AI is the concentrated statistical distillate of the history of human thought. The crookedness is more pronounced, not less. To invent New Nature, we must first discover, and grapple with, the essential crookedness of our primary medium, as wood-workers must with wood.

Isaiah Berlin is the philosopher we must look to, for cogent thinking on how to navigate the civilizational challenges posed by AI. Not just at the philosophical level, but increasingly at the technological level, as we progress steadily from the realm of low-level device and code engineering to human-scale protocol engineering at all levels from MCP servers to international treaties.

In this essay I want to set the stage for both, and offer a taste of what we might find as we embark on our voyages of discovery into the growing pile of AI’s crooked timber.

Why do we need this approach?

Because people in general, but especially leaders at frontier AI labs, repeatedly seem to commit a distinctive philosophical error. They persist in interpreting AI through architectonic, almost Cartesian concepts – AGI, alignment, agency, intelligence – that are treated as ontological primitives rather than provisional summaries of observed behaviors of AI models and their varied embodiments. As a result, each curious new phenomenon is assimilated, usually in a distorted and reductive way, into an existing conceptual scheme instead of prompting a reconsideration of increasingly fragile primitives.

This tendency of thought is arguably rapidly snowballing into AI philosophy’s analogue of the Bitter Lesson: Philosophies of AI grounded in mindful attention to the actual behavior of models are beginning to outpace philosophies grounded in increasingly overwrought a priori conceptual architectures that began as prefigurations of types of AI we did not actually end up with.

Two recent examples are Demis Hassabis’ A Framework for Frontier AI and the Dawning of a New Age, and Mira Murati’s The Future Worth Building is Human. Both can be read as awkward political PR documents (by people not used to doing political PR), resting on top of unreconstructed and dated philosophy of science commitments, resting on top of impeccable technical credentials in the business of AI machinery, which I will distinguish from AI proper – the behavioral space of models.

I would not challenge either author on technical matters concerning the machinery of course, and both are doing about as well as you might expect (ie poorly, but sincerely and acceptably) when scientists reluctantly lurch into political arenas. The problem is with the middle layer – the philosophy of science that we’re getting from all AI leaders (not just Hassabis and Murati) is uniformly somewhere between underwhelming and alarmingly confused, even after making allowances for the fact that they have to be much more careful and circumspect about what they say than the rest of us.

Why are people who are handing us all tools of near-magical power offering only philosophical pablum to go with it, to make sense of what we’re holding in our hands?

Why in spite of having extraordinarily privileged views of what is going on, and being genuinely thoughtful legitimate geniuses, are they unable to say extraordinarily insightful things about AI that are significantly more philosophically dense than the average dabbler-commentator with no privileged access?

What is the nature of the looming Philosophical Bitter Lesson that is limiting the visions of the best minds in AI, and what can we do about it?

---

Discovery vs. Invention

My basic diagnosis is simple: Unlike previous epochal developments in technology and computing, AI is more discovery than invention, and offers no particularly privileged view of the new territories it opens up to anyone.

Our inadequate philosophizing is downstream of this basic miscategorization of what we’re philosophizing about.

There is a good deal of invention in store for us in the future, as we switch from installation to deployment mode. Much of what I have dubbed New Nature remains to be invented. The entirety of civilization will likely need reinvention.

But we are currently in a discovery phase, examining the knots and cracks of a bounty of crooked timber. And we are attempting to make sense of what we’re seeing through constitutions and sermons.

To be somewhat more precise, while there are a lot of elements of inventive genius in modern AI, the part that is epic is not an invention, and the parts that are inventions are not epic.

In making philosophical sense of an epic discovery, being a genius doesn’t help. Being among the first few people to witness the vistas opened up by a new model doesn’t help. Influencing political agendas and the movement of billions of dollars doesn’t help. Working hard to make the technology safe doesn’t help. Even having an unlimited token budget for exploration doesn’t help that much – you may be able to coax more monumental stunts out of models, but you won’t necessarily “understand” what’s happening any better. Exploring the territories opened up by an epic discovery requires expeditions, not monuments.

I put understand in scare quotes because the word means entirely different things for discoveries as opposed to inventions.

You understand a discovery by exploring and mapping the territories it opens up, taking specific risks as you push the frontier outwards, making more regions safely habitable as you go, taming wildernesses as needed (and ideally only as much as needed).

You understand an invention by reflecting on it, constructing models of how it works, and trying to make rigorous, ideally mathematical statements about them, and trying to enforce verifiable properties on its behavior.

Applying sense-making and regulation techniques meant for inventions to discoveries results in the famous spherical cow in a vacuum syndrome.

To “understand” America after discovering it is a different matter from understanding a jet engine (once you’ve got a prototype working) or general relativity (understood as a conceptual invention). Columbus, for instance, went to his grave still thinking he had discovered a new route to Asia rather than something vastly more significant – two massive new continents. Many of the pioneering explorers who followed, for a long time, radically underestimated just how huge the Americas were, how much there was to be learned about them, and perhaps most importantly, the mechanisms by which their discovery would radically transform the rest of the planet (through what is now known as the Columbian Exchange).

The hundreds of explorers who did eventually construct an accurate geographic understanding of the Americas, and mediate its influence on the rest of the world, were perhaps daring, enterprising, ambitious, and curious. But they were not generally intellectual heavyweights (with a few exceptions like Alexander von Humboldt).

Discoveries require active exploration to comprehend. Genius offers no particular advantage (and can even be a liability as we’ll see), and resources you bring with you from the civilizational core only get you so far. An ordinary person in the right corner of the newly discovered terrain can often learn more than a genius armed with billions. Advantages in the Old World (such as resistance to mosquito-borne illnesses, or inventive genius at the level of AI machinery) can turn into liabilities in the New World (becoming a target for enslavement by the less mosquito-resistant, or stumbling on philosophical questions).

Most pertinent for the leaders of AI, who must direct the energies and resources of others besides themselves, up to the level of entire nation-states, the philosophy of science required for discovery is very different from the philosophy of science required for invention.

---

Daily Walks in Logical Geographies

Hassabis’ essay is, in my opinion, not even wrong in its implicit philosophy of science, and close to what Gilbert Ryle called philosophical nonsense. In part because it relies for its arguments on what we might call architectonic maps (constructed out of a priori ontological notions like “AGI” for example) rather than phenomenological accounts (noticing behaviors of specific models under specific conditions while trying to do specific things for example).

Ryle makes use of a geography metaphor that is particularly apt for our discovery-over-invention diagnosis. As the Stanford Encyclopedia of Philosophy page on Ryle notes:

We are like the villager with respect to our employment of words and phrases. Knowledge by wont of the use of expressions and of concrete ideas is something everybody learns in the course of growing up speaking and understanding a language: “their ‘logical geography’ is taught by one’s daily walks.” So too do we know how to operate with ordinary, non-technical, and even semi-technical and technical expressions as well as with relatively concrete and some abstract ideas without being able to codify the rules, permissions, or sanctions that govern their operations.

Ryle was the pioneering philosopher of ordinary language and category errors, best known for finally putting Cartesian dualism out of its misery. He was also, like his Oxford colleague Isaiah Berlin, an anti-idealist and attuned to the “thickness” of everyday phenomenological experience, and suspicious of rarefied conceptual axiomatics. Hassabis’ essay is clearly on shaky ground from a Rylean perspective.

Murati’s essay, while significantly more coherent, and approaching tantalizingly close to the crooked-timber view of AI I’m advocating for here, still disappoints, and ultimately retreats to the same tired architectonic language that has become the default for AI thought leadership. Her vision is humanist in a stylized Arendtian way without seriously concerning itself with the nature of actual humans (crooked timber or otherwise) and how it is being transformed. It is pluralist but only in an impoverished analytic way, relying on a platonic notion of “decentralized alignment” for instance, rather than grappling with the crookedness of the timber, and the difficulties that might be involved in crafting particular pieces of furniture out of particular pieces of crooked timber.

Genius shines when there is architectonic work to be done. When precise entailments have to be worked out starting from axiomatic commitments, and esoteric mathematics deployed to tease out subtleties. Phenomenological work – the construction of maps based on explorations of new territories – is a bottom-up test of stamina where genius doesn’t help, but mindful attunement to experience does.

There is definitely room and need for inventive genius in the world of AI, but as I observed earlier, the part that is epic is not invention, and the part that is invention is not epic.

Take, for example, three critical enabling components of the machinery of AI: convolution, back propagation, and the attention mechanism. Much to my shock, I understood the basic idea of all three the first time I read about them. All three ideas are mathematically straightforward, and accessible with high-school math. So is much of the mathematics of the complicated ramifications that appear once you’re trying to juggle millions of fragments of matrices with thousands of processors in a computing cluster. It is complicated but not complex. Overwhelming, but not mystifying. The complexity and mystery of AI do not dwell at these machinic levels.

Contrast that with (say) the basic workings of a rocket engine or the conceptual machinery of relativity theory, neither of which I understand anywhere near as well, despite spending vastly more time trying (and despite having a PhD in aerospace engineering in the former case).

The hard aspects of AI are simply not the sorts of hard things geniuses are great at figuring out better than the rest of us. Predictably, AI itself has rapidly become very useful in figuring out those kinds of hard things, perhaps feeding some of the AI psychosis geniuses seem particularly prone to.

Being a genius might get you one of the scarce million-dollar jobs at a frontier lab, wrangling large-scale matrix black magic problems, but it doesn’t seem to help understand what’s going on. It doesn’t seem to point to adequate philosophies.

What does seem to help is having an expeditionary, cartographic mindset, and perhaps specialized tooling, such as software that can take “fMRI” images of models in action, capable of revealing things like patterns of weight activation. Like telescopes, cameras, and microscopes, these are tools of discovery, not engines of invention. Or harnesses that keep you oriented as you explore.

In the Age of Exploration, the invention elements – advanced sailing ships, marine chronometers, navigation techniques, astronomical computations – were of little use in understanding the discovery itself. Our understanding of the Americas and the Columbian Exchange does not come from the expert clockmakers who built the marine chronometers that made reliably voyaging to the new world possible.

Our understanding of AI will not come from the expert matrix-multipliers who are building the mathematical and silicon machines that allow us to experience it.

---

Bitter Lessons and Bewildered Inventors

Much of the confused thinking around AI is perhaps attributable to the fact that our first attempt at creating AI approached it like an invention problem and enjoyed significant success, even though the program fell radically short of its stated objectives and triggered an infamous academic winter.

Unlike Deep Learning, GOFAI was a design space where inventive genius could flourish. SAT solvers, STRIPS planners, complex hand-tuned learning algorithms, early chess-playing machines, common-language reasoners from Cyc to Watson – these were genuine, complex, and sometimes even elegant inventions. Inventions that took genius to imagine and build. Understood independently of the “AI” vision they failed to realize in aggregate, these were great inventions, and remain useful. It was the vision itself that was flawed, not the accomplishments logged on the road towards it.

It just so happened that the term “AI” makes far more sense as a pointer to a new continent of complex discovered phenomenology (for which “latent space” has emerged as a metonymous name) than as a name for a collection of clever inventions.

The bitter lessons of AI, including the philosophy bitter lesson, are best understood as the bewilderment of talented inventors stumbling upon a major discovery. Clockmakers faced with continents opened up by their cunning instruments, and called upon to make sense of them with reference to clock-making knowledge. They cannot be faulted for failing. But they can be faulted for conflating clocks and continents.

AI leaders are falling prey to the philosophical bitter lesson because they treat AI as a highly malleable and homogeneous engineering material, rather than one with knots, grains, anisotropy, cracks, large-scale variations in gross morphology and microstructure, variable material properties, and so on, inherited from the phenomenology of the training data. Like biological organisms, AI models are what they eat at least as much as they are products of their “DNA.”

So far, we have assumed that AI is philosophically characterizable by a small set of what material scientists call intensive properties, such as density, tensile strength, and so on, which can be measured at a point. The AI equivalents are relatively legible mechanism elements like number of parameters, attention heads, context lengths, and cache sizes, measurable for a given model. These are awkwardly yoked to constructs like “AGI” and “alignment” for philosophical analysis and synthesis.

We are learning that we need to explore, in addition to intensive properties of AI machinery, the extensive properties of the behavior spaces of models, in every nook and cranny. And that we must yoke what we learn to thick, everyday language that emerges from actually “walking the logical geography” in order to philosophize.

Like 19th century naturalists, we find we have to go around collecting and labeling curious new species, taking note of geological strata and fossil records, noting similarities and differences across models, mapping traces of deep time, and so on.

---

Where the Mysteries Lurk

The emergent behaviors of AI are mystifying in ways that the underlying mathematical and electronic machinery is not. That is where AI philosophy must focus, if it is to be at all useful.

It is worth being specific about where the mysteries seem to lie, based on what we have discovered so far. Things have been surprising, to say the least.

To date, AI has arguably exhibited no phenomenology salient to the sorts of mysteries that philosophers of mind concern themselves with, such as the nature of subjectivity and selfhood, the “hard problem” of consciousness, inverted spectra, and so on. At best, certain computation-derived metaphors, such as Searle’s Chinese Room argument, have been undermined. As someone who was deeply interested in that subject for about a decade before the dawn of deep learning, I have found no reason to update any of my beliefs or positions on philosophy of mind matters.

What AI has surfaced though, is mysteries we had no idea were lurking in the depths of language and historical data. The emergent mysteries of AI, it turns out, concern the nature of language and data, the relationship between knower and knowing, and the relations between time, energy, and memory. Here, I mean “language” in the broadest sense, to include such domains as robotic motion or protein folding, not just symbol-mediated human natural language.

These mysteries are mysteries of discovery rather than invention. Akin to a continent shrouded in fog, glimpsed in the distance from an exploring ship, rather than an idea like “time-travel machine” or “heavier-than-air flight.”

You do not tackle discovery mysteries by thinking about foggy data in an armchair with a limited set of first principles. You tackle them by equipping yourself for an expedition and pushing towards the frontier through the fog, mapping, taming, and building as you go.

If you do try to axiomatically reason your way through the major problems of AI philosophy, you will find yourself circling either the philosophical Bitter Lesson in Cartesian bewilderment, or deriving irrelevant and likely lurid conclusions from inappropriate analytic concepts that drift farther and farther from the actual data pouring in from millions of voyages of discovery.

Is there a better way to philosophize AI?

Isaiah Berlin, arguably, was a philosopher concerned with similar discoveries from the realm of human behavior under the stresses of modernity, the original pile of crooked timber that has bequeathed us our new pile. What might we learn from him?

---

A Post-Romanticism for AI

Like Ryle, Berlin was anti-idealist. Unlike Ryle, he was also what we might call post-Romantic with an essentially tragic sense of the limits of intelligibility of our moral universe.

Post-Romanticism, understood as Berlin’s retrospective synthesis of the ideas of the Enlightenment and the Romantic era that followed it, in light of the two bloody world wars sparked by the tensions between them, turns out to have a lot to say on matters that should sound very familiar to people thinking about the philosophy of AI.

In The Apotheosis of the Romantic Will, Berlin has this to say on the question of what we might call the “alignment” problem today in an AI context (illustrating, along the way, why “alignment” is a terrible geometric metaphor):

But if it is the case that not all ultimate human ends are necessarily compatible, there may be no escape from choices governed by no overriding principle, some among them painful, both to the agent and to others. From this it would follow that the creation of a social structure that would, at the least, avoid morally intolerable alternatives, and at the most promote active solidarity in the pursuit of common objectives, may be the best that human beings can be expected to achieve, if too many varieties of positive action are not to be repressed, too many equally valid human goals are not to be frustrated. But a course demanding so much skill and practical intelligence – the hope of what would be no more than a better world, dependent on the maintenance of what is bound to be an unstable equilibrium in need of constant attention and repair – is evidently not inspiring enough for most men, who crave a bold, universal, once-and-for-all panacea. It may be that men cannot face too much reality, or an open future, without a guarantee of a happy ending – providence, the self-realising spirit, the hidden hand, the cunning of reason or of history, or of a productive and creative social class.

To the list in the last sentence, we might now add “aligned AGI.”

Elsewhere in the same essay, commenting on the ultimate failure of Romanticism, Berlin offers a cautionary opinion on the role of subjectivity, art and aesthetics, which should give pause to those who think, as the Jena Romantics did, that “taste” or “poetry” will somehow unlock answers to the thorniest AI philosophy challenges, and leave humans with a lofty and unassailable role in a different sort of perfected future.

Berlin argued that while aesthetics and art are valuable insofar as they subvert dangerous universalisms, they cannot conjure meaningful alternatives and may precipitate disasters if we try:

If some ends recognised as fully human are at the same time ultimate and mutually incompatible, then the idea of a golden age, a perfect society compounded of a synthesis of all the correct solutions to all the central problems of human life, is shown to be incoherent in principle. This is the service rendered by romanticism and in particular the doctrine that forms its heart, namely, that morality is moulded by the will and that ends are created, not discovered. When this movement is justly condemned for the monstrous fallacy that life is, or can be made, a work of art, that the aesthetic model applies to politics, that the political leader is, at his highest, a sublime artist who shapes men according to his creative design, a fallacy that leads to dangerous nonsense in theory and savage brutality in practice, this at least may be set to its credit: that it has permanently shaken the faith in universal, objective truth in matters of conduct, in the possibility of a perfect and harmonious society, wholly free from conflict or injustice or oppression – a goal for which no sacrifice can be too great if men are ever to create Condorcet’s reign of truth, happiness and virtue, bound by an indissoluble chain – an ideal for which more human beings have, in our time, sacrificed themselves and others than, perhaps, for any other cause in human history.

Reflecting on the history of a century of failure of both rational and romantic approaches to the modern human condition, Berlin argued that moral philosophy can only ever be contingent, situated in history, and limited to the stated specificity of a particular situation.

There are no once-and-for-all panaceas, and no lofty overriding principles. “Taste” will not do as a substitute for difficult, incomplete, and non-general moral reasoning.

There are only difficult tensions that must be navigated one at a time and contingent solutions that require constant maintenance and management.

But whether you view this as a tragedy, or the most sublime kind of liberation, depends on whether you view crookedness in timber (human or AI) as a distortion to be eliminated, or the essence of all meaning.

Here, I perhaps part ways with Berlin. Though the moral conclusions of crooked-timber philosophy are sobering, ultimately I find the crookedness of the timber to be what makes working the wood worthwhile at all.

---

A Post-Romantic Planetarity

Here at the Protocol Institute, a few of us are beginning initial deliberations on what we hope will evolve into a “New Nature” approach to the challenges of AI invention, adoption, regulation, and governance, rooted in a crooked-timber philosophy of what it is. With or without AI in the picture, human ends are going to be constructed and irreducibly incompatible. And with or without AI, we will be working with crooked timber to try and achieve them.

This will be a long-term, ongoing project for PI, and we are actively seeking financial support for this work, so if you are interested, do reach out.

One reason we have gravitated towards a crooked timber approach is that our multi-year study of protocols has repeatedly reinforced at a technological level precisely the lessons of history that Berlin offers at a philosophy-of-history level, down to some rather uncanny similarities in phrasing.

For instance, avoiding “once-and-for-all” panaceas has become a central tenet of our design principles for protocol-based solutions to problems. Another instance: Disciplined futures techniques to think about “incompatible ultimate human ends” have been a staple of much of our programming since 2024.

And though protocols, especially technological ones, can have cosmetic architectonic and high-modernist appearances, in general they tend to have an underlying phenomenological bias – good protocols emerge as paved cowpaths more often than they do from Cartesian imaginaries. They may feature straight lines, but are born of crooked realities, and creatively manage tensions between smooth and striated, arborescent and rhizomatic, wild and civilized.

Their evolution tends to be attuned to Rylean “daily walks in local geographies.” Vocabularies of use that emerge around protocols, even when specialized, tend to constitute an everyday language of thick experience.

Protocols, it turns out, are a post-romantic technological language, remarkably well suited to the deployment phase of AI that is already upon us.

In terms of Carlota Perez’s models of technological revolution, we are currently nearing the end of the installation phase of AI (likely the fastest such phase in history), focused on the proliferation of infrastructure and the emergence of early economic and political patterns. Unlike the installation phases of comparably epochal technologies, such as railroads, electricity, and the internet, the installation of AI features much more discovery than invention. We’re making our maps of latent space, so to speak, at the same time as we are laying down our railroad tracks to access it.

And even though there is a lot of discovery and installation left to do, and the maps far from complete, the invention and deployment phase has already begun. Already the settlers and town-planners following on the heels of the pioneers and map-makers are faced with difficult design choices. And available AI philosophies have little to offer.

How do you adopt AI in existing organizations? How do you design AI-native new organizations? How should existing AIs navigate the challenges of sovereign AI? How should post-national infrastructures such as blockchains prepare for the deployment of AI? How should war be waged in an AI world? How can peace be sustained?

Crooked-timber philosophy, I believe, can provide useful post-romantic answers to such questions where existing AI philosophies mostly offer platitudes and theaters.

Most of these questions, as I argued in The Fabric and the Brain, are protocol design questions, and as with the installation-phase infrastructural AI technologies, a tension is emerging between architectonic and phenomenological approaches. Our sympathies and energies are obviously being directed towards the latter.

These are not questions to be definitively answered, or even posed, once-and-for-all. A crooked-timber approach would suggest that we muddle through, armed with whatever provisional constructs we can carry with us, as we attempt to discover, map, and build.

This is a major part of what we’ll be attempting to do over the next year in our work at PI. We hope you’ll come along for the ride.

---

Enter the Jamverse Jam!

Protocolized’s fourth open submission contest is live for two more weeks: artists and writers are invited to extend and connect the Jamverse, a network of interoperable, strange worlds designed by four of our frequent contributors.

We have a $1,000 grand prize, with at least 10 other entries receiving $200 prizes. Deadline for entries is July 31. Read more and enter at the Jamverse microsite.