The Baby and the Bathwater Let us be neither too rapturous, nor too fearful, of LLM’s. Since the advent of ChatGPT, we’ve been whipsawed between rhapsodic infatuation, and a kind of weary lamentation of its lack of true understanding. There is no doubt that these Large Language Models (LLMs) are…
What is the opposite of water?
The Infinite Monkey Theorem, Revisited What is the opposite of water? There is none. Or is there? Two points determine a line. You need one other point to make sense of this. One possibility is fire. On the fire/water line, they’re opposites. Another is air. On the air/water line, they’re…
On Editing
Editing is an example of finding and fixing ‘mistakes’. As you read through something you’re working on writing, you make changes here and there. It is to be expected that this process would be very similar to the process of stepping down a ladder and suddenly discovering that that last…
On Alignment
“Alignment” refers to the AI carrying out the intent of the human that is calling it. If ML models are black boxes, then we are in trouble. Let’s say I’m in conversation with another person, who I’ve asked to translate a body of text, from English into Chinese. I don’t…
On Intelligence
Jeff Hawkins had an interesting conversation with Lex Fridman, referring to Hawkins’s two books — On Intelligence, and A Thousand Brains. Hawkins proposes that the same mechanism that we use to model a water bottle can be used to model high-level thoughts. This means that knowledge itself is fractal, reflecting…
Animal Magnetism
What Are We Pointing At? Some years ago, I wrote a play about a Viennese doctor in Paris in 1784 named Franz Anton Mesmer. (From him we get the word ‘mesmerize’). His most famous case was a young blind pianist named Maria Theresa Paradis. His patients grasped iron bars that protruded…
Representing Reality
Leveraging Film The attempt to embody the real world is the central aim of Artificial General Intelligence (AGI). Fortunately, there are some tools already at our disposal from the medium that has made its business precisely that: the world of film. One hopes that, until the day we have sensory-empowered…
The Trouble With Linguistics
Language is beautiful. The study of language, linguistics, less so. Consider the following, from the Internet Encyclopedia of Philosophy: Compositionality of stereotype: the stereotype associated with a complex expression E in a natural language is determined by (and only by) (i) E’s morphosyntactic structure and (ii) the stereotypes associated with…
Begging The Question
A brief personal rant “Begs the question” does not mean the same as “raises the question,” though it is increasingly being used that way. Merriam-Webster defines “begging the question” as “to pass over or ignore a question by assuming it to be established or settled.” In other words, it means…
The Assignation of the Labels
The assignation of the labels is what tells the network what you’re trying to teach it. Not the more granular features that constitute the reason that this particular input has the label and that one over there doesn’t, but the one-level-up-in-abstraction label. It isn’t the two eyes, the nose, the…