Some time ago I wrote a post about the writers’ strike, and what I thought the real threat was, not just to writers, but to the film studios as well. This past week, my prediction came true, at least in preview, in the form of OpenAI’s Sora. It’s a high-fidelity…
The Atomization of Meaning – 1
Toward Interpretable Word Embeddings Brash means bold, audacious, brazen. It can also have secondary meanings with a negative hue, like impertinent, impudent, insolent, rude. And mix in a bit of pushy, reckless, rash, impetuous (in the sense that it is quick, not slow; reckless, not cautious; rash, not considered; pushy,…
RAGs to Riches
I’ve been diving into the world of Retrieval Augmented Generation (RAG), a framework that empowers large language models (LLMs) to access external data sources. This is a game-changer for overcoming the limitations of LLMs, particularly the issue of knowledge cutoffs. While retraining the model with new data is an option,…
The Writers’ Strike
The writers’ strike is in its fifth month. I support the writers in their efforts to share in the wealth of the film industry, and I understand their concern about being terminated (so to speak) by AI. But I think they have some things very wrong. The thrust of the…
Tethering The Model to Human Intent
Tethering the model to human intent is, indeed, a difficult problem. It won’t be solved, at least, not directly, through the NWITS model. In order to tether it to human intent, it must be tethered to language and its corresponding intent. How do you do that? Well, here’s a way:…
The Physics of Meaning: 1
It is possible that by decoding language, we are decoding reality. Not just because language reflects reality in a general way, but if it is true that the columns in the neocortex all do the same thing, and that some of them are responsible for creating, emitting, and receiving language…
Generally speaking
There should be a way to designate negative vs. positive, which doesn’t rely on word similarity, since their meanings are, though on the same dimension, oppositional to one another. What kind of a mathematical function would represent this general idea? For instance, the positive would likely be addition/multiplication. You’re moving…
Noise is the Norm
The Noisy Hypernym A word is a commonality at the intersection of multiple meanings. A hypernym can be thought of as a superset of any given word. For instance, ‘animal’ would be a hypernym of cat, dog, walrus, and of course, dung beetle. Examples of hypernyms from wordnet: bay: {‘hypernyms’:…
Overflowing
Abstraction via Repetition In a fastAI class, I asked the following: Do any language models attempt to provide meaning? For instance, “I’m going to the store” is the opposite of “I’m not going to the store.” Or “I barely understand this stuff” and “That ball came so close to my…
The Calculus of Everyday Life
Toward Experimental Techniques If I recognize a pattern, a machine should be able to as well, given enough training data. However, a machine might also recognize something other than what I am seeing. Therefore it is better to indicate to the machine what exactly I have in mind. This is…