Toward Interpretable Word Embeddings Brash means bold, audacious, brazen. It can also have secondary meanings with a negative hue, like impertinent, impudent, insolent, rude. And mix in a bit of pushy, reckless, rash, impetuous (in the sense that it is quick, not slow; reckless, not cautious; rash, not considered; pushy,…
Tethering The Model to Human Intent
Tethering the model to human intent is, indeed, a difficult problem. It won’t be solved, at least, not directly, through the NWITS model. In order to tether it to human intent, it must be tethered to language and its corresponding intent. How do you do that? Well, here’s a way:…
The Physics of Meaning: 1
It is possible that by decoding language, we are decoding reality. Not just because language reflects reality in a general way, but if it is true that the columns in the neocortex all do the same thing, and that some of them are responsible for creating, emitting, and receiving language…
Generally speaking
There should be a way to designate negative vs. positive, which doesn’t rely on word similarity, since their meanings are, though on the same dimension, oppositional to one another. What kind of a mathematical function would represent this general idea? For instance, the positive would likely be addition/multiplication. You’re moving…
Overflowing
Abstraction via Repetition In a fastAI class, I asked the following: Do any language models attempt to provide meaning? For instance, “I’m going to the store” is the opposite of “I’m not going to the store.” Or “I barely understand this stuff” and “That ball came so close to my…
The Calculus of Everyday Life
Toward Experimental Techniques If I recognize a pattern, a machine should be able to as well, given enough training data. However, a machine might also recognize something other than what I am seeing. Therefore it is better to indicate to the machine what exactly I have in mind. This is…
On Intelligence
Jeff Hawkins had an interesting conversation with Lex Fridman, referring to Hawkins’s two books — On Intelligence, and A Thousand Brains. Hawkins proposes that the same mechanism that we use to model a water bottle can be used to model high-level thoughts. This means that knowledge itself is fractal, reflecting…
Animal Magnetism
What Are We Pointing At? Some years ago, I wrote a play about a Viennese doctor in Paris in 1784 named Franz Anton Mesmer. (From him we get the word ‘mesmerize’). His most famous case was a young blind pianist named Maria Theresa Paradis. His patients grasped iron bars that protruded…
The Assignation of the Labels
The assignation of the labels is what tells the network what you’re trying to teach it. Not the more granular features that constitute the reason that this particular input has the label and that one over there doesn’t, but the one-level-up-in-abstraction label. It isn’t the two eyes, the nose, the…
Convolving on Language
Convolving on Language In image recognition we are scanning pixels, ie data at a low level, to allow the computer to classify things that are high-level. This classification, though it is done by means that may or may not mirror what we actually do, arrives at a human-recognizable result. And it…