Toward Interpretable Word Embeddings Brash means bold, audacious, brazen. It can also have secondary meanings with a negative hue, like impertinent, impudent, insolent, rude. And mix in a bit of pushy, reckless, rash, impetuous (in the sense that it is quick, not slow; reckless, not cautious; rash, not considered; pushy,…
The Physics of Meaning: 1
It is possible that by decoding language, we are decoding reality. Not just because language reflects reality in a general way, but if it is true that the columns in the neocortex all do the same thing, and that some of them are responsible for creating, emitting, and receiving language…
Premature Emasculation
The Baby and the Bathwater Let us be neither too rapturous, nor too fearful, of LLM’s. Since the advent of ChatGPT, we’ve been whipsawed between rhapsodic infatuation, and a kind of weary lamentation of its lack of true understanding. There is no doubt that these Large Language Models (LLMs) are…
The Trouble With Linguistics
Language is beautiful. The study of language, linguistics, less so. Consider the following, from the Internet Encyclopedia of Philosophy: Compositionality of stereotype: the stereotype associated with a complex expression E in a natural language is determined by (and only by) (i) E’s morphosyntactic structure and (ii) the stereotypes associated with…
The NWITS model
Prediction of the Next Word In The Sentence (NWITS) is the very definition and measure of a language model. It leaves out meaning, and for this reason it has become the subject of some derision, bordering on scorn, in spite of the remarkable capabilities of ChatGPT. If we modify this…