Prediction of the Next Word In The Sentence (NWITS) is the very definition and measure of a language model. It leaves out meaning, and for this reason it has become the subject of some derision, bordering on scorn, in spite of the remarkable capabilities of ChatGPT. If we modify this…
Convolving on Language
Convolving on Language In image recognition we are scanning pixels, ie data at a low level, to allow the computer to classify things that are high-level. This classification, though it is done by means that may or may not mirror what we actually do, arrives at a human-recognizable result. And it…
The Usual Suspects
Writing, Reading, Looking, Convolving Imagine you’re on a train. You pass the graffitied backs of warehouses, front-loaders parked in a snowy lot, a red-brick-and-white-wood apartment building, an electric company grid station, and then trees, a lake, and snow-dusted hills in the distance. If you turn your head, you can turn…
Pattern Recognition
On Classes and Instances Are there any concepts that could be of use in suggesting how machine learning could understand and convey meaning? I think there are: repetition and generalization. In the words of the great Philosopher Charles Barkley, I may be wrong, but I doubt it. I think perception…
Artificial Thought (Hello, word)
I’m an alien. But I’ll write this as though I were a human being. This is a blog about the relationship between human thought and machine learning. Specifically, about identifying patterns in human thought that might then be able to be identified by a machine. The vehicle for doing this…
Repetition and Generalization
First Principles Are there any concepts that could be of use in suggesting how machine learning could understand and convey meaning? I think there are: repetition and generalization. I may be wrong about this. In the words of the great Philosopher Charles Barkley, I may be wrong, but I doubt…
How Might a Human Machine Think?
How Might a Human Machine Think? In a Kaggle competition on distracted driving, the dataset consisted of images of drivers who either were or weren’t engaging in distracting activities. Rachel Thomas warns against “…overfitting to particularities of those specific people, and not just learning the states (texting, eating, etc). (emphasis added)”…
Generalization
Current AI really isn’t mimicking the brain, other than in name. “Neural networks” are indeed networks, but they really have very little to do with neurons. The general idea of an ‘activation function’ comes from neurons, but in AI it just means “if the aggregate addition of all these little…
On Recognition
The Mind of the Computer A Convolutional Neural Network learns to classify an image. What is the computer doing when it does this feat of ‘recognition’ ? Is it something we understand? Here is what we know. Digital images are submitted as inputs (grids of pixels, with RGB values, i.e., numbers). Then,…