191–200 of 359 posts
Even AI companies have a hard time figuring out how emergent capabilities work.
Almost nobody in the general audience understands how LLMs work.
Many people who claim that people don't understand how AI works often have a very simplified view of the short comings of LLMs themselves, e.g. "it's just predicting the next token", "it's just statistics", "stochastic parrot" and seems to be grounded in what AI was 2-3 years ago. Rarely have they actually read the recent research on interpretability. It's clear LLMs are doing more than just pattern matching. They ma…
Earlier quoted context omitted.
I can only agree with you. And I find it disturbing that every time someone points out what you just said, the counter argument is to reduce human experience and human consciousness to the shallowest possible interpretation so they can then say, “look, it's the same as what the machine does”.
I think it’s because the brain is simply a set of chemical and electrical interactions. I think some believe when we understand how the brain works it won’t be some “soulful” other worldly explanation. It will be some science based explanation that will seem very unsatisfying to some that think of us as more than complex machines. The human brain is different than LLMs, but I think we will eventually say “hey we can…
https://thebullshitmachines.com/
Not everything needs to pass a NASA quality inspection to be useful.
Earlier quoted context omitted.
>Deepmind already have project Astra, a model but not just language but also visual and probably some other stuff where you can point a phone at something and ask about it and it seems to understand what it is quite well. Operative phrase "seems to understand". If you had some bizarre image unlike anything anyone's ever seen before and showed it to a clever human, the human might manage to figure out what it is after…
> If you had some bizarre image unlike anything anyone's ever seen before and showed it to a clever human, the human might manage to figure out what it is after thinking about it for a time Out of curiosity, what sort of 'bizarre image' are you imagining here? Like a machine which does something fantastical? I actually think the quantity of bizarre imagery whose content is unknown to humans is pretty darn low. I'm no…
Historically, this just hasn't ever been the case. There are images today that wouldn't have merely been outlandish 150 years ago, but absolutely mysterious. A picture of a spiral galaxy perhaps, or electron-microscopy of some microfauna. Humans would have been able to do little more than describe the relative shapes. And thus there are more images that no one will be familiar with for centuries. But if we were to somehow see them early, even without the context of how the image was produced I suspect strongly that clever people might manage to figure out what those images represent. No model could do this.
The quantity of bizarre imagery is finite... each pixel in a raster has a finite number of color values, and there are finite numbers of pixels in a raster image after all. But the number is staggeringly large, even the subset of images that represent real things, even the subset of that which represents things which humans have no concept of. My imagination is too modest to even touch the surface of that, but my cognition is sufficient to surmise that it exists.
Earlier quoted context omitted.
Thinking in humans is prior to language. The language apparatus is embedded in a living organism which has a biological state that produces thoughts and feelings, goals and desires. Language is then used to communicate these underlying things, which themselves are not linguistic in nature (though of course the causality is so complex that the may be _influenced_ by language among other things).
This is really over indexing on language for LLMs. It’s about taking input and generating output. Humans use different types of senses as their input, LLMs use text. What makes thinking an interesting form of output is that it processes the input in some non-trivial way to be able to do an assortment of different tasks. But that’s it. There may be other forms of intelligence that have other “senses” who deem our abil…
Earlier quoted context omitted.
This sounds very wise but doesn’t seem to describe any of my use cases. Maybe some use cases are divination but it is a stretch to call all of them that. Just looking at my recent AI prompts: I was looking for the name of the small fibers which form a bird’s feather. ChatGPT told me they are called “barbs”. Then using straight forward google search i could verify that indeed that is the name of the thing i was lookin…
Sure, some people stepped up to the Oracle and asked how to conquer Persia. Others probably asked where they left their sandals. The quality of the question doesn't change the structure of the act. You presented clear, factual queries. Great. But even there, all the components are still in play: you asked a question into a black box, received a symbolic-seeming response, evaluated its truth post hoc, and interpreted…
I agree with the substance, but would argue the author fails to "understand how AI works" in an important way: LLMs are impressive probability gadgets that have been fed nearly the entire internet, and produce writing not by thinking but by making statistically informed guesses about which lexical item is likely to follow another Modern chat-tuned LLMs are not simply statistical models trained on web scale datasets.…
Many like the author fail to convince me because they never also explain how human minds work. They just wave their hand, look off to a corner of the ceiling with, "But of course that's not how humans think at all," as if we all just know that.
Now, you could argue that, even though the substrate is different, some important operations might be equivalent in some way. But that is entirely up to you to argue, if you wish to. The one thing we can say for sure is that they are nothing even remotely similar at the physical layer, so the default assumption has to be that they are nothing alike period.
LLMs are divinatory instruments, our era's oracle, minus the incense and theatrics. If we were honest, we'd admit that "artificial intelligence" is just a modern gloss on a very old instinct: to consult a higher-order text generator and search for wisdom in the obscure. They tick all the boxes: oblique meaning, a semiotic field, the illusion of hidden knowledge, and a ritual interface. The only reason we don't call i…
The terminology is so confusing in AI right now. I use LLMs, I enjoy them, I'm more productive with them. Then I go read a blog from some AI devs and they use terms like "thinking" or similar terms. I always have to ask "We're still s stringing words together with math right? Not really thinking right?" The answer is always yes ... but then they go back to using their wonky terms.
Earlier quoted context omitted.
Oh, interesting, TIL. Didn't realize there was a second step to training these models.
Got 3½ hours? https://youtu.be/7xTGNNLPyMI (I watched it all, piecemeal, over the course of a week, ha, ha.)
here's a one hour version that helped me understand a lot