11–20 of 191 posts
That the current LLMs have not achieved AGI is fair, but calling them cargo cult machines goes a bit far.
(We could have a cargo cult discussion about science, though, which put extreme titles on articles with little substance. This is cargo cult in my opinion.)
Every time I read about AI I am reminded of the mouse running a maze. Any AI algorithm can learn to complete a maze in record time. It can memorize every corner. It can run a search pattern perfectly and improve that pattern iteratively, to the point that it may create new search patterns, applying what appear to be novel ideas. But the mouse actually understands the concept of a maze. It knows that the cheese exists…
Although I more or less agree with you, in general, how complex would the AI have to be before you can consider it aware?
Intelligence has been constantly redefined over the years as other animals and then as mechanical means have met the previous standards for intelligence. Intelligence is a moving target. With no background in that, the author boldly declares that scientific thinking is the real goal . . . the real intelligence. It's a little hard to believe.
ACM clickbait?
I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.
For example, I get _unbelievably_ excited with knowledge extraction and question answering demos on PDFs. Why? Because i've built similar systems for over a decade and know how difficult it is to build on top of messy archival data. Now, with very little code, i'm getting SOTA results.
If you didn't have experience with this, you'd probably thing "Huh, that's not impressive, XYZ does this already". But it's the moving of the baseline that's what's really impressive.
===
AI hype-beasts aside of course - the breathless pontificating of "influencers" and former crypto bros is cringe.
I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.
Just made me more aware of how bulshit is the norm and not the exception.
I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.
Simple things that don't seem impressive to you, but are impressive to others, may be because you haven't experienced the first hand difficulty of doing that thing pre-transformers/GPT. For example, I get _unbelievably_ excited with knowledge extraction and question answering demos on PDFs. Why? Because i've built similar systems for over a decade and know how difficult it is to build on top of messy archival data. N…
* complains that we still lack a "comprehensive theory to explain what intelligence is or how it emerges from first principles,"
* argues that deep neural nets like LLMs may not be capable of artificial general intelligence (AGI), and
* contends that achieving AGI will require "new algorithmic paradigms."
Rich Sutton wrote what I think is the perfect counter-argument to these points some years ago:
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. These two need not run counter to each other, but in practice they tend to. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or the other. And the human-knowledge approach tends to complicate methods in ways that make them less suited to taking advantage of general methods leveraging computation."[a]
Go read the whole thing.
---
I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.
I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.