31–40 of 289 posts
Nothing I’ve seen or learned about LLMs leads me to believe that LLMs are in fact a pathway to AGI.
LLMs trained on more data with more efficient algorithms will make for more interesting tools built with LLMs, but I don’t see this technology as a foundation for AGI.
LLMs don’t “reason” in any sense of the word that I understand and I think the ability to reason is table stakes for AGI.
Almost everything interesting about AI so far has been unexpected emergent behavior, and huge gains through minor insights. While I don't doubt that the current architecture is likely to have a current ceiling below that of peak human intelligence in certain dimensions, it's already surpassed it in some, and there are still gains to be made in others through things like synthetic data. I also don't understand the cla…
> problems that I can absolutely guarantee were not in its training set Can you share the strongest example?
Almost everything interesting about AI so far has been unexpected emergent behavior, and huge gains through minor insights. While I don't doubt that the current architecture is likely to have a current ceiling below that of peak human intelligence in certain dimensions, it's already surpassed it in some, and there are still gains to be made in others through things like synthetic data. I also don't understand the cla…
I mentioned this to another commenter as well: You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled: "Are Emergent Abilities of Large Language Models a Mirage?" https://arxiv.org/abs/2304.15004 https://blog.neurips.cc/2023/12/11/announcing-the-neurips-20...
Also, consider that some work gets a lot of positivity not for the work itself, but for the people who wrote it. Timnit Gebaru's work was effectively ignored until she got famous for her spat with jeff dean at google. Her citations have exploded as a result, and I don't think that most in the field think that the "stochastic parrot" paper was especially good, and certainly not her other papers which include significant amounts of work dedicated to claiming that LLM training is really bad for the environment (despite a single jet taking AI researchers to conferences being worse for the environment than LLM training circa that paper being written was taking). Doesn't matter that the paper was wrong, it's now highly cited because you get brownie points for citing her work in grievance studies influenced subfields of AI.
I think there’s a huge assumption here that more LLM will lead to AGI. Nothing I’ve seen or learned about LLMs leads me to believe that LLMs are in fact a pathway to AGI. LLMs trained on more data with more efficient algorithms will make for more interesting tools built with LLMs, but I don’t see this technology as a foundation for AGI. LLMs don’t “reason” in any sense of the word that I understand and I think the ab…
Earlier quoted context omitted.
And yet, we reached the moon, and I would say airplanes were a necessary step on the way, even if only for psychological reasons. For airplanes we had at least an example in nature, birds. But I am not aware of any animal that travelled from earth to the moon on its own, except us.
But we didn't use airplanes to get there. It needed a new approach, different propulsion, different fuel, different attitude control, etc. etc. LLM may be a necessary step to get to AGI, but it (probably) won't be the one that achieves that goal.
Earlier quoted context omitted.
LLMs are closer to discoveries on the spectrum than inventions. Nobody predicted or planned the many emergent capabilities we’ve seen. Almost like magic. Now is a period of moving along the axis to invention with many intentional design, architecture, and feature development alongside testing and evaluation. We are far from done with LLMs, plenty of room for many more discoveries, lots to explore. It’s definitely a p…
> It’s definitely a precursor to AGI. What are you basing this claim on? There is no intelligence in an LLM, only humans fooled by randomness.
However whatever we're doing seems to be different from what LLMs do, at least because of the huge difference in how we train.
It's possible that it will end up like airplanes and birds. Airplanes can bring us to the other side of the world in a day by burning a lot of fuel. Birds can get there too in a much longer time and more cheaply. They can also land on a branch of a tree. Airplanes can't and it's too risky for drones.
Maintaining the illusion is important to keep the money flowing in.
I think there’s a huge assumption here that more LLM will lead to AGI. Nothing I’ve seen or learned about LLMs leads me to believe that LLMs are in fact a pathway to AGI. LLMs trained on more data with more efficient algorithms will make for more interesting tools built with LLMs, but I don’t see this technology as a foundation for AGI. LLMs don’t “reason” in any sense of the word that I understand and I think the ab…
If humans are basically evolved LLMs, which i think is likely; Reasoning will be an emergent property of LLMs within context with appropriate weights.
Almost everything interesting about AI so far has been unexpected emergent behavior, and huge gains through minor insights. While I don't doubt that the current architecture is likely to have a current ceiling below that of peak human intelligence in certain dimensions, it's already surpassed it in some, and there are still gains to be made in others through things like synthetic data. I also don't understand the cla…
Latest research shows emergent behavior is illusory. It doesn't preclude future emergence but currently models show 0 emergent behavior. To me the most interesting aspect of LLMs is the way that they reveal cognitive 0-days in humans. The human race needs patches to cognitive firmware to deal with predictive text... Which is a fascinating revelation to me. Sure it's backed up by psych analysis for decades but it's in…
Earlier quoted context omitted.
> problems that I can absolutely guarantee were not in its training set Can you share the strongest example?
Pretty much any coding problem in a unique or private codebase