101–110 of 489 posts
I believe the next step will be robotics and getting A.I. to interact with the physical world at human fidelity. Maybe we can finally have a Rosie from the Jetsons.
just what I want, a mobile Alexa that spews ads and spies on me 24/7
I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.
From where I look at it, LLMs are flawed in many ways, and people who see progress as inevitable do not have a mental model of the foundation of those systems to be able to extrapolate. Also, people do not know any other forms of AI or have though hard about this stuff on their own.
The most problematic things are:
1) LLMs are probabilistic and a continuous function, forced by gradient descent. (Just having a "temperature" seems so crazy to me.) We need to merge symbolic and discrete forms of AI. Hallucinations are the elephant in the room. They should not be put under the rug. They should just not be there in the first place! If we try to cover them with a layer of varnish, the cost will be very large in the long run (it already is: step-by-step reasoning, mixture of experts, RAG, etc. are all varnish, in my opinion)
2) Even if generalization seems ok, I think it is still really far from where it should be, since humans need exponentially less data and generalize to concepts way more abstract than AI systems. This is related to HASA and ISA relations. Current AI systems do not have any of that. Hierarchy is supposed to be the depth of the network, but it is a guess at best.
3) We are just putting layer upon layer of complexity instead of simplifying. It is the victory of the complexifiers and it is motivated by the rush to win the race. However, I am not so sure that, even if the goal seems so close now, we are going to reach it. What are we gonna do? Keep adding another order of magnitude of compute on top of the last one to move forward? That's the bubble that I see. I think that that is not solving AI at all. And I'm almost sure that a much better way of doing AI is possible, but we have fallen into a bad attractor just because Ilya was very determined.
We need new models, way simpler, symbolic and continuous at the same time (i.e. symbolic that simulate continuous), non-gradient descent learning (just store stuff like a database), HAS-A hierarchies to attend to different levels of structure, IS-A taxonomies as a way to generalize deeply, etc, etc, etc.
Even if we make progress by brute forcing it with resources, there is so much work to simplify and find new ideas that I still don't understand why people are so optimistic.
I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.
I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.
In the end leaving the world changed, but not as meaningfully or positively as promised.
I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.
I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.
Reminds me of crypto/Web-3.0 hype. Lots of bluster about changing economic systems, offering people freedom and wealth, only to mostly be scams, and coming with too serious inherent drawbacks/costs to solve many of the big problems it promises to solve. In the end leaving the world changed, but not as meaningfully or positively as promised.
I think that people doing work in many professions with these offline tools alone could more than double their productivity compared to their productivity two years ago. Furthermore if the usage was shared in order to lower idle time, such as 20 machines for 100 workers, the initial capital outlay is even lower.
Perhaps investors will not see the returns they expect, but it is difficult to image how even the current state of AI doesn't vastly change the economy. There could be significant business failures among cloud providers and attempts to rapidly increase the cost of admission to closed models, but there's essentially no possibility of productivity regressing to a pre-AI levels.
I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.
“The ability to speak does not make you intelligent.”
> GPUs that have a 1-3 year lifespan In 10 years GPUs will have a lifespan for 5-7 years. The rate of improvement on this front has been slowing down faster then CPU.
There was a video on YT (gamers nexus I think) and an excel sheet comparing jumps in performance gains between each new nvidia generation. They are becoming smaller and smaller, probably now driven by AI boom where most of the silicon is used for data centers. Regardless of that, I have a feeling we are approaching the ceiling of chip performance. Just comparing PS3 and PS4 and then PS4 with PS5, performance jump is…
What is interesting is that it seems like the ever larger sums of money sloshing around are resulting in bigger, faster hype cycles. We are already seeing some companies face issues after blowback from adopting AI too fast.