211–220 of 266 posts
To the people who claim that we’re running out of data, I would just say: the world is largely undigitized. The Internet digitized a bunch of words but not even a tiny fraction of all that humans express every day. Same goes for sound in general. CCTV captures a lot of images, far more than social media, but it is poorly processed and also just a fraction of the photons bouncing off objects on earth. The data part of…
I will worry when I see Startups competing on products with companies 10x, 100x, or 1000x times their size. Like a small team producing a Photoshop replacement. So far I haven't seen anything like that. Big companies don't seem to be launching new products faster either, or fixing some of their products that have been broken for a long time (MS teams...) AI obviously makes some easy things much faster, maybe helps wi…
I think the real turning point is when there isn’t the need for something like photoshop. Creatives that I speak to yearn for the day when they can stop paying the adobe tax.
I could imagine a world where a small team + AI creates an open source tool that is better than current day Photoshop. However if that small team has that power, so does adobe, and what we perceive as "good" or "high quality" will shift.
Earlier quoted context omitted.
Indeed you can't be sure. But on the other hand a bunch of the commentariat has been claiming (with no evidence) that we're at the midpoint of the sigmoid for the last three years. They were wrong. And then you had the AI frontier lab insiders who predicted an accelerating pace of progress for the last three years. They were right. Now, the frontier labs rarely (never?) provide evidence either, but they do have about…
> Indeed you can't be sure. But on the other hand a bunch of the commentariat has been claiming (with no evidence) that we're at the midpoint of the sigmoid for the last three years. I haven’t followed things closely, but I’ve seen more statements that we may be near the midpoint of a sigmoid than that we are at it. > Thy were wrong. And then you had the AI frontier lab insiders who predicted an accelerating pace of…
A year ago, would you have thought that a pure LLM with no tools could get a gold medal level score in the 2025 IMO finals? I would have thought that was crazy talk. Given the rates of progress over the previous few years, maybe 2027 would have been a realistic target.
Predicting exponential growth is exceptionally difficult. Asymptotes are ordinary, and they often are not obvious until circumstances make them appear (in other words, they are commonly unpredictable).
(I do agree with the author regarding the potential of LLM's remaining underestimated by much of the public, however I cannot hang around such abysmal reasoning.)
I am constantly astonished that articles like this even pass the smell test. It is not rational to predict exponential growth just because you've seen exponential growth before! Incidentally, that is not what people did during COVID, they predicted exponential growth for reasons . Specific, articulable reasons, that consisted of more than just "look, like go up. line go up more?". Incidentally, the benchmarks quoted…
The claim is "At time t0, an AI can solve a task that would take a human 2 minutes. At time t0+dt, they can solve 4-minutes tasks. At time t0+2dt, it's 8 minutes" and so on.
I still find these claims extremely dubious, just wanted to clarify.
I am flabbergasted by the naivety around predicting the future. While we have hints and suggestions, our predictions are best expressed as ranges of possibilities with varying weights. The hyperbolic among us like to pretend that predictions come in the form of precise lines of predetermined direction and curve; how foolish! Predicting exponential growth is exceptionally difficult. Asymptotes are ordinary, and they o…
Earlier quoted context omitted.
I am extremely confident that AGI, if it is achievable at all (which is a different argument and one I'm not getting into right now), requires a world model / fact model / whatever terminology you prefer, and is therefore not achievable by models that simply chain words together without having any kind of understanding baked into the model. In other words, LLMs cannot lead to AGI.
Agreed, it surely does require a world-model. I disagree that generic LLMs plus CoT/reasoning/tool calling (ie the current stack) cannot in principle implement a world model. I believe LLMs are doing some sort of world modeling and likely are mostly lacking a medium-/long-term memory system in which to store it. (I wouldn’t be surprised if one or two more architectural overhauls end up occurring before AGI, I also wo…
Earlier quoted context omitted.
My point is that the limits of LLMs will be hit long before we they start to take on human capabilities. The problem isn’t that exponential growth is hard to visualise. The problem is that LLMs, as advanced and useful a technique as it is, isn’t suited for AGI and thus will never get us even remotely to the stage of AGI. The human like capabilities are really just smoke and mirrors. It’s like when people anthropomorp…
There is no particular reason why AI has to stick to language models though. Indeed if you want human like thinking you pretty much have to go beyond language as we do other stuff too if you see what I mean. A recent example: "Google DeepMind unveils its first “thinking” robotics AI" https://arstechnica.com/google/2025/09/google-deepmind-unvei...
From the paper [0]:
> Gemini Robotics 1.5 model family. Both Gemini Robotics 1.5 and Gemini Robotics-ER 1.5 inherit Gemini’s multimodal world knowledge.
> Agentic System Architecture. The full agentic system consists of an orchestrator and an action model that are implemented by the VLM and the VLA, respectively:
> • Orchestrator: The orchestrator processes user input and environmental feedback and controls the overall task flow. It breaks complex tasks into simpler steps that can be executed by the VLA, and it performs success detection to decide when to switch to the next step. To accomplish a user-specified task, it can leverage digital tools to access external information or perform additional reasoning steps. We use GR-ER 1.5 as the orchestrator.
> • Action model: The action model translates instructions issued by the orchestrator into low-level robot actions. It is made available to the orchestrator as a specialized tool and receives instructions via open-vocabulary natural language. The action model is implemented by the GR 1.5 model.
AI researchers have been trying to discover workable architectures for decades, and LLMs are the best we've got so far. There is no reason to believe that this exponential growth on test scores would or even could transfer to other architectures. In fact, the core advantage that LLMs have here is that they can be trained on vast, vast amounts of text scraped from the internet and taken from pirated books. Other model architectures that don't involve next-token-prediction cannot be trained using that same bottomless data source, and trying to learn quickly from real-world experiences is still a problem we haven't solved.
[0] https://storage.googleapis.com/deepmind-media/gemini-robotic...
> People notice that while AI can now write programs, design websites, etc, it still often makes mistakes or goes in a wrong direction, and then they somehow jump to the conclusion that AI will never be able to do these tasks at human levels, or will only have a minor impact. When just a few years ago, having AI do these things was complete science fiction! Both things can be true, since they're orthogonal. Having AI…
> We would have 1 trillion transistor cpus following Moore's "exponential curve" Cerebras wafer scale chip has 4 trillion transistors. https://www.cerebras.ai/chip
It is also, notably, _wafer-scale_. The metric is not just "number of transistors", but in fact "number of transistors per cm2"
> Given consistent trends of exponential performance improvements over many years and across many industries, it would be extremely surprising if these improvements suddenly stopped. I'm sure people were saying that about commercial airline speeds in the 1970's too. But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors. With LLM's at the moment, the limiting fac…
Also elegantly formulated by: https://idlewords.com/talks/web_design_first_100_years.htm