[flagged]
LLMs have hit the wall since ChatGPT came out in 2022?
Big time.
Three Years from GPT-3 to Gemini 3
101–110 of 336 posts
Re: Three Years from GPT-3 to Gemini 3
#102Earlier quoted context omitted.
Education is not just a funding issues. Policy choices, like making it impossible for students to fail which means they have no incentive to learn anything, can be more impactful.
But holy shit is it also a funding issue when teachers make nothing.
As far as I understand it, the problem isn’t that teachers are shit. Giving more money would bring in better teachers, but I don’t know that they’d be able to overcome the other obstacles
Re: Three Years from GPT-3 to Gemini 3
#103Earlier quoted context omitted.
Investing in education is a trap because no matter how much money is pumped into the current model, it’s not making a difference. We need different models and then to invest in the successes, over and over again…forever.
Because education alone in a vacuum won't fix the issues. Even if the current model was working, just continuing to invest money in it while ignoring other issues like early childhood nutrition, a good and healthy home environment, environmental impacts, etc. will just continue to fail people. Schooling alone isn't going to help the kid with a crappy home life, with poor parents who can't afford proper nutrition, and…
Exactly correct.
Re: Three Years from GPT-3 to Gemini 3
#104Earlier quoted context omitted.
In theory yeah, but in practice 54% will also vote against funding education. Catch-22.
In WA they always pass levies for education funding at local and state level however results are not there. Mississipi is doing better on reading, the biggest difference being that they use phonics approach to teaching how to read, which is proven to work, whereas WA uses whole language theory ( https://en.wikipedia.org/wiki/Whole_language ), which is a terrible idea I don't know how it got traction. So the gist of i…
First time hearing of whole language theory, and man, it sounds ridiculous. Sounds similar to the old theory that kids who aren't taught a language at all will simply speak perfect Hebrew.
Re: Three Years from GPT-3 to Gemini 3
#105Earlier quoted context omitted.
HN tends to be very weird around the topic of AI. No idea why opinions like this are downvoted without having to offer any criticism.
For one, I can't even understand this part: > I don't know if it represents research as a field would not be immune to advances in AI tech And then there's the opinion that for some reason we should 'value' manual labor over using AI, which seems rather disagreeable.
> For one, I can't even understand this part:
Let me explain. My belief was that research as a task is non-trivial and would have been relatively out of reach for AI. Given the advances, that doesn't seem to be true.
> And then there's the opinion that for some reason we should 'value' manual labor over using AI, which seems rather disagreeable.
Could you explain why? I'm specifically talking about research. Of course, I would value what a veteran in the field says higher than a probability machine.
Re: Three Years from GPT-3 to Gemini 3
#106Earlier quoted context omitted.
> Remember 54% of US adults read at or below the equivalent of a sixth-grade level. The sane conclusion would be to invest in education, not to dump hundreds of billions of llms, but ok
Education funding is highest in places that have the worst results. Try again.
New Mexico (where I live) is dead last in education out of all 50 states. They are currently advertising for elementary school teachers between 65-85K per year. Summers off. Nice pension. In this low cost of living state that is a very good salary, particularly the upper bands.
I don't think it's a money issue at this point.
Re: Three Years from GPT-3 to Gemini 3
#107Earlier quoted context omitted.
It's gotten more and more shippable, especially with the latest generation (Codex 5.1, Sonnet 4.5, now Opus 4.5). My metric is "wtfs per line", and it's been decreasing rapidly. My current preference is Codex 5.1 (Sonnet 4.5 as a close second, though it got really dumb today for "some reason"). It's been good to the point where I shipped multiple projects with it without a problem (with eg https://pine.town being one…
Have you tried Gemini 3 yet? I haven't done any coding with it, but on other tasks I've been impressed compared to gpt 5 and Sonnet 4.5.
It's very good but it feels kind of off-the-rails in comparison to Sonnet 4.5 - at least with Cursor it does strange things like putting its reasoning in comments that are about 15 lines long, deleting 90% of a file for no real reason (especially when context is reaching capacity) and making the same error that I just told it not to do.
Re: Three Years from GPT-3 to Gemini 3
#108I find Gemini 3 to be really good. I'm impressed. However, the responses still seem to be bounded by the existing literature and data. If asked to come up with new ideas to improve on existing results for some math problems, it tends to recite known results only. Maybe I didn't challenge it enough or present problems that have scope for new ideas?
Novel solutions require some combination of guided brute-force search over a knowledge-database/search-engine (NOT a search over the models weights and NOT using chain of thought), combined with adaptive goal creation and evaluation, and reflective contrast against internal "learned" knowledge. Not only that, but it also requires exploration of the lower-probability space, i.e. results lesser explored, otherwise you're always going to end up with the most common and likely answers. That means being able to quantify what is a "less-likely but more novel solution" to begin with, which is a problem in itself. Transformer architecture LLMs do not even come close to approaching AI in this way.
All the novel solutions humans create are a result of combining existing solutions (learned or researched in real-time), with subtle and lesser-explored avenues and variations that are yet to be tried, and then verifying the results and cementing that acquired knowledge for future application as a building block for more novel solutions, as well as building a memory of when and where they may next be applicable. Building up this tree, to eventually satisfy an end goal, and backtracking and reshaping that tree when a certain measure of confidence stray from successful goal evaluation is predicted.
This is clearly very computationally expensive. It is also very different to the statistical pattern repeaters we are currently using, especially considering that their entire premise works because the algorithm chooses the next most probable token which is a function of the frequency of which that token appears in the training data. In other words, the algorithm is designed explicitly NOT to yield novel results, but rather return the most likely result. Higher temperature results tend to reduce textual coherence rather than increase novelty, because token frequency is a literal proxy for textual coherence in coherent training samples, and there is no actual "understanding" happening, nor reflection of the probability results at this level.
I'm sure smart people have figured a lot of this out already - we have general theory and ideas to back this, look into AIXI for example, and I'm sure there is far newer work. But I imagine that any efficient solutions to this problem will permanently remain in the realm of being a computational and scaling nightmare. Plus adaptive goal creation and evaluation is a really really hard problem, especially if text is your only modality of "thinking". My guess would be that it would require the models to create simulations of physical systems in text-only format, to be able to evaluate them, which also means being able to translate vague descriptions of physical systems into text-based physics sims with the same degrees of freedom as the real world - or at least the target problem, and then also imagine ideal outcomes in that same translated system, and develop metrics of "progress" within this system, for the particular target goal. This is a requirement for the feedback loop of building the tree of exploration and validation. Very challenging. I think these big companies are going to chase their tails for the next 10 years trying to reach an ever elusive intelligence goal, before begrudgingly conceding that existing LLM architectures will not get them there.
Re: Three Years from GPT-3 to Gemini 3
#109Every time I see an article like this, it's always missing --- but is it any good, is it correct? They always show you the part that is impressive - "it walked the tricky tightrope of figuring out what might be an interesting topic and how to execute it with the data it had - one of the hardest things to teach." Then it goes on, "After a couple of vague commands (“build it out more, make it better”) I got a 14 page p…
Loads of AI chatter is the Murray Gell-Mann Amnesia Effect on steroids
Re: Three Years from GPT-3 to Gemini 3
#110It is interesting that most of our modes of interaction with AI is still just textboxes. The only big UX change in that the last three years has been the introduction of the Claude Code / OpenAI Codex tools. They feel amazing to use, like you're working with another independent mind. I am curious what the user interfaces of AI in the future will be, I think whoever can crack that will create immense value.
Unix CLI utilities have been all text for 50 years. Arguably that is why they are still relevant. Attempts to impose structured data on the paradigm like those in PowerShell have their adherents and can be powerful, but fail when the data doesn't fit the structure. We see similar tendency toward the most general interfaces in "operator mode" and similar the-AI-uses-the-mouse-and-keyboard schemes. It's entirely possib…
PowerShell is completely suitable. People are just used to bash and don’t feel the incentive to switch, especially with Windows becoming less relevant outside of desktop development.