Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Three Years from GPT-3 to Gemini 3

oneusefulthing.org

Post reply on HN
Print view View original

111–120 of 336 posts

Re: Three Years from GPT-3 to Gemini 3

#111

Post by worble »

timschmidt wrote: post #99
taurath wrote: post #95

Earlier quoted context omitted.

> You could trust the expert analysis of people in that field That’s the problem - the experts all promise stuff that can’t be easily replicated. The promises the experts send doesn’t match the model. The same request might succeed and might fail, and might fail in such a way that subsequent prompts might recover or might not.

That's how working with junior team members or open source project contributors goes too. Perhaps that's the big disconnect. Reviewing and integrating LLM contributions slotted right into my existing workflow on my open source projects. Not all of them work. They often need fixing, stylistic adjustments, or tweaking to fit a larger architectural goal. That is the norm for all contributions in my experience. So the LL…

Junior's grow into mids, and eventually into seniors. OSS contributor's eventually learn the codebase, you talk to them, you all get invested in the shared success of the project and sometimes you even become friends.

For me, personally, I just don't see the point of putting that same effort into a machine. It won't learn or grow from the corrections I make in that PR, so why bother? I might as well have written it myself and saved the merge review headache.

Maybe one day it'll reach perfect parity of what I could've written myself, but today isn't that day.

Re: Three Years from GPT-3 to Gemini 3

#113

Post by jaccola »

acters wrote: post #20
Humorist2290 wrote: post #12

> Again, we have moved past hallucinations and errors to more subtle, and often human-like, concerns. From my experience we just get both. The constant risk of some catastrophic hallucination buried in the output, in addition to more subtle, and pervasive, concerns. I haven't tried with Gemini 3 but when I prompted Claude to write a 20 page short story it couldn't even keep basic chronology and characters straight. I…

I feel like hallucinations have changed over time from factual errors randomly shoehorned into the middle of sentences to the LLMs confidently telling you they are right and even provide their own reasoning to back up their claims, which most of the time are references that don't exist.

I've noticed the new OpenAI models do self contradiction a lot more than I've ever noticed before! Things like:

- Aha, the error clearly lies in X, because ... so X is fine, the real error is in Y ... so Y is working perfectly. The smoking gun: Z ...

- While you can do A, in practice it is almost never a good idea because ... which is why it's always best to do A

Re: Three Years from GPT-3 to Gemini 3

#114

Post by p1necone »

MinimalAction wrote: post #11

> So is this a PhD-level intelligence? In some ways, yes, if you define a PhD level intelligence as doing the work of a competent grad student at a research university. But it also had some of the weaknesses of a grad student. As a current graduate student, I have seen similar comments in academia. My colleagues agree that a conversation with these recent models feels like chatting with an expert in their subfields.…

> if you define a PhD level intelligence as doing the work of a competent grad student at a research university. But it also had some of the weaknesses of a grad student.

With coding it feels more like working with two devs - one is a competent intermediate level dev, and one is a raving lunatic with zero critical thinking skills whatsoever. Problem is you only get one at a time and they're identical twins who pretend to be each other as a prank.

Re: Three Years from GPT-3 to Gemini 3

#115

Post by krainboltgreene »

brightball wrote: post #64
lm28469 wrote: post #53

Earlier quoted context omitted.

> Remember 54% of US adults read at or below the equivalent of a sixth-grade level. The sane conclusion would be to invest in education, not to dump hundreds of billions of llms, but ok

Investing in education is a trap because no matter how much money is pumped into the current model, it’s not making a difference. We need different models and then to invest in the successes, over and over again…forever.

It's so hilarious to look at 10k years of education history and be like "Nah, funding doesn't make a difference."

Incredible.

Re: Three Years from GPT-3 to Gemini 3

#116

Post by krainboltgreene »

wmeredith wrote: post #66
tehjoker wrote: post #61

Earlier quoted context omitted.

Not true, most people are not upper-middle class anti-tax wackos. They benefit from those people being taxed.

The electorate in the U.S. commonly votes against its own interests.

Pithy, but not true.

Re: Three Years from GPT-3 to Gemini 3

#117

Post by jwrallie »

remich wrote: post #81
randyrand wrote: post #39

For Caude Code, Antigrav, etc, do people really just let an LLM loose on their own personal system? I feel like these should run in a cloud enviroment, or at least on some specific machine where I don't care what it does.

Can't speak to Claude Code/Desktop, but any of the products that are VS Code forks have workspace restrictions on what folders they're allowed to access (for better and worse). Other products (like Warp terminal) that can give access to the whole filesystem come with pre-set strict deny/allow lists on what commands are allowed to be executed. It's possible to remove some of these restrictions in these tools, or to op…

Talking about VS Code itself (with Copilot), I have witnessed it accessing files referenced from within a project folder but stored outside of it without being given explicit permission to, so I am pretty sure it can leak information and potentially even wreak havoc outside its boundaries.

Re: Three Years from GPT-3 to Gemini 3

#118

Post by krainboltgreene »

acheron wrote: post #74
lm28469 wrote: post #53

Earlier quoted context omitted.

> Remember 54% of US adults read at or below the equivalent of a sixth-grade level. The sane conclusion would be to invest in education, not to dump hundreds of billions of llms, but ok

Education funding is highest in places that have the worst results. Try again.

Just flatly not true.

Re: Three Years from GPT-3 to Gemini 3

#119

Post by emodendroket »

acters wrote: post #20
Humorist2290 wrote: post #12

> Again, we have moved past hallucinations and errors to more subtle, and often human-like, concerns. From my experience we just get both. The constant risk of some catastrophic hallucination buried in the output, in addition to more subtle, and pervasive, concerns. I haven't tried with Gemini 3 but when I prompted Claude to write a 20 page short story it couldn't even keep basic chronology and characters straight. I…

I feel like hallucinations have changed over time from factual errors randomly shoehorned into the middle of sentences to the LLMs confidently telling you they are right and even provide their own reasoning to back up their claims, which most of the time are references that don't exist.

I like when they tell you they’ve personally confirmed a fact in a conversation or something.

Re: Three Years from GPT-3 to Gemini 3

#120

Post by AdieuToLogic »

PostOnce wrote: post #67
Herring wrote: post #48

Earlier quoted context omitted.

I think the point is we’re getting there. These models are growing up real fast. Remember 54% of US adults read at or below the equivalent of a sixth-grade level.

A question for the not-too-distant future: What use is an LLM in an illiterate society?

> What use is an LLM in an illiterate society?

The ability to feign literacy such that critical thought and ability to express same is not a prerequisite.

Post reply on HN