Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3

blog.google

Post reply on HN
Print view View original

921–930 of 1001 posts

Re: Gemini 3

#921

Post by jacquesm »

red75prime wrote: post #877
nomilk wrote: post #837

Earlier quoted context omitted.

Terrence Tao claims [0] contributions by the public are counter -productive since the energy required to check a contribution outweighs its benefit: > (for) most research projects, it would not help to have input from the general public. In fact, it would just be time-consuming, because error checking Since frontier LLMs make clumsy mistakes, they may fall into this category of 'error-prone' mathematician whose net c…

Unlike general public the models can be trained. I mean if you train a member of general public, you've got a specialist, who is no longer a member of general public.

Unlike the general public though, these models have advanced dementia when it comes to learning from corrections, even within a single session. They keep regressing and I haven't found a way to stop that yet.

What boggles the mind: we have gone for so long to try to strive for correctness and suddenly being right 70% of the time and wrong the remaining 30% is fine. The parallel with self driving is pretty strong here: solving 70% of the cases is easy, the remaining 30% are hard or maybe even impossible. Statistically speaking these models do better than most humans, most of the time. But they do not do better than all humans, and they can't do it all of the time and when they get it wrong they make such tremendously basic mistakes that you have to wonder how they manage to get things right.

Maybe it's true that with an ever increasing model size and more and more (proprietary, the public sources are exhausted by now so private data is the frontier where model owners can still gain an edge) we will reach a point where the models will be right 98% of the time or more but what would be the killer feature for me is an indication of the confidence level of the output. Because no matter whether junk or pearls it all looks the same and that is more dangerous than having nothing at all.

Re: Gemini 3

#922

Post by jacquesm »

junon wrote: post #435
JBiserkov wrote: post #402

Earlier quoted context omitted.

The requested prompt does not exist or you do not have access. If you believe the request is correct, make sure you have first allowed AI Studio access to your Google Drive, and then ask the owner to share the prompt with you.

I thought this was a joke at first. It actually needs drive access to run someone else's prompt. Wild.

Not a chance I'll ever click 'ok'. I'd love to be able to opt-out of anything AI related near my google environment.

Re: Gemini 3

#923

Post by tim333 »

Hassabis interview on Gemini 3, with Hard Fork (nyt podcast), also Josh Woodward https://youtu.be/rq-2i1blAlU?t=428 Some points -

Good at vibe coding 10:30 - step change where it's actually useful

AGI still 5-10 years. Needs reasoning, memory, world models.

Is it a bubble? - Partly 22:00

What's fun to do with Gemini to show the relatives? Suggested taking a selfie with the app and having it edit. 24:00 (I tried and said make me younger. Worked pretty well.)

Also interesting - apparently they are doing an agent to go through your email inbox and propose replies automatically 4:00. I could see that getting some use.

Re: Gemini 3

#924

Post by slightwinder »

tonyhart7 wrote: post #686
aussieguy1234 wrote: post #652

Earlier quoted context omitted.

Not sure what you mean here, but the only real jobs at risk from AI right now are middle/upper management. Not a single engineer has ever been laid off because of AI. Any company claiming this is the case is trying to cover up bad decisions. "Were automating with AI" sounds better to investors than "We over hired and now need to downsize" or "We made some bad market bets, now need to free up cash flow"

"Not a single engineer has ever been laid off because of AI." are you insane??? big tech literally make one of the most biggest layoff for the past few months

But not because of AI, they only use that as pretext for normal layoffs. Sometimes they also use it to hire cheaper workers fresh from school or a cheaper country, so just replacing expensive seniors.

Re: Gemini 3

#925

Post by rkozik1989 »

osn9363739 wrote: post #781
ogig wrote: post #711

I just gave it a short description of a small game I had an idea for. It was 7 sentences. It pretty much nailed a working prototype, using React, clean css, Typescript and state management. It event implemented a Gemini query using the API for strategic analysis given a game state. I'm more than impressed, I'm terrified. Seriously thinking of a career change.

Can you share the code?

No because this story didn't happen.

Re: Gemini 3

#926

Post by rkozik1989 »

spookie wrote: post #133
aliljet wrote: post #82

Understanding precisely why Gemini 3 isn't front of the pack on SWE Bench is really what I was hoping to understand here. Especially for a blog post targeted at software developers...

Does anyone trust benchmarks at this point? Genuine question. Isn't the scientific consensus that they are broken and poor evaluation tools?

Honestly, I am inclined to think a lot of the people who are wowed by benchmarks and simple tech demos probably aren't doing very much at their day job and if they're either working on simple codebases or ones that don't have very many users(more users == more bugs found). When you throw these models at complex software projects like SOAs, big object-oriented codebases, etc. their output can be totally unusable.

Re: Gemini 3

#927

Post by ogig »

osn9363739 wrote: post #781
ogig wrote: post #711

I just gave it a short description of a small game I had an idea for. It was 7 sentences. It pretty much nailed a working prototype, using React, clean css, Typescript and state management. It event implemented a Gemini query using the API for strategic analysis given a game state. I'm more than impressed, I'm terrified. Seriously thinking of a career change.

Can you share the code?

https://ai.studio/apps/drive/1E-aYovHHoY8jrF6bsl_AZ8VszIN66N...

The initial prompt was, in case people doesn't want to log in:

Make a turn based chess like game. Instead of normal chess board use an hexagonal grid. Make the board diagonal shaped. Instead of traditional chess pieces we are going to use spaceship designs. Each spaceship has unique abilities that influence the board or their own skill. For 2 players, turn based. Show me what you got.

Re: Gemini 3

#928

Post by nbardy »

stephc_int13 wrote: post #305
tylervigen wrote: post #164

I am personally impressed by the continued improvement in ARC-AGI-2, where Gemini 3 got 31.1% (vs ChatGPT 5.1's 17.6%). To me this is the kind of problem that does not lend itself well to LLMs - many of the puzzles test the kind of thing that humans intuit because of millions of years of evolution, but these concepts do not necessarily appear in written form (or when they do, it's not clear how they connect to specif…

What I would do if I was in the position of a large company in this space is to arrange an internal team to create an ARC replica, covering very similar puzzles and use that as part of the training. Ultimately, most benchmarks can be gamed and their real utility is thus short-lived. But I think this is also fair to use any means to beat it.

This isn’t gaming the benchmark though. If training on similar data generalizes that’s called learning. Training on the exact set is memorization.

There is for a fact teams creating puzzles to RL against as training environments. As it’s beneficial to RL training and in particular compute efficient if you schedule the environment difficulty throughout training. There was a great recent paper on this. Creating environment data that generalizes outside the environment is a challenging engineering task and super valuable whether it looks like AGC AGI or not.

Also ARC AGI is general enough that if you create similar data you’re just creating generic visual puzzle data. Should all visual puzzle data be off limits ?

Re: Gemini 3

#929

Post by column »

tngranados wrote: post #920
column wrote: post #863

Earlier quoted context omitted.

"[a photoshopped picture of a dog with 5 legs]...please count the legs" Meanwhile you could benchmark for something actually useful. If you're about to say "But that means it won't work for my use case of identifying a person on a live feed" or whatever, then why don't you test that? I really don't understand the kick people get of successfully tricking LLMs on non productive task with no real world application. Just…

The point of benchmarking that is checking for hallucinations and overfitting. Does the model actually check the picture to count the legs or does it just see it's a dog and answer four because it knows dogs usually has four legs? It's a perfectly valid benchmark and very telling.

Very telling of what?

Re: Gemini 3

#930

Post by hbbio »

tkgally wrote: post #885
hbbio wrote: post #847

Earlier quoted context omitted.

You should add ChatGPT. I tried the first one and 5 Pro gives this: https://imgur.com/a/EhYroCE

Thanks for the suggestion. I’m not sure why I didn’t include an OpenAI model in my first round. Here’s the updated page with GPT-5.1 results added: https://gally.net/temp/20251107pelican-alternatives/index.ht... As your example shows, GPT-5 Pro would probably be better that GPT-5.1, but the tokens are over ten times more expensive and I didn’t feel like paying for them.

Thanks for adding!

Extending beyond the pelican is very interesting, especially until your page gets enough recognition to be "optimized" by the AI companies.

It seems both Gemini 3 and latest ChatGPTs get a deep understanding of the representation of SVGs that seems a difficult task. I would be incapable of writing a SVG without visualizing the result and a graphical feedback loop.

PS: Would be fun to add "animated" in the short prompt since some models think of animation by themselves. Tried manually with 5 Pro (using the subscription), and in a sense it's worse than the static image. To start, there's a error: https://bafybeie7gazq46mbztab2etpln7sqe5is6et2ojheuorjpvrr2u...

Post reply on HN