Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini AI

deepmind.google

Post reply on HN
Print view View original

61–70 of 1001 posts

Re: Gemini AI

#61

Post by Veraticus »

So just a bunch of marketing fluff? I can use GPT4 literally right now and it’s apparently within a few percentage points of what Gemini Ultra can do… which has no release date as far as I can tell.

Would’ve loved something more substantive than a bunch of videos promising how revolutionary it is.

Re: Gemini AI

#63

Post by tkellogg »

wiz21c wrote: post #27

The improvement over ChatGPT are counted in (very) few percents. Does it mean they have entered a diminishing returns phase or is it that each percent is much harder to get compared to the previous ones ?

not sure, but you could also look at the inverse. e.g. a 90% to 95% improvement could also be interpreted as 10% failure to 5% failure, i.e. half the amount of failures, a very big improvement. It depends on a lot of things, but it's possible that this could feel like a very big improvement.

Re: Gemini AI

#64

Post by epups »

Benchmark results look awesome, but so does every new open source release these days - it is quite straightforward to make sure you do well in benchmarks if that is your goal. I hope Google cracked it and this is more than PR.

Re: Gemini AI

#65

Post by kubb »

obiefernandez wrote: post #7

> For Gemini Ultra, we’re currently completing extensive trust and safety checks, including red-teaming by trusted external parties, and further refining the model using fine-tuning and reinforcement learning from human feedback (RLHF) before making it broadly available. > As part of this process, we’ll make Gemini Ultra available to select customers, developers, partners and safety and responsibility experts for ear…

I’m afraid it won’t be nearly as good as GPT4, because of how lax Open AI can be with intellectual property. Google will not be able to train their model on Libgen or Opensubtitles, because they can’t afford the risk.

Re: Gemini AI

#66

Post by logicchains »

kolinko wrote: post #41
obiefernandez wrote: post #7

> For Gemini Ultra, we’re currently completing extensive trust and safety checks, including red-teaming by trusted external parties, and further refining the model using fine-tuning and reinforcement learning from human feedback (RLHF) before making it broadly available. > As part of this process, we’ll make Gemini Ultra available to select customers, developers, partners and safety and responsibility experts for ear…

It won't be available to regular devs until Q2 next year probably (January for selected partners). So they are roughly a year behind OpenAI - and that is assuming their model is not overtrained to just pass the tests slightly better than GPT4

>So they are roughly a year behind OpenAI

Possibly by that time GPT5 will already be out.

Re: Gemini AI

#67

Post by paulpan »

dcchambers wrote: post #13

This marketing page feels very apple-like (and I mean that in a good way). If the benchmarks are any indication, Gemini seems legit, excited to see what it can do.

Well they sure copied Apple's "Pro" and "Ultra" branding. I'm fully expecting a "Gemini Max" version in the near future!

Re: Gemini AI

#69

Post by technics256 »

rolisz wrote: post #44

What is up with that eval @32? Am I reading it correctly that they are generating 32 responses and taking majority? Who will use the API like that? That feels like such a fake way to improve metrics

This also jumped out at me. It also seems that they are selectively choosing different promoting strategies too, one lists "CoT@32".

Makes it seem like they really needed to get creative to have it beat GPT4. Not a good sign imho

Re: Gemini AI

#70

Post by __void »

it's really amazing how in IT we always recycle the same ten names... in the last three years, "gemini" refers (at least) to:

- gemini protocol, the smolnet companion (gemini://geminiprotocol.net/ - https://geminiprotocol.net/)

- gemini somethingcoin somethingcrypto (I will never link it)

- gemini google's ML/AI (here we are)

Post reply on HN