Gemini AI • Hacker News Board

Re: Gemini AI

#311

Post by jeron » Wed, Dec 06, 2023, 4:52 PM UTC

milesward wrote: post #136

This demo is nuts: https://youtu.be/UIZAiXYceBI?si=8ELqSinKHdlGlNpX

It’s technically very impressive but the question is how many people will use the model in this way? Does Gemini support video streaming?

Re: Gemini AI

#312

Post by pknerd » Wed, Dec 06, 2023, 4:53 PM UTC

zaptheimpaler wrote: post #34

Bard still not available in Canada so i can't use it ¯\_(ツ)_/¯. Wonder why Google is the only one that can't release their model here.

Quite surprising to me. Bard has been available in Pakistan for a couple of months I believe.

Re: Gemini AI

#313

Post by beardog » Wed, Dec 06, 2023, 4:53 PM UTC

OfSanguineFire wrote: post #303

__void wrote: post #70

it's really amazing how in IT we always recycle the same ten names... in the last three years, "gemini" refers (at least) to: - gemini protocol, the smolnet companion (gemini://geminiprotocol.net/ - https://geminiprotocol.net/ ) - gemini somethingcoin somethingcrypto (I will never link it) - gemini google's ML/AI (here we are)

Google is so big a player that they don’t even need to check if the name has already been applied to a technology. As soon as they apply it to their product name, that will become the main association for the term. And as fond as some are of the Gemini protocol, it never got widely known outside of HN/Lobster circles.

They didn't even check if Go was taken: https://en.wikipedia.org/wiki/Go!_(programming_language)

Re: Gemini AI

#314

Post by ChatGTP » Wed, Dec 06, 2023, 4:53 PM UTC

It is over for OpenAI.

Re: Gemini AI

#315

Post by becausecurious » Wed, Dec 06, 2023, 4:54 PM UTC

Benchmarks: https://imgur.com/DWNQcaY ([Table 2 on Page 7](https://storage.googleapis.com/deepmind-media/gemini/gemini_...)) - Gemini Pro (the launched model) is worse than ChatGPT4, but a bit better than GPT3.5. All the examples are for Ultra (the actual state of the art model), which won't be available until 2024.

Re: Gemini AI

#316

Post by Palmik » Wed, Dec 06, 2023, 4:54 PM UTC

Curious that the metrics [1] of Gemini Ultra (not released yet?) vs GPT4 are for some tasks computed based on "CoT @ 32", for some "5-shot", for some "10-shot", for some "4-shot", for some "0-shot" -- that screams cherry-picking to me.

Not to mention that the methodology is different for Gemini Ultra and Gemini Pro for whatever reason (e.g. MMLU Ultra uses CoT @ 32 and Pro uses CoT @ 8).

[1] Table 2 here: https://storage.googleapis.com/deepmind-media/gemini/gemini_...

Re: Gemini AI

#317

Post by ChatGTP » Wed, Dec 06, 2023, 4:55 PM UTC

sockaddr wrote: post #229

SeanAnderson wrote: post #218

Not impressed with the Bard update so far. I just gave it a screenshot of yesterday's meals pulled from MyFitnessPal, told it to respond ONLY in JSON, and to calculate the macro nutrient profile of the screenshot. It flat out refused. It said, "I can't. I'm only an LLM" but the upload worked fine. I was expecting it to fail maybe on the JSON formatting, or maybe be slightly off on some of the macros, but outright ref…

> I just gave it a screenshot of yesterday's meals pulled from MyFitnessPal, told it to respond ONLY in JSON, and to calculate the macro nutrient profile of the screenshot > Not impressed This made me chuckle Just a bit ago this would have been science fiction

I think this goes for nearly all material things, as fantastic as they are, they're not magic. We get used to them very fast.

Re: Gemini AI

#318

Post by cchance » Wed, Dec 06, 2023, 4:56 PM UTC

jm547ster wrote: post #195

netcraft wrote: post #82

Lots of comments about it barely beating GPT-4 despite the latter being out for a while, but personally ill be happy to have another alternative, if nothing else for the competition. But I really dislike these pre-availability announcements - we have to speculate and take their benchmarks for gospel for a week, while they get a bunch of press for unproven claims. Back to the original point though, ill be happier havi…

Is it not already available via bard?

Only pro apparently which is not as good as ultra, ultras the one that actually beats got4 by a hair

Re: Gemini AI

#319

Post by xeckr » Wed, Dec 06, 2023, 4:56 PM UTC

I wish Google shortened the time between their announcements and making their models available.

Re: Gemini AI

#320

Post by Traubenfuchs » Wed, Dec 06, 2023, 4:56 PM UTC

m3at wrote: post #186

For others that were confused by the Gemini versions: the main one being discussed is Gemini Ultra (which is claimed to beat GPT-4). The one available through Bard is Gemini Pro . For the differences, looking at the technical report [1] on selected benchmarks, rounded score in %: Dataset | Gemini Ultra | Gemini Pro | GPT-4 MMLU | 90 | 79 | 87 BIG-Bench-Hard | 84 | 75 | 83 HellaSwag | 88 | 85 | 95 Natural2Code | 75 |…

formatted nicely:

  Dataset        | Gemini Ultra | Gemini Pro | GPT-4

  MMLU           | 90           | 79         | 87

  BIG-Bench-Hard | 84           | 75         | 83

  HellaSwag      | 88           | 85         | 95

  Natural2Code   | 75           | 70         | 74

  WMT23          | 74           | 72         | 74