Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini AI

deepmind.google

Post reply on HN
Print view View original

421–430 of 1001 posts

Re: Gemini AI

#421

Post by pcshah1996 »

albertzeyer wrote: post #42

So, better than GPT4 according to the benchmarks? Looks very interesting. Technical paper: https://goo.gle/GeminiPaper Some details: - 32k context length - efficient attention mechanisms (for e.g. multi-query attention (Shazeer, 2019)) - audio input via Universal Speech Model (USM) (Zhang et al., 2023) features - no audio output? (Figure 2) - visual encoding of Gemini models is inspired by our own foundational work o…

Some discussion on twitter about misleading evaluation: https://twitter.com/a_a_cabrera/status/1732454328307511807

(nitter: https://nitter.net/a_a_cabrera/status/1732454328307511807#m)

Re: Gemini AI

#422

Post by machiaweliczny »

paradite wrote: post #344

To me it doesn't look impressive at all. In this video: https://www.youtube.com/watch?v=LvGmVmHv69s , Google talked about solving a competitive programming problem using dynamic programming. But DP is considered only an intermediate level technique in National Olympiad in Informatics/USACO level competitions, which are targeted at secondary school students. For more advanced contests the tough questions usually requi…

Can you say what are those?

Re: Gemini AI

#423

Post by yalogin »

This is great. I always thought OpenAI's dominance/prominence will be short lived and it will see a lot of competition.

Does anyone know how they "feed" the input to the AI in the demo here? Looks like there is an API to ask questions. Is that what they say will be available Dec 13?

Re: Gemini AI

#424

Post by rvnx »

brrrrrm wrote: post #347
milesward wrote: post #136

This demo is nuts: https://youtu.be/UIZAiXYceBI?si=8ELqSinKHdlGlNpX

I once met a Google PM whose job was to manage “Easter eggs” in the Google home assistant. I wonder how many engineers effectively “hard coded” features into this demo. (“What the quack” seems like one)

Probably not "hard coded" in the literal way, but instead, if the model is using RLHF, they could thumbs up the answer.

Re: Gemini AI

#426

Post by kolinko »

EZ-E wrote: post #358
milesward wrote: post #136

This demo is nuts: https://youtu.be/UIZAiXYceBI?si=8ELqSinKHdlGlNpX

Out of curiosity I fed ChatGPT 4 a few of the challenges through a photo (unclear if Gemini takes live video feed as input but GPT does not afaik) and it did pretty well. It was able to tell a duck was being drawn at an earlier stage before Gemini did. Like Gemini it was able to tell where the duck should go - to the left path to the swan. Because and I quote "because ducks and swans are both waterfowl, so the swan d…

I showed the choice between a bear and a duck to GPT4, and it told me that it depends on whether the duck wants to go to a peaceful place, or wants to face a challenge :D

Re: Gemini AI

#427

Post by yalogin »

Deepmind is a great name, Google should over index on that. Bard on the other hand is an unfortunate name, may be they should have just called it deepmind instead.

Re: Gemini AI

#428

Post by kolinko »

thunkshift1 wrote: post #333
milesward wrote: post #136

This demo is nuts: https://youtu.be/UIZAiXYceBI?si=8ELqSinKHdlGlNpX

They should do this live instead of a pre recorded video for it to be more awe inspiring. Googles hype machine cannot be trusted.

+1. Or at least with no cuts, and more examples.

This is obviously geared towards non-technical/marketing people that will catch on to the hype. Or towards wall street ;)

Re: Gemini AI

#429

Post by brucethemoose2 »

aeternum wrote: post #398
brucethemoose2 wrote: post #355

Earlier quoted context omitted.

Everyone in the open source LLM community know the standard benchmarks are all but worthless. Cheating seems to be rampant, and by cheating I mean training on test questions + answers. Sometimes intentional, sometimes accidental. There are some good papers on checking for contamination, but no one is even bothering to use the compute to do so. As a random example, the top LLM on the open llm leaderboard right now has…

The issue is you really need to create a brand new benchmark with each release. Users will invariably test variants of existing benchmarks/questions and thus they will be included in the next training run. Academia isn't used to using novel benchmark questions every few months so will have trouble adapting.

Then its not really a benchmark? Model trainers and researchers are not continuously testing, they dump something then move on.

The answer is standard "secret" closed source tests, performed in a controlled environment.

I know, I don't like the sound of it either, but in this case I think closed source + a single overseeing entity is the best solution, by far. Facebook already made something like this, but they only went halfway (publishing the questions while keeping the answers secret).

Re: Gemini AI

#430

Post by yieldcrv »

albertzeyer wrote: post #42

So, better than GPT4 according to the benchmarks? Looks very interesting. Technical paper: https://goo.gle/GeminiPaper Some details: - 32k context length - efficient attention mechanisms (for e.g. multi-query attention (Shazeer, 2019)) - audio input via Universal Speech Model (USM) (Zhang et al., 2023) features - no audio output? (Figure 2) - visual encoding of Gemini models is inspired by our own foundational work o…

I wish Google’s UI would have the same chat interface as ChatGPT or even command line ones I’ve encountered
Post reply on HN