Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.0: our new AI model for the agentic era

blog.google

Post reply on HN
Print view View original

111–120 of 512 posts

Re: Gemini 2.0: our new AI model for the agentic era

#111

Post by kwindla »

dandiep wrote: post #95

Gemini multimodal live docs here: https://cloud.google.com/vertex-ai/generative-ai/docs/model-... A little thin... Also no pricing is live yet. OpenAI's audio inputs/outputs are too expensive to really put in production, so hopefully Gemini will be cheaper. (Not to mention, OAI's doesn't follow instructions very well.)

The Multimodal Live API is free while the model/API is in preview. My guess is that they will be pretty aggressive with pricing when it's in GA, given the 1.5 Flash multimodal pricing.

If you're interested in this stuff, here's a full chat app for the new Gemini 2 API's with text, audio, image, camera video and screen video. This shows how to use both the WebSocket API and to route through WebRTC infrastructure.

https://github.com/pipecat-ai/gemini-multimodal-live-demo

Re: Gemini 2.0: our new AI model for the agentic era

#112

Post by mupuff1234 »

dagmx wrote: post #107
mupuff1234 wrote: post #103

Earlier quoted context omitted.

I think it's a more likely assumption that on device performance will trail off device models by a significant margin for at least the next few years - of course if magically you can make it work locally with the same level of performance it would be better. Plus a lot of the "agentic" stuff is interaction with the outside world, connectivity is a must regardless.

My point is that you do NOT need the same level of performance. You need an adequate level of performance that the cost to get more performance isn’t worth it to most people.

And my point is that it's way too early to try to optimize for running locally, if performance really stabilizes and comes to a halt (which may likely happen) then it makes more sense to optimize.

Plus once you start with on device features you start limiting your development speed and flexibility.

Re: Gemini 2.0: our new AI model for the agentic era

#113

Post by simonw »

I released a new llm-gemini plugin with support for the Gemini 2.0 Flash model, here's how to use that in the terminal:

    llm install -U llm-gemini
    llm -m gemini-2.0-flash-exp 'prompt goes here'
LLM installation: https://llm.datasette.io/en/stable/setup.html

Worth noting that the Gemini models have the ability to write and then execute Python code. I tried that like this:

    llm -m gemini-2.0-flash-exp -o code_execution 1 \
      'write and execute python to generate a 80x40 ascii art fractal'
Here's the result: https://gist.github.com/simonw/0d8225d62e8d87ce843fde471d143...

It can't make outbound network calls though, so this fails:

    llm -m gemini-2.0-flash-exp  -o code_execution 1 \
      'write python code to retrieve https://simonwillison.net/ and use a regex to extract the title, run that code'
Amusingly Gemini itself doesn't know that it can't make network calls, so it tries several different approaches before giving up: https://gist.github.com/simonw/2ccfdc68290b5ced24e5e0909563c...

The new model seems very good at vision:

    llm -m gemini-2.0-flash-exp describe -a https://static.simonwillison.net/static/2024/pelicans.jpg
I got back a solid description, see here: https://gist.github.com/simonw/32172b6f8bcf8e55e489f10979f8f...

Re: Gemini 2.0: our new AI model for the agentic era

#114

Post by nightski »

ramoz wrote: post #105
nightski wrote: post #79

Anyone else annoyed how the ML/AI community just adopted the word "reasoning" when it seems like it is being used very out of context when looking at what the model actually does?

These models take an instruction, along with any contextual information, and are trained to produce valid output. That production of output is a form of reasoning via _some_ type of logical processing. No? Maybe better to say computational reasoning. That’s a mouthful.

Static computation is not reasoning (these models are not building up an argument from premises, they are merely finding statistically likely completions). Computational thinking/reasoning would be breaking down a problem into an algorithmic steps. The model is doing neither. I wouldn't confuse the fact that it can break it into steps if you ask it, because again that is just regurgitation. It's not going through that process without your prompt. That is not part of its process to arrive at an answer.

Re: Gemini 2.0: our new AI model for the agentic era

#115

Post by griomnib »

kridsdale1 wrote: post #71
YetAnotherNick wrote: post #68

Earlier quoted context omitted.

If the model weights is not open, you can't run it on device anyways.

The Pixel 9 runs many small proprietary Gemini models on the internal TPU.

And yet these new models still haven’t reached feature parity with Google Assistant, which can turn my flashlight on, but with all the power of burning down a rainforest, Gemini still cannot interact with my actual phone.

Re: Gemini 2.0: our new AI model for the agentic era

#116

Post by bloomingkales »

panabee wrote: post #86
crowcroft wrote: post #15

Big companies can be slow to pivot, and Google has been famously bad at getting people aligned and driving in one direction. But, once they do get moving in the right direction the can achieve things that smaller companies can't. Google has an insane amount of talent in this space, and seems to be getting the right results from that now. Remains to be seen how well they will be able to productize and market, but hard…

With many research areas converging to comparable levels, the most critical piece is arguably vertical integration and forgoing the Nvidia tax. They haven't wielded this advantage as powerfully as possible, but changes here could signal how committed they are to slaying the search cash cow. Nadella deservedly earned acclaim for transitioning Microsoft from the Windows era to cloud and mobile. It will be far more impr…

They have to not get blind sided by Sora, while at the same time fighting the cloud war against MS/Amazon.

Weirdly Google is THE AI play. If AI is not set to change everything and truly is a hype cycle, then Google stock withstands and grows. If AI is the real deal, then Google still withstands due to how much bigger the pie will get.

Re: Gemini 2.0: our new AI model for the agentic era

#117

Post by rybosome »

losvedir wrote: post #74

This naming is confusing... Anyway, I'm glad that this Google release is actually available right away! I pay for Gemini Advanced and I see "Gemini Flash 2.0" as an option in the model selector. I've been going through Advent of Code this year, and testing each problem with each model (GPT-4o, o1, o1 Pro, Claude Sonnet, Opus, Gemini Pro 1.5). Gemini has done decent, but is probably the weakest of the bunch. It failed…

I can't comment on why the model gave you that code, but I can tell you why it was not correct.

`queue.remove(0)` gives you an `int[]`, which is also what you were assigning to `current`. So logically it's a single element, not an iterable. If you had wanted to iterate over each item in the array, it would need to be:

``` for (int[] current : queue) { for (int c : current) { // ...do stuff... } } ```

Alternatively, if you wanted to iterate over each element in the queue and treat the int array as a single element, the revised solution is the correct one.

Re: Gemini 2.0: our new AI model for the agentic era

#118

Post by griomnib »

mupuff1234 wrote: post #28
dagmx wrote: post #14

Earlier quoted context omitted.

Regarding TPU’s, sure for the stuff that’s running on the cloud. However their on device TPUs lag behind the competition and Google still seem to struggle to move significant parts of Gemini to run on device as a result. Of course, Gemini is provided as a subscription service as well so perhaps they’re not incentivized to move things locally. I am curious if they’ll introduce something like Apple’s private cloud comp…

Majority of people want better performance, running locally is just a nice to have feature.

Latency is a huge factor in performance, and local models often have a huge edge. Especially on mobile devices that could be offline entirely.

Re: Gemini 2.0: our new AI model for the agentic era

#119

Post by vineyardmike »

dagmx wrote: post #53
mupuff1234 wrote: post #28

Earlier quoted context omitted.

Majority of people want better performance, running locally is just a nice to have feature.

They’ll care though when they have to pay for it, or when they’re in an area with poor reception.

Poor reception is rapidly becoming a non-issue for most of the developed world. I can’t think of the last time I had poor reception (in America) and wasn’t on an airplane.

As the global human population increasingly urbanizes, it’ll become increasingly easy to blanket it with cell towers. Poor(er) regions of the world will increase reception more slowly, but they’re also more likely to have devices that don’t support on-device models.

Also, Gemini Flash is basically positioned as a free model, (nearly) free API, free in GUI, free in Search Results, Free in a variety of Google products, etc. No one will be paying for it.

Re: Gemini 2.0: our new AI model for the agentic era

#120

Post by falcor84 »

SV_BubbleTime wrote: post #49
siliconc0w wrote: post #44

What's everyone's favorite LLM leaderboard? Gemini 2 seems to be edging out 4o on chatbot arena( https://lmarena.ai/?leaderboard )

AI benchmarks and leaderboards are complete nonsense though. Find something you like, use it, be ready to look again in a month or two.

With the accelerating progress, the "be ready to look again" is becoming a full time job that we need to be able to delegate in some way, and I haven't found anything better than benchmarks, leaderboards and reviews.

EDIT: Typo

Post reply on HN