Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The killer app of Gemini Pro 1.5 is using video as an input

simonwillison.net

Post reply on HN
Print view View original

121–130 of 507 posts

Re: The killer app of Gemini Pro 1.5 is using video as an input

#121

Post by ryukoposting »

tekni5 wrote: post #57

I was thinking about this a while back, once AI is able to analyze video, images and text and do so cheap & efficiently. It's game over for privacy, like completely. Right now massive corps have tons of data on us, but they can't really piece it together and understand everything. With powerful AI every aspect of your digital life can be understood. The potential here is insane, it can be used for so many different t…

You nailed it on the head. People dismissing this because it isn't perfectly accurate are missing the point. For the purposes of analytics and surveillance, it doesn't need to be perfectly accurate as long as you have enough raw data to filter out the noise. The Four have already mastered the "collecting data" part, and nobody in North America with the power to rein in that situation seems interested in doing so (this isn't to say the GDPR is perfect, but at least Europe is trying).

It's depressing that the most extraordinary technologies of our age are used almost exclusively to make you buy shit.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#122

Post by chamomeal »

MyFirstSass wrote: post #71

Ok, crazy tangent; Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably. Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this…

Not crazy! I listened to a software engineering daily episode about pieces.app. Right now it’s some dev productivity tool or something, but in the interview the guy laid out a crazy vision that sounds like what you’re talking about.

He was talking about eventually having an agent that watches your screen and remembers what you do across all apps, and can store it and share it with you team.

So you could say “how does my teammate run staging builds?” or “what happened to the documentation on feature x that we never finished building”, and it’ll just know.

Obviously that’s far away, and it was just the ramblings of excited founder, but it’s fun to think about. Not sure if I hate it or love it lol

Re: The killer app of Gemini Pro 1.5 is using video as an input

#123

Post by dist-epoch »

TheCaptain4815 wrote: post #22

I wonder if the real killer app is Googles hardware scale verses OpenAi' s(or what Microsoft gives them). Seems like nothing Google's done has been particular surprising to OpenAi's team, it's just they have such huge scale maybe they can iterate faster.

The real moat is that Google has access to all the video content from YouTube to train the AI on, unlike anyone else.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#124

Post by slg »

MyFirstSass wrote: post #71

Ok, crazy tangent; Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably. Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this…

>Isolated, encrypted and local of course.

And what is the likelihood of that "of course" portion actually happening? What is the business model that makes that route more profitable compared to the current model all the leaders in this tech are using in which they control everything?

Re: The killer app of Gemini Pro 1.5 is using video as an input

#125

Post by simonw »

zacmps wrote: post #117
loudmax wrote: post #99

At the end of the article, a single image of the bookshelf uploaded to Gemini is 258 tokens. Gemini then responds with a listing of book titles, coming to 152 tokens. Does anyone understand where the information for the response came from? That is, does Gemini hold onto the original uploaded non-tokenized image, then run an OCR on it to read those titles? Or are all those book titles somehow contained in those 258 to…

Remember, if it's using a similar tokeniser to GPT-4 (cl100k_base iirc), each token has a dimension of ~100,000. So 258x100,000 is a space of 25,800,000 floats, using f16 (a total guess) that's 51.6kB, probably enough to represent the image at ok quality with JPG.

I don't think that's right. A token in GPT-4 is a single integer, not a vector of floats.

Input to a model gets embedded into vectors later, but the actual tokens are pretty tiny.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#126

Post by underdeserver »

ghxst wrote: post #115
Animats wrote: post #106

Earlier quoted context omitted.

> Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. And then announcing "I can do your job now. You're fired."

That's why we would want it to run locally! Think about a fully personalized model that can work out some simple tasks / code while you're going out for groceries, or potentially more complex tasks while you're sleeping.

It's local to your employer's computer.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#128

Post by hackerlight »

simonw wrote: post #96
nox101 wrote: post #89

Earlier quoted context omitted.

honest question: Why is it bad? I see that posted over and over. Right now I watch SF and LA feel like 3rd world countries. Nothing appears to be enforced. Traffic laws, car break-ins, car theft, garage break-ins, house break-ins. I'd personally choose a little less privacy if it meant less people were getting injured by drivers ignoring the traffic laws and, less people were having to shell out for all the costs ass…

It's bad because while you may trust the government right now, there are no guarantees that a government you do NOT trust won't be elected in the future. Also important to consider that government institutions are made up of individuals. Do you want a police officer who is the abuser in an bad domestic situation being given the power to track their partner using the resources made available to them in their work?

> It's bad because while you may trust the government right now, there are no guarantees that a government you do NOT trust won't be elected in the future.

Yes, but this ignores the reverse causality component.

If people feel unsafe then the probability that a bad government gets elected goes up. Look at El Salvador. Freedom can't survive if people's basic needs (such as physical safety) aren't met.

The freedom vs safety dichotomy isn't a simple spectrum. There are feedback dynamics.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#129

Post by foolfoolz »

MyFirstSass wrote: post #71

Ok, crazy tangent; Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably. Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this…

you could design a similar product to do the opposite and anonymize your work automatically

Re: The killer app of Gemini Pro 1.5 is using video as an input

#130

Post by mixmastamyk »

MyFirstSass wrote: post #71

Ok, crazy tangent; Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably. Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this…

"It looks like you're writing a suicide note... care for any help?"

https://www.reddit.com/r/memes/comments/bb1jq9/clippy_is_qui...

Post reply on HN