Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The killer app of Gemini Pro 1.5 is using video as an input

simonwillison.net

Post reply on HN
Print view View original

241–250 of 507 posts

Re: The killer app of Gemini Pro 1.5 is using video as an input

#241

Post by hodanli »

DariusKocar wrote: post #143
MyFirstSass wrote: post #71

Ok, crazy tangent; Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably. Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this…

I'm working on this! https://www.perfectmemory.ai/ It's encrypted (on top of Bitlocker) and local. There's all this competition who makes the best, most articulate LLM. But the truth is that off-the-shelf 7B models can put sentences together with no problem. It's the context they're missing.

statistics about the usage would be cool

Re: The killer app of Gemini Pro 1.5 is using video as an input

#242

Post by EGreg »

MyFirstSass wrote: post #71

Ok, crazy tangent; Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably. Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this…

Imagine if it starts suggesting the ideal dating partner as both of you browse profiles. Actually, dating sites can do that now.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#243

Post by jjjjj55555 »

MyFirstSass wrote: post #104
mdanger007 wrote: post #97

Earlier quoted context omitted.

Reading The Four by Scott Galloway, Apple, Facebook, Google, and Amazon were dominating the market 7 years ago generating 2.3 trillion in wealth. They're worth double that now. The Four, especially with its AI, is going to control the market in ways that will have a deep impact on government and society.

Yeah, that's one of the developments i'm unable to spin positively. As technological society advances the threshold to enter the market with anything not completely laughable becomes exponentially harder, only consolidating old money or the already established right? What i found so amazing about the early internet, or even just the internet 2.0 was the possibility to create a platform/marketplace/magazine or whateve…

Seems like the big tech cos are going to build the underlying infrastructure but you'll still be able to identify those small market opportunities and develop and sell solutions to fit them.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#244

Post by fragmede »

slg wrote: post #235
fragmede wrote: post #224

Earlier quoted context omitted.

Given that http://rewind.ai is doing just that, the odds are pretty good!

No they aren't. Rewind uses ChatGPT so data is sent off your local device[1]. I understand the actual screen recordings don't leave your machine, but that just creates a catch-22 of what does. Either the text based summaries of those recordings are thorough enough to still be worthy of privacy or the actual answers you get won't actually include many details from those recordings. [1] - https://help.rewind.ai/en/arti…

ah yeah fair point. it's the screen recordings I'm worried about leaving my computer

Re: The killer app of Gemini Pro 1.5 is using video as an input

#245

Post by EGreg »

ssl-3 wrote: post #163
underdeserver wrote: post #126

Earlier quoted context omitted.

It's local to your employer 's computer.

It can be. It can also be local to my own computer. People do write software while they're away from work.

How quaint.

You humans think that the AI will have someone in charge of it. Look, that's a thin layer that can be eliminated quickly. It's like when you build a tool that automates the work of, say, law firms but you don't want law firms getting mad that you're giving it away to their clients, so you give it to the law firms and now they secretly use the automating software. But it's only a matter of time before the humans are eliminated from the loop:

https://www.youtube.com/watch?v=SrIf0oYTtaI

The employee will be eliminated. But also the employer. The whole thing can be run by AI agents, which then build and train other AI agents. Then swarms of agents can carry out tasks over long periods of time, distributed, while earning reputation points etc.

This movie btw is highly recommended, I just can't find it anywhere anymore due to copyright. If you think about it, it's just a bunch of guys talking in rooms for most of the movie, but it's a lot more suspenseful than Terminator: https://www.youtube.com/watch?v=kyOEwiQhzMI

Re: The killer app of Gemini Pro 1.5 is using video as an input

#246

Post by pennomi »

rlt wrote: post #215
crooked-v wrote: post #162

Earlier quoted context omitted.

I feel like the storage requirements are really going to be these issue for these apps/services that run on "take screenshots and OCR them" functionality with LLMs. If you're using something like this a huge part of the value proposition is in the long term, but until something has a more efficient way to function, even a 1-year history is impractical for a lot of people. For example, consider the classic situation o…

I think ultimately you’d want it to summarize that down to something like: “Purchased socks from Amazon for $10 on 12/4/2024 at 5:04PM, shipped to Mom, 1600 Pennsylvania Av NW, Washington DC 20500, order number 1463355337 Probably stored in a vector DB for RAG.

Maybe. Until we find there’s a better way to encode the information and need the unfiltered, original context so it can be used with that new method.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#247

Post by kristofferR »

lillecarl wrote: post #196
jascination wrote: post #190

Earlier quoted context omitted.

> That's 108 GB/year. HDD/SSDs are getting bigger than that every year. Cries in MacBook Pro

PerfectMemory is only available on Windows at the moment.

https://Rewind.ai is the macOS equivalent

Re: The killer app of Gemini Pro 1.5 is using video as an input

#248

Post by flextheruler »

QuercusMax wrote: post #229

Guess the author didn't bother to check that those books actually are correct? The first one I checked, "Growing Up with Lucy by April Henry" doesn't exist. The actual book is by Steve Grand, and it's very obviously so in the video used as input. So a cool demo, but sadly useless for anything more.

I think this post and others reactions and then your comment this far down really encapsulates where we’re at with this technology.

Nearly 90 percent of comments on posts about LLMs are people talking about how the near future is about to boggle our minds and that general intelligence is near, but all my experiences with these LLMs show they’re capable of making the most basic of mistakes and doing so confidently and that’s just the tip of the iceberg in terms of their problems.

I’m having a hard time buying into the hype that these will be able to competently replace nearly any job anytime soon. They’re useful tools but they all come with a big asterisk of human hand holding.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#249

Post by kristofferR »

m-GDEV wrote: post #214
DariusKocar wrote: post #143

Earlier quoted context omitted.

I'm working on this! https://www.perfectmemory.ai/ It's encrypted (on top of Bitlocker) and local. There's all this competition who makes the best, most articulate LLM. But the truth is that off-the-shelf 7B models can put sentences together with no problem. It's the context they're missing.

Any plan to implement this on macOS or Linux?

macOS: https://www.rewind.ai/

Re: The killer app of Gemini Pro 1.5 is using video as an input

#250

Post by davidmurdoch »

wepple wrote: post #186
nostromo wrote: post #140

> It looks like the safety filter may have taken offense to the word “Cocktail”! I opened up the safety settings, dialled them down to “low” for every category and tried again. It appeared to refuse a second time. Google really is its own worst enemy. Their risk management people have completely taken over the organization to a point where somehow the smartest computers ever created are afraid of using dangerous word…

At the same time, nearly daily there’s a “google did a bad thing” post on HN front page. Can’t win I guess?

There are things Google does itself, and then there are the things Google won't allow users to do.
Post reply on HN