Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The killer app of Gemini Pro 1.5 is using video as an input

simonwillison.net

Post reply on HN
Print view View original

251–260 of 507 posts

Re: The killer app of Gemini Pro 1.5 is using video as an input

#251

Post by baobabKoodaa »

wepple wrote: post #186
nostromo wrote: post #140

> It looks like the safety filter may have taken offense to the word “Cocktail”! I opened up the safety settings, dialled them down to “low” for every category and tried again. It appeared to refuse a second time. Google really is its own worst enemy. Their risk management people have completely taken over the organization to a point where somehow the smartest computers ever created are afraid of using dangerous word…

At the same time, nearly daily there’s a “google did a bad thing” post on HN front page. Can’t win I guess?

Nobody would complain on HN if Google Gemini was generating pictures of Lincoln existing as a... gasp... white person. This absurd level of woke censorship is not doing them any good.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#253

Post by ebisoka »

lucubratory wrote: post #171
Vicinity9635 wrote: post #165

"So Google’s new Gemini chatbot is racist as fuck." https://twitter.com/JoshWalkos/status/1760423141942178037

This has nothing to do with the model's capabilities and isn't substantially different from the vast majority of mainstream values in content moderation on social media.

Ah yes the "mainstream values" where there is no problem with "reverse" racism or "reverse" sexism.

Who cares about the model when the owners are a bunch of racists and sexists, altough I guess some people who share these disgusting and regressive "values" will think it's great.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#254

Post by ssl-3 »

EGreg wrote: post #245
ssl-3 wrote: post #163

Earlier quoted context omitted.

It can be. It can also be local to my own computer. People do write software while they're away from work.

How quaint. You humans think that the AI will have someone in charge of it. Look, that's a thin layer that can be eliminated quickly. It's like when you build a tool that automates the work of, say, law firms but you don't want law firms getting mad that you're giving it away to their clients, so you give it to the law firms and now they secretly use the automating software. But it's only a matter of time before the…

We've all seen the historical documents. We know how this will all end up, and that the end result is simply inevitable.

And since that has to be the case, we might as well find fun and profit wherever we can -- while we still can.

If that means that my desktop robot is keeping tabs on me while I write this, then so be it as long as I get some short-term gain. (There can be no long-term gain.)

Re: The killer app of Gemini Pro 1.5 is using video as an input

#255

Post by Uptrenda »

simonw wrote: post #178
Uptrenda wrote: post #169

[flagged]

Rude! I knew putting a video of a bookshelp up would risk people judging me based on the books they saw there, but I got over that by deciding that if anyone did that it would reflect badly on them, not badly on me. (Unsurprisingly, I own a lot more books than the ones visible in those videos.)

Lol, sorry. I'll give you the benefit of the doubt.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#256

Post by sdenton4 »

dav43 wrote: post #239
crooked-v wrote: post #162

Earlier quoted context omitted.

I feel like the storage requirements are really going to be these issue for these apps/services that run on "take screenshots and OCR them" functionality with LLMs. If you're using something like this a huge part of the value proposition is in the long term, but until something has a more efficient way to function, even a 1-year history is impractical for a lot of people. For example, consider the classic situation o…

I set up two years ago a cron to screenshot every minute. Just did the second phase of using ocrmac (vision kit cli on GitHub) that extracts text and dumps it in a SQLite with FTS5. It’s simplistic but does the job for now. I looked at reducing storage requirements by using image magik to only store the difference between images - some 5 min sequence are essentially the same screen - but let that one go.

/using image magik to only store the difference between images/

Well, that's basically how video codecs work... So might as well just find some codec params which work well with screen capture, and use an existing encoder.

Re: The killer app of Gemini Pro 1.5 is using video as an input

#259

Post by jasonjayr »

crooked-v wrote: post #162
DariusKocar wrote: post #143

Earlier quoted context omitted.

I'm working on this! https://www.perfectmemory.ai/ It's encrypted (on top of Bitlocker) and local. There's all this competition who makes the best, most articulate LLM. But the truth is that off-the-shelf 7B models can put sentences together with no problem. It's the context they're missing.

I feel like the storage requirements are really going to be these issue for these apps/services that run on "take screenshots and OCR them" functionality with LLMs. If you're using something like this a huge part of the value proposition is in the long term, but until something has a more efficient way to function, even a 1-year history is impractical for a lot of people. For example, consider the classic situation o…

This is where Microsoft (and Apple) has a leg up -- they can hook the UI at the draw level and parse the interface far more reliably + efficently than screenshot + OCR.
Post reply on HN