The killer app of Gemini Pro 1.5 is using video as an input

Post by RogerL » Thu, Feb 22, 2024, 5:58 AM UTC

lucubratory wrote: post #166

oconnor663 wrote: post #158

Earlier quoted context omitted.

A version of this that seems both easier and less weird would be an AI that listens to you all the time when you're learning a foreign language. Imagine how much faster you could learn, and how much more native you could ultimately get, if you had something that could buzz your watch whenever you said something wrong. And of course you'd calibrate it to understand what level you're at and not spam you constantly. I w…

I think even aside from the more outlandish ideas like that one, just having a fluent native speaker to talk to as much as you want would be incredibly valuable. Even more valuable if they are smart/educated enough to act as a language teacher. High-quality LLMs with a conversational interface capable of seamless language switching are an absolute killer app for language learning. A use that seems scientifically poss…

Post by dlandau » Thu, Feb 22, 2024, 6:06 AM UTC

simonw wrote: post #334

QuercusMax wrote: post #229

Guess the author didn't bother to check that those books actually are correct? The first one I checked, "Growing Up with Lucy by April Henry" doesn't exist. The actual book is by Steve Grand, and it's very obviously so in the video used as input. So a cool demo, but sadly useless for anything more.

I called out one hallucination - "The Personal MBA by Josh Kaufman" wasn't on my shelf. I didn't bother fact-checking every other book because I thought highlighting one mistake would illustrate that the results weren't accurate - which is pretty much expected for anything related to LLMs at this point.

Post by behnamoh » Thu, Feb 22, 2024, 6:09 AM UTC

joebob42 wrote: post #266

jasonjayr wrote: post #259

Earlier quoted context omitted.

This is where Microsoft (and Apple) has a leg up -- they can hook the UI at the draw level and parse the interface far more reliably + efficently than screenshot + OCR.

Google too, for all practical purposes, since presumably this is mostly just watching you use chrome 90% of the time.

Post by alex_suzuki » Thu, Feb 22, 2024, 6:11 AM UTC

goatlover wrote: post #307

Nition wrote: post #287

Earlier quoted context omitted.

Eventually someone will realise that it'd also be great for telling you where you left your keys, if it'd film everything you see instead of just your screen.

I simply am not going to have my entire life filmed by an form of technology, I don't care what the advantages are. There's a limit to the level of dystopian dependent uses of these technologies I'm going to put up with. I sincerely hope the majority of the human race feels the same way.

Post by Aerroon » Thu, Feb 22, 2024, 6:21 AM UTC

ugh123 wrote: post #100

CamelCaseName wrote: post #50

Earlier quoted context omitted.

For now, but in a year? You could also stagger the moderation to reduce costs. E.g. Text analysis: 2 views Audio analysis: 300 views Frame analysis: 5,000 views I would be very surprised if even 20% of content uploaded to YouTube passes 300 views.

Or.. google supplies some kind of local LLM tool which processes your videos before uploaded. You pay for the gpu/electricity costs. Obviously this would need to be done in a way that can't be hacked/manipulated. Might need to be highly integrated with a backend service that manages the analyzed frames from the local machine and verifies hashes/tokens after the video is fully uploaded to YouTube.