> 7 second video consumed just 1,841 tokens How? Video is a massive amount of data
It turns out it's 258 tokens per frame, and they only sample one frame every second.
The killer app of Gemini Pro 1.5 is using video as an input
181–190 of 507 posts
Re: The killer app of Gemini Pro 1.5 is using video as an input
#182Earlier quoted context omitted.
I'm working on this! https://www.perfectmemory.ai/ It's encrypted (on top of Bitlocker) and local. There's all this competition who makes the best, most articulate LLM. But the truth is that off-the-shelf 7B models can put sentences together with no problem. It's the context they're missing.
Your website and blog are very low on details on how this is working. Downloading and installing an mai directly feels unsafe imo. Especially when I don't know how this software is working. Is it recording a video, performing OCR continuously, taking just screenshots No mention of using any LLMs in there at all which is how you are presenting it in your comment here.
Feedback taken. I'll add more details on how this works for us technical people. LLM integration is in progress and coming soon.
Any idea what would make you feel safe? 3rd party verification? I had it verified and published by the Microsoft Store. I feel eventually it all comes down to me being a decent person.
Re: The killer app of Gemini Pro 1.5 is using video as an input
#183Ok, crazy tangent; Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably. Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this…
https://www.rewind.ai/ seems to be exactly this
Re: The killer app of Gemini Pro 1.5 is using video as an input
#184Earlier quoted context omitted.
I would hate that so much.
IKR, Who wouldn't want another Clippy constantly nagging you, but this time with a higher IQ and more intimate knowledge of you? /s
Clippy, definition: bot created by mega corp.
Clippy + high IQ: red flag, right here
Clippy + high IQ + intimate knowledge of you: do you seriously want that? Why?
Re: The killer app of Gemini Pro 1.5 is using video as an input
#185Great demo, but let's not forget this is essentially OCR. The real killer case is content understanding and discovery. I am building an app, maybe someone from G wants to team up? :)
Re: The killer app of Gemini Pro 1.5 is using video as an input
#186> It looks like the safety filter may have taken offense to the word “Cocktail”! I opened up the safety settings, dialled them down to “low” for every category and tried again. It appeared to refuse a second time. Google really is its own worst enemy. Their risk management people have completely taken over the organization to a point where somehow the smartest computers ever created are afraid of using dangerous word…
At the same time, nearly daily there’s a “google did a bad thing” post on HN front page.
Can’t win I guess?
Re: The killer app of Gemini Pro 1.5 is using video as an input
#187Earlier quoted context omitted.
War on drags? I thought that was just in Florida
please consider commenting more thoughtfully. I understand this is a joke but we don't want this site to devolve into Reddit.
It is sad that we live in a world where this could be interpreted both ways.
Re: The killer app of Gemini Pro 1.5 is using video as an input
#188Earlier quoted context omitted.
That's impel - https://tryimpel.com
The "smart tasks" functionality looks like the most compelling part of that to me, but it would have to be REALLY reliable for me to use it. 50% reliability in capturing tasks is about the same as 0% reliability when it comes to actually being a useful part of anything professional.
The hard part of any smart automation system, and probably 95% of the UX is timing and managing the prompts/notifications you get.
It can do as much as it wants in the background turning that into timely and non-intrusive actionable behaviours is extremely challenging.
I spent a long time thinking about a global notification consumption system that would parse all desktop, mobile, email, slack, web app, etc notifications into a single stream and then intelligently organizes it with adaptive timing and focus streams.
The cross platform nature made it infeasible but it was a fun thought experiment because we often get repeated notifications on every different device/interface and most of the time we just zone it out cuz it’s overload.
Adding a new nanny to your desktop is just going to pile it on even more so you have to be careful.
Re: The killer app of Gemini Pro 1.5 is using video as an input
#189"So Google’s new Gemini chatbot is racist as fuck." https://twitter.com/JoshWalkos/status/1760423141942178037
This has nothing to do with the model's capabilities and isn't substantially different from the vast majority of mainstream values in content moderation on social media.
[dead]
Re: The killer app of Gemini Pro 1.5 is using video as an input
#190Earlier quoted context omitted.
I feel like the storage requirements are really going to be these issue for these apps/services that run on "take screenshots and OCR them" functionality with LLMs. If you're using something like this a huge part of the value proposition is in the long term, but until something has a more efficient way to function, even a 1-year history is impractical for a lot of people. For example, consider the classic situation o…
It's not that bad. With Perfect Memory AI I see ~9GB a month. That's 108 GB/year. HDD/SSDs are getting bigger than that every year. The storage also varies by what you do, your workflow and display resolution. Here's an article I wrote on my finding of storage requirements. https://www.perfectmemory.ai/support/storage-resources/stora... And if you want to use the data for LLM only, then you don't need to store the sc…
> That's 108 GB/year. HDD/SSDs are getting bigger than that every year.
Cries in MacBook Pro