Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ferret: A Multimodal Large Language Model

github.com

Post reply on HN
Print view View original

321–330 of 332 posts

Re: Ferret: A Multimodal Large Language Model

#321

Post by fennecbutt »

shrimpx wrote: post #227
harryVic wrote: post #220

Earlier quoted context omitted.

Can you give an example? I switched to android because i use personal assistant a lot while driving and siri was absolutely horrible.

- FaceID - Facial recognition in Photos - "Memories" in Photos - iOS keyboard autocomplete using LLMs. I am bilingual and noticed in the latest iOS it now does multi-language autocomplete and you no longer have to manually switch languages. - Event detection for Calendar - Depth Fusion in the iOS camera app, using ML to take crisper photos - Probably others... The crazy thing is most/all of these run on the device.

Yeah face id is pretty good, no android phones seem to use the ir dot camera which makes me think Apple has a "patent" on it ...lame considering the dot projector is ripped straight out of kinekt.

Google does the memory photo thing too, but only if you use their app which I don't.

Android has had multi language keyboard support for the longest time, in addition to being able to install whatever keyboard I like (I use SwiftKey, it's brilliant) I can also install an llm based one as I please.

Android/my keyboard already does event detection/suggestion in text and has been doing for as long as I remember.

None of these are reasons to buy an iPhone...just reasons to buy a phone, lmao.

Re: Ferret: A Multimodal Large Language Model

#322

Post by fennecbutt »

hansoolo wrote: post #253
shrimpx wrote: post #227

Earlier quoted context omitted.

- FaceID - Facial recognition in Photos - "Memories" in Photos - iOS keyboard autocomplete using LLMs. I am bilingual and noticed in the latest iOS it now does multi-language autocomplete and you no longer have to manually switch languages. - Event detection for Calendar - Depth Fusion in the iOS camera app, using ML to take crisper photos - Probably others... The crazy thing is most/all of these run on the device.

To be honest I distrust Microsoft with swift key, but the it recognizes the change in language just smooth. I could switch languages in one sentence and it would understand what I am writing just fine, no Sill suggestions

SwiftKey is great and if someone distrusts Microsoft for it...then fine, but various different companies "control" various different parts of my phone.

With an iPhone, it's only one company, that controls every little thing, and we have no insight into Apple at all. They can basically do whatever the hell they want.

Re: Ferret: A Multimodal Large Language Model

#323

Post by fennecbutt »

zamalek wrote: post #308
shrimpx wrote: post #216

Apple has been looking sleepy on LLMs, but they've been consistently evolving their hardware+software AI stack, without much glitzy advertising. I think they could blow away Microsoft/OpenAI and Google, if suddenly a new iOS release makes the OpenAI/Bard chatbox look laughably antiquated. They're also a threat to Nvidia, if a significant swath of AI usage switches over to Apple hardware. Arm and TSMC would stand to w…

You're suggesting that Apple could fit what can't be done with a 4090 into a laptop? Color me doubtful.

But Apple will just make a magical chip that's different to regular hardware cause they're the best company and invent all the things even if they've been seen before Apple still invented it first, just wait until their Super Unicorn Ultra™ chip comes out with Hyperdrive Retinated LLM™ support, they don't name normal hardware different just for marketing...it's really unique, new and inventive hardware that we're happy to pay a huge premium for because it's so advanced and inventive.

Re: Ferret: A Multimodal Large Language Model

#324

Post by fennecbutt »

yreg wrote: post #21

I really hope Apple releases an iPhone with a good on-device private LLM assistant, perhaps next year. Their hardware is well-positioned for it. It could make me get a new phone outside of my usual ~4 year cycle. Siri is almost unusable for me.

Nobody can tame LLM models yet not even Apple.

I can still get chatgpt to say the most vile things and if Apple release something on device I'll get that to be a bad, baaaad robot, too.

LLMs are not yet safe for public facing production use,imo.

Re: Ferret: A Multimodal Large Language Model

#325

Post by spaceman_2020 »

klabb3 wrote: post #266
spaceman_2020 wrote: post #196

Earlier quoted context omitted.

GPT-4 voice is so, so good. Really what you would want a voice tool to be like. I can talk to it like a normal human being, unlike issuing specific commands loudly as with Siri.

But no matter the Siri shittiness (which I agree with) an LLM can only interact with the outside world – ie run commands – that exist and have a reasonable API surface, no? Apple has had automation for ages with Automator, Shortcuts etc but nothing that actually integrates well with day to day flow. So.. setting a timer when my hands are wet already works ok, and that’s about what I need. I honestly wonder what type…

I would be happy if it does my morning routine for me. Give me a brief summary of my emails over the night, tell me the headlines from the news outlets I follow, summarize tweets from my favorited accounts, and give me an overview of the market based on my investments and watchlists.

Re: Ferret: A Multimodal Large Language Model

#326

Post by kergonath »

pants2 wrote: post #255
shrimpx wrote: post #227

Earlier quoted context omitted.

- FaceID - Facial recognition in Photos - "Memories" in Photos - iOS keyboard autocomplete using LLMs. I am bilingual and noticed in the latest iOS it now does multi-language autocomplete and you no longer have to manually switch languages. - Event detection for Calendar - Depth Fusion in the iOS camera app, using ML to take crisper photos - Probably others... The crazy thing is most/all of these run on the device.

The iPhone's built in text OCR and image subject cutouts are also extremely good, just in the photos app.

The combination of automatic OCR and translation almost everywhere in the OS is great.

Re: Ferret: A Multimodal Large Language Model

#327

Post by jmisavage »

fennecbutt wrote: post #321
shrimpx wrote: post #227

Earlier quoted context omitted.

- FaceID - Facial recognition in Photos - "Memories" in Photos - iOS keyboard autocomplete using LLMs. I am bilingual and noticed in the latest iOS it now does multi-language autocomplete and you no longer have to manually switch languages. - Event detection for Calendar - Depth Fusion in the iOS camera app, using ML to take crisper photos - Probably others... The crazy thing is most/all of these run on the device.

Yeah face id is pretty good, no android phones seem to use the ir dot camera which makes me think Apple has a "patent" on it ...lame considering the dot projector is ripped straight out of kinekt. Google does the memory photo thing too, but only if you use their app which I don't. Android has had multi language keyboard support for the longest time, in addition to being able to install whatever keyboard I like (I use…

FYI Apple bought the company that made the kinect sensors. The Face ID module on iPhones is a mini kinect.

Re: Ferret: A Multimodal Large Language Model

#329

Post by slowmovintarget »

shrimpx wrote: post #216

Apple has been looking sleepy on LLMs, but they've been consistently evolving their hardware+software AI stack, without much glitzy advertising. I think they could blow away Microsoft/OpenAI and Google, if suddenly a new iOS release makes the OpenAI/Bard chatbox look laughably antiquated. They're also a threat to Nvidia, if a significant swath of AI usage switches over to Apple hardware. Arm and TSMC would stand to w…

Personal ML systems running on hardware you own is the killer app. If these are "good enough" they'll be significantly preferable to using large subscription-based models, where those companies could pull a Lucy any day.

Re: Ferret: A Multimodal Large Language Model

#330

Post by qgin »

ghusbands wrote: post #301
shrimpx wrote: post #230

Earlier quoted context omitted.

Yeah. Siri supports text input already, anyway. Siri is their ChatGPT-style bot that's going to keep improving.

But does it even work sensibly, yet? Almost every time my partner asks Siri something, it works so badly that we end up asking Android/Google's Assistant, which responds well to most things.

What sort of things don’t work well? Phone actions or knowledge / info type questions?
Post reply on HN