Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ferret: A Multimodal Large Language Model

github.com

Post reply on HN
Print view View original

281–290 of 332 posts

Re: Ferret: A Multimodal Large Language Model

#281

Post by dbish »

spookthesunset wrote: post #140
dbish wrote: post #72

Earlier quoted context omitted.

This is part of the mismatch between comparing Alexa/Siri/Cortana to a chat based LLM right now. If you just want chat and info retrieval, today’s LLMs are way better then the conversational dialogue, search, and q&a capabilities any of those assistants have. But, if you want relatively predicable task completion for things like smart home control, timers and alarms, real time weather updates, etc. (basic but frequen…

I would hope that it would at last get you out of the hell that is “what the heck did I name that stupid light?”. Device naming is, in my opinion, the worst part of any of the voice based home assistant things. Is it “porch led”, “porch light” or “outdoor light”? Or is “outdoor light” actually the one in the front yard? What is the one by my kids nightstand? And what routine name do I use to set the mood for watching…

Maybe, seems like overkill to solve a relatively straight forward problemt that is about resolution to real entities with a generative model that has no entity grounding (and may introduce other problems). It's really just not solved because Alexa/Siri leadership doesn't actually care enough about that use case to solve it. Device entity resolution and disambiguation does not require an LLM to solve for a limited number of devices, it just requires people to prioritze that over other things, and smart home device owners are the kind of early adopter that would not be the current market focus for growth (my guess, haven't worked on Alexa in many years).

I know a half dozen different ways to improve that today without LLMs.

Re: Ferret: A Multimodal Large Language Model

#282

Post by smolder »

ex3ndr wrote: post #270
zitterbewegung wrote: post #168

Earlier quoted context omitted.

Honestly if they are coming out with a paper now Apple has probably been working on it for a year or two at minimum . Next year releases of macOS / iOS are rumored to have LLMs as a feature .

They literally mention that they built on top of llava that was released half year ago.

What's the "literally" here? Try: They mention that they built on top of llava, which was released half a year ago.

Re: Ferret: A Multimodal Large Language Model

#283

Post by superb_dev »

smolder wrote: post #282
ex3ndr wrote: post #270

Earlier quoted context omitted.

They literally mention that they built on top of llava that was released half year ago.

What's the "literally" here? Try: They mention that they built on top of llava, which was released half a year ago.

In this context “literally” is being used to draw attention to the perceived obviousness of the information. Essentially saying “how could you not see this?”

Re: Ferret: A Multimodal Large Language Model

#284

Post by nmstoker »

barbecue_sauce wrote: post #35
CaptainOfCoit wrote: post #8

Maybe the abstract of the paper is a better introduction to what this is: > We introduce Ferret, a new Multimodal Large Language Model (MLLM) capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions. To unify referring and grounding in the LLM paradigm, Ferret employs a novel and powerful hybrid region representation that integrates d…

>>spatial referring I can't seem to nail down the meaning of this phrase on its own. All the search results seem to turn up are "spatial referring expressions".

Yes, I wondered whether "referring" had some special meaning, since the way they seem to use it suggests the word reference would normally be more appropriate there (unless it's a special meaning that warrants the different word).

Re: Ferret: A Multimodal Large Language Model

#285

Post by walteweiss »

FLT8 wrote: post #271
Booourns wrote: post #235

Earlier quoted context omitted.

Out of curiosity have you experienced their autocorrect on iOS 17 because that’s when they updated to be LLM based?

I don't recall exactly when it started happening, but I've been having lots of issues with recent iOS versions rewriting not the last word entered, but the word before that. For example, if I start entering "I went to", it'll sometimes correct to "I want to", but it'll do that after I've typed the "to". I've found lots of similar examples. The retrospective nature of the edits mean I miss a lot of them and makes me a…

Same happens to me quite very often on a mobile, even here. But I use iPhone SE 1st Gen. with iOS 15.8.

Re: Ferret: A Multimodal Large Language Model

#286

Post by theferalrobot »

Aerbil313 wrote: post #274
theferalrobot wrote: post #258

Earlier quoted context omitted.

>They just have to build a compelling consumer product that can only function with AI Yeah and I'm not talking exclusively about developer trust. Given Apple's current consumer lineup (see Siri, Apple photos, predictive text etc)... we only have evidence that they suck at ML. What makes you think they are going to suddenly transform overnight?

> see Siri, Apple photos, predictive text etc)... we only have evidence that they suck at ML …or that they only deal with mature tech and not the shiny new thing. Makes sense to me. I don’t doubt everyone will have a personal LLM-based assistant in their phones soon, but with the current rate of improvements to LLMs and AI in general, I’d wait for at least a year more while doing R&D in-house if I were Apple.

You could use that apologist language for any company. If they suck at something just say they are “biding their time” No. Apple is just demonstrably behind.

Having terrible predictive text, voice to text, image classification etc isn’t just a quark of the way they do business. Those are problems with years of established work put into them and they just flat out aren’t keeping up.

Re: Ferret: A Multimodal Large Language Model

#287

Post by saagarjha »

Someone wrote: post #170
behnamoh wrote: post #62

Earlier quoted context omitted.

> Rumors suggest they’re gearing up to make iOS 18 an AI focused release. Don't underestimate Apple at disappointing enthusiasts like you and me. We've been hearing many awesome stories about the next thing Apple will do, only to realize their marketing team chose to keep it for future iOS/MBP/iPhone generations to keep the profits high.

Running a LLM on-device alongside other apps (i.e. without it taking up all phone resources), and it being reasonably fast may well require more powerful hardware than they ever sold. A voice assistant that takes 3 seconds to reply and then takes half a second per word is a nice demo, but not a product Apple wants to sell. And yes, some people will say they rather have that than nothing on their hardware, but “the In…

Siri is often slower than this…

Re: Ferret: A Multimodal Large Language Model

#288

Post by fodkodrasz »

para_parolu wrote: post #55
aaronbrethorst wrote: post #33

Earlier quoted context omitted.

Rumors suggest they’re gearing up to make iOS 18 an AI focused release. It’ll be interesting to see if they offer different capabilities for online/offline scenarios, or if their offerings are strictly offline. Here’s one story to offer some context. There are others. https://archive.is/en3VL

I really hope they will make siri usable. In the current state it’s only good for fixed phrases. And even then it fails time to time

For me fixed phrases would do it often, I use Siri mostly when driving, but keeps saying for almost any command/query: Sorry I can't do that while you are driving. (other main usecase is setting tea/cooking timers with hands full/greasy. This alone makes it pretty useful.)

I guess this won't change, because it is probably for legal reasons, to avoid being sued by some "I just dried my cat in the microwave" style genius after making a car accident (unrelated to Siri, but trying to shift the blame).

Adding support for smaller languages would be nice actually. When its reading out of Hungarian messages loud it sounds incredibly retarded. I always have a great time listening to those and trying to guess the message. :) It would be nice if I could send a message to my significant other about being stuck in traffic in Hungarian. (the iPhone keyboard already does pretty decent voice recognition in the language)

Re: Ferret: A Multimodal Large Language Model

#289

Post by cezart »

shrimpx wrote: post #236
pkage wrote: post #231

Earlier quoted context omitted.

I just wish you could turn the multilingual keyboard off—I find that I usually only type in one language at a time and having the autocomplete recommend the wrong languages is quite frustrating

That's true, I have found that mildly annoying sometimes. But most of the time it's a win. It was really annoying manually switching modes over and over when typing in mixed-language, which I do fairly often. It'd be great if there was a setting though.

I had the opposite problem, the languages I usually typed in(Romanian + English) didn't have a multi language mode on iOS. So it was a constant pain to switch btw them when I needed to insert some English terms in Romanian sentences. IOS didn't support multi language for this language pair. On Android it always worked like a charm.

Re: Ferret: A Multimodal Large Language Model

#290

Post by yreg »

ignoramous wrote: post #217
behnamoh wrote: post #62

Earlier quoted context omitted.

> Rumors suggest they’re gearing up to make iOS 18 an AI focused release. Don't underestimate Apple at disappointing enthusiasts like you and me. We've been hearing many awesome stories about the next thing Apple will do, only to realize their marketing team chose to keep it for future iOS/MBP/iPhone generations to keep the profits high.

> marketing team chose to keep it for future iOS/MBP/iPhone generations to keep the profits high. VisionPro is nice. I can see costs coming down over a period of time. Also, we've been waiting long enough for that AI car.

Apple has never promised any car. From what is known, project Titan has been cancelled years ago.

The only exeption of not delivering I can recall was AirPower. It's a product they've announced and then embarassingly weren't able to finish up to their standards (or up to what was promised), so they have cancelled it altogether.

Post reply on HN