Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ferret: A Multimodal Large Language Model

github.com

Post reply on HN
Print view View original

91–100 of 332 posts

Re: Ferret: A Multimodal Large Language Model

#91

Post by TrueDuality »

barbecue_sauce wrote: post #35
CaptainOfCoit wrote: post #8

Maybe the abstract of the paper is a better introduction to what this is: > We introduce Ferret, a new Multimodal Large Language Model (MLLM) capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions. To unify referring and grounding in the LLM paradigm, Ferret employs a novel and powerful hybrid region representation that integrates d…

>>spatial referring I can't seem to nail down the meaning of this phrase on its own. All the search results seem to turn up are "spatial referring expressions".

I'm just inferring myself, but I believe it's referring to discussing things in the foreground / background or in a specific location in the provided image (such as top right, behind the tree, etc) in user queries.

Re: Ferret: A Multimodal Large Language Model

#93

Post by SpaceManNabs »

Gorgor wrote: post #65
tambourine_man wrote: post #58

Earlier quoted context omitted.

Apple doesn’t even sell NVidia cards on their Mac Pros. Are they training it on Linux? I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried. AI/LLM is the new hot thing. If people are using Windows or Linux, you’re loosing momentum, hearts and minds… and sales, obviously.

But no one is training these kinds of models on their personal device. You need compute clusters for that. And they will probably run Linux. I'd be surprised if Microsoft trains their large models in anything else than Linux clusters.

> But no one is training these kinds of models on their personal device

on-device transfer learning/fine tuning is def a thing for privacy and data federation reasons. Part of the reason why model distillation was so hot a few years ago.

Re: Ferret: A Multimodal Large Language Model

#95

Post by ayewo »

smoldesu wrote: post #71
ssijak wrote: post #56

Earlier quoted context omitted.

I guess they also have Samsung fridges in the offices..

I don't get it, does Apple also make fridges now?

They are implying that even though Apple is a wealthy consumer hardware company that has major spats with nVidia and Samsung, it doesn't always make economic sense to make tools they might need in-house when they can simply buy them from a rival.

So rather than invest engineering resources to re-imagine the fridge, they can simply buy them from established manufacturers that make household appliances like Samsung, Sony etc.

Re: Ferret: A Multimodal Large Language Model

#96

Post by nexuist »

CaptainOfCoit wrote: post #24
yreg wrote: post #21

I really hope Apple releases an iPhone with a good on-device private LLM assistant, perhaps next year. Their hardware is well-positioned for it. It could make me get a new phone outside of my usual ~4 year cycle. Siri is almost unusable for me.

You're unlikely to get a better experience with Siri if she becomes equipped with a 7B or 13B LLM, unless Apple figured out something revolutionary.

Siri is really quite dumb. I am confident that a 7B model would be able to provide better responses in over 90% of user queries. I can't even get Siri to reliably set a timer.

Re: Ferret: A Multimodal Large Language Model

#97

Post by cryogenicfire »

causal wrote: post #76
ssijak wrote: post #56

Earlier quoted context omitted.

I guess they also have Samsung fridges in the offices..

And probably Intel processors and Linux in their datacenters.

Well apple was a prime Intel client for years until they released M1, and ARM on the cloud isn't really a thing for now... Ultimately it's all about what makes the most sense for what will make the most money, and on a datacenter that means x86 with Linux/Unix

Re: Ferret: A Multimodal Large Language Model

#98

Post by hmottestad »

tambourine_man wrote: post #45

> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, Apple Silicon

Apple had a falling out with Nvidia a number of years ago. I believe they were using chips from Nvidia in their MacBook Pros, the first to come with both integrated and discrete graphics, but the solder between the chips and the motherboard kept cracking and a rather large number of MacBooks needed to be repaired.

https://techcrunch.com/2008/12/09/scientists-nvidia-put-faul...

Re: Ferret: A Multimodal Large Language Model

#99

Post by zdragnar »

baz00 wrote: post #80
behnamoh wrote: post #62

Earlier quoted context omitted.

> Rumors suggest they’re gearing up to make iOS 18 an AI focused release. Don't underestimate Apple at disappointing enthusiasts like you and me. We've been hearing many awesome stories about the next thing Apple will do, only to realize their marketing team chose to keep it for future iOS/MBP/iPhone generations to keep the profits high.

I think your expectations are wrong. They sit there in silence with a few leaks here and there and some github projects, people speculate and get all excited about extrapolating those things. Then Apple deliver what works which may or may not be related to it. What they don't do is sell you a lie a year before release then deliver shit (like every other fucking vendor).

They're plenty capable of delivering garbage. Certain year models of the MacBook pro were inherently faulty. I've had the displeasure of having two of them bought for me at work.

All of Apple's windows software (iTunes, Safari, etc) has been, at best, a barely working port.

I'm assuming they are putting a lot more thought and care into it than the touchbar, crappy keyboards and the rest, but I'm also not holding out much hope either.

Re: Ferret: A Multimodal Large Language Model

#100

Post by tambourine_man »

cryogenicfire wrote: post #90
tambourine_man wrote: post #45

> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, Apple Silicon

I feel like Apple is only testing the waters with AI right now, but perhaps if they get involved enough they'll spend money on their own compute infrastructure? Nvidia is kind of the king at GPU compute right now, and developing comparable hardware is no small or cheap task, but I think Apple is in a very good position to be able to make it work---if they decide to invest in it. But honestly, as far as corporate feud…

I think you’re completely correct, but if they were caught off guard by the AI train, they shouldn’t be testing the waters now. It should be treated as an existential threat.
Post reply on HN