Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ferret: A Multimodal Large Language Model

github.com

Post reply on HN
Print view View original

51–60 of 332 posts

Re: Ferret: A Multimodal Large Language Model

#51

Post by sxg »

tambourine_man wrote: post #45

> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, Apple Silicon

By "A series" are you referring to the Nvidia A100 or the Apple A-series iPhone/iPad chips? If the latter, I don't think you can draw that conclusion. Training has memory and processor requirements that are very different from inference. You don't need iPhones and iPads to train models—you need them to run models. These are two very different things.

Re: Ferret: A Multimodal Large Language Model

#52

Post by bbor »

pests wrote: post #16
schaefer wrote: post #4

Earlier quoted context omitted.

Multimodal Large Language Model

why not LLMM?

Ok our options

  Multimodal large language model
  Large multimodal language model
  Large language multimodal model
  Large language model (multimodal)
I prefer 1, because this is a multimodal type of an existing technique already referred to as LLM. If I was king, I’d do Omnimodal Linguistic Minds, but no one asks me such things, thank god

Re: Ferret: A Multimodal Large Language Model

#53

Post by tambourine_man »

sxg wrote: post #51
tambourine_man wrote: post #45

> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, Apple Silicon

By "A series" are you referring to the Nvidia A100 or the Apple A-series iPhone/iPad chips? If the latter, I don't think you can draw that conclusion. Training has memory and processor requirements that are very different from inference. You don't need iPhones and iPads to train models—you need them to run models. These are two very different things.

Apple Silicon, sorry for the ambiguity. Apple sells Macs too. That’s where I’d hope they would train their models.

Re: Ferret: A Multimodal Large Language Model

#55

Post by para_parolu »

aaronbrethorst wrote: post #33
yreg wrote: post #21

I really hope Apple releases an iPhone with a good on-device private LLM assistant, perhaps next year. Their hardware is well-positioned for it. It could make me get a new phone outside of my usual ~4 year cycle. Siri is almost unusable for me.

Rumors suggest they’re gearing up to make iOS 18 an AI focused release. It’ll be interesting to see if they offer different capabilities for online/offline scenarios, or if their offerings are strictly offline. Here’s one story to offer some context. There are others. https://archive.is/en3VL

I really hope they will make siri usable. In the current state it’s only good for fixed phrases. And even then it fails time to time

Re: Ferret: A Multimodal Large Language Model

#57

Post by andy99 »

One big plus if this takes off as a base model is the abundance of weasel family animals to use in naming the derivatives. Ermine, marten, fisher, ... I'd like to call Wolverine. Llama didn't have much room for some interesting variety beyond alpaca and vicuna.

Re: Ferret: A Multimodal Large Language Model

#58

Post by tambourine_man »

hhh wrote: post #48
tambourine_man wrote: post #45

> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, Apple Silicon

Why would they dogfood Apple Silicon for training models? Seems like a waste of developer time to me.

Apple doesn’t even sell NVidia cards on their Mac Pros. Are they training it on Linux?

I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried.

AI/LLM is the new hot thing. If people are using Windows or Linux, you’re loosing momentum, hearts and minds… and sales, obviously.

Re: Ferret: A Multimodal Large Language Model

#59

Post by behnamoh »

andy99 wrote: post #57

One big plus if this takes off as a base model is the abundance of weasel family animals to use in naming the derivatives. Ermine, marten, fisher, ... I'd like to call Wolverine. Llama didn't have much room for some interesting variety beyond alpaca and vicuna.

Yes, because that's the main concern and limitation in the LLM community. /s

If anything, I think people should use meaningful and relevant names, or invent new ones.

Re: Ferret: A Multimodal Large Language Model

#60

Post by reissbaker »

jurmous wrote: post #34
CaptainOfCoit wrote: post #24

Earlier quoted context omitted.

You're unlikely to get a better experience with Siri if she becomes equipped with a 7B or 13B LLM, unless Apple figured out something revolutionary.

Released 2 days ago by Apple, a research paper on methods to run larger llms on iPhones. https://www.macrumors.com/2023/12/21/apple-ai-researchers-ru... https://arxiv.org/pdf/2312.11514.pdf

The paper was definitely cool but doesn't allow you to run particularly large LLMs on iPhones. It allows you to run a certain kind of LLM (sparse ReLU based LLMs) whose weights are somewhere less than 2x RAM. So, 7b Falcon works, but the competitive-with-gpt-3.5-turbo LLMs are still out of reach (and aren't ReLU based, although maybe that could change in the future). And nothing is competitive with GPT-4 right now.

Of course in the long run I think it will happen — smaller and more efficient models are getting better regularly, and Apple can also just ship their new iPhones with larger amounts of RAM. But I'd be very surprised if there was GPT-4 level intelligence running locally on an iPhone within the next couple years — that sized model is so big right now even with significant memory optimizations, and I think distilling it down to iPhone size would be very hard even if you had access to the weights (and Apple doesn't). More likely there will be small models that run locally, but that fall back to large models running on servers somewhere for complex tasks, at least for the next couple years.

Post reply on HN