Live data from Hacker News

Ferret: A Multimodal Large Language Model

github.com

31–40 of 332 posts

Re: Ferret: A Multimodal Large Language Model

#31
post #21

I really hope Apple releases an iPhone with a good on-device private LLM assistant, perhaps next year. Their hardware is well-positioned for it. It could make me get a new phone outside of my usual ~4 year cycle. Siri is almost unusable for me.

You're unlikely to get a better experience with Siri if she becomes equipped with a 7B or 13B LLM, unless Apple figured out something revolutionary.

Have you ever actually used Siri?

Re: Ferret: A Multimodal Large Language Model

#33
post #21

I really hope Apple releases an iPhone with a good on-device private LLM assistant, perhaps next year. Their hardware is well-positioned for it. It could make me get a new phone outside of my usual ~4 year cycle. Siri is almost unusable for me.

Rumors suggest they’re gearing up to make iOS 18 an AI focused release. It’ll be interesting to see if they offer different capabilities for online/offline scenarios, or if their offerings are strictly offline.

Here’s one story to offer some context. There are others. https://archive.is/en3VL

Re: Ferret: A Multimodal Large Language Model

#34
post #21

I really hope Apple releases an iPhone with a good on-device private LLM assistant, perhaps next year. Their hardware is well-positioned for it. It could make me get a new phone outside of my usual ~4 year cycle. Siri is almost unusable for me.

You're unlikely to get a better experience with Siri if she becomes equipped with a 7B or 13B LLM, unless Apple figured out something revolutionary.

Released 2 days ago by Apple, a research paper on methods to run larger llms on iPhones.

https://www.macrumors.com/2023/12/21/apple-ai-researchers-ru... https://arxiv.org/pdf/2312.11514.pdf

Re: Ferret: A Multimodal Large Language Model

#35

Maybe the abstract of the paper is a better introduction to what this is: > We introduce Ferret, a new Multimodal Large Language Model (MLLM) capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions. To unify referring and grounding in the LLM paradigm, Ferret employs a novel and powerful hybrid region representation that integrates d…

>>spatial referring

I can't seem to nail down the meaning of this phrase on its own. All the search results seem to turn up are "spatial referring expressions".

Re: Ferret: A Multimodal Large Language Model

#37
post #13

We're watching Apple fill the moat in.

OpenAI can just copy this.

Yes, OpenAI can copy this, but they’ll still have less of a moat. That’s the problem with moats, once they’re gone even if you copy what others do, you don’t have a moat anymore.

Think of it in a physical sense. OpenAI is a high walled castle surrounded by a physical moat. This protects them and their business model. Apple comes along and builds a super tall tower right next to the moat. They can now see into OpenAI’s castle, fire arrows, catapult in a giant wooden badger, etc. Even if Open AI copies the design of Apple’s really tall tower and built it behind the moat and castle walls, it wouldn’t do much because Apple still would be able to get stuff over the moat and walls. The moat doesn’t matter anymore for the most part. The castle (OpenAI) can be compromised and needs bigger walls, relocating to someplace with a bigger, or a way of attacking the tower (Apple). Copying doesn’t really accomplish any of those three.

Re: Ferret: A Multimodal Large Language Model

#39
post #22

Earlier quoted context omitted.

They cannot integrate it deeply into Apple's platforms.

Hence OpenAI is looking at hardware themselves. https://www.reuters.com/technology/chatgpt-owner-openai-is-e... https://techcrunch.com/2023/09/27/openai-is-reportedly-in-ta...

It does not matter much if they do not make competitor for iPhones and get consumers to choose them. Because consumers keep buying iPhones and will have then Apple hardware.

And they cannot bring software ecosystem for their hardware without Google, at least easily.

Re: Ferret: A Multimodal Large Language Model

#40
post #36

I wonder if these models are trained to have some kind of identification in case you use them for non-research purposes for example. "Tell me who is your manufacturer" for example

From Bard:

My situation is a bit unique, so the term "manufacturer" might not be the most accurate way to describe who created me. Here's a breakdown of what you need to know:

    Developed by Google AI: I was created by a team of researchers and engineers at Google AI, specializing in language models and artificial intelligence.
    Trained on a massive dataset: My knowledge and abilities come from being trained on a massive dataset of text and code, containing books, articles, code, and other forms of information.
    Continuously learning and evolving: I'm still under development, constantly learning and improving as I interact with users and process new information.
So, while I don't have a single manufacturer in the traditional sense, I'm the result of collaboration and advancement in AI research and development at Google.

I hope this helps clarify things! Let me know if you have any other questions.

Post reply on HN