MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
1–10 of 65 posts
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#2I wonder if this has anything to do with their acquisition of DarwinAI. After a decade of mediocrity, I'd love to see Siri get smarter. Any improvement would be welcome at this point.
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#3This looks competitive against CLIP, and surprisingly great at VQA style prompts, but it doesn't seem like the paper supports comparing it to GPT-4. We don't see any tests for coding performance, math homework, legal document review, or any of the myriad other things that people use GPT-4 for on a daily basis.
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#4The paper lists "first authors", "core authors", and "senior authors".
My dream is to one day be listed on a seminal paper as "secondary forum reply author".
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#5This looks competitive against CLIP, and surprisingly great at VQA style prompts, but it doesn't seem like the paper supports comparing it to GPT-4. We don't see any tests for coding performance, math homework, legal document review, or any of the myriad other things that people use GPT-4 for on a daily basis.
Besides homework, all of these things seem to be professional uses of GPT-4. If they’re trying to bake this into a consumer platform like Siri, I don’t see why they’d need to focus on those use cases. Besides MDM/Enterprise, which will be curious if they try and attack this market or just their army of consumer devices.
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#6The paper explores different design choices for various parts of the model and draws conclusions about the relative importance of optimizing each area (image encoder very important, vision-language connector less so).
The actual set of models produced (up to 30B parameters) seems secondary to the intent of the paper, and is more validation of the best design choices in each area.
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#7MM1 is a research paper, not a release of a competing product. I'm sure the paper is interesting and am looking forward to reading an analysis of it by someone who understands these things better than I do, but this is not that analysis, it's an extremely low-effort puff piece that is more interested in getting attention than in accurately describing a research paper.
I don't usually say this, but TFA frankly feels like it was written by AI:
> The release of MM1 by Apple contributes significantly to the artificial intelligence domain, offering a detailed roadmap for the development of future MLLMs. By sharing the insights and design principles gleaned from MM1, Apple not only challenges the current capabilities of models like ChatGPT but also invites the broader AI community to build upon their findings, potentially leading to more sophisticated and capable AI systems.
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#8Biggest model is 30b MoE trained on 100b tokens, max sequence length 4096. A bit underwhelming compared to recent announcements like the open source Large World Model [1].
Absolutely no benchmarks against GPT4 present in the paper.
Notably they used instruction response pairs generated from GPT4 for supervised fine tuning. Which has always felt like an experimental hack to me, but that’s how many folks are bootstrapping smaller models these days, and the effectiveness is hard to argue with.
Apple’s axlearn framework was used which leverages JAX and XLA [2].
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#9MM1 is a research paper, not a release of a competing product. I'm sure the paper is interesting and am looking forward to reading an analysis of it by someone who understands these things better than I do, but this is not that analysis, it's an extremely low-effort puff piece that is more interested in getting attention than in accurately describing a research paper. I don't usually say this, but TFA frankly feels l…
I believe most run-of-the-mill marketing language will sound like it is written in AI. The easiest thing to do for technology writing is to write the complete, factual article, then ask an LLM to dumb it down to whatever level you need for communication.
Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
#10The paper lists "first authors", "core authors", and "senior authors". My dream is to one day be listed on a seminal paper as "secondary forum reply author".
Holy inferiority complex batman!
You can aspire higher and just use one of these LLMs to be a "first author" in a published peer reviewed paper.