Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

arxiv.org

Post reply on HN
Print view View original

51–60 of 65 posts

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#51

Post by astrange »

ksubedi wrote: post #25
erulabs wrote: post #23

If it’s going to take general artificial intelligent to get a voice assistant that can remember not one, but two entirely separate cooking timers, then so be it. Imagine the GPUs required! I’m still baffled at Siri and Google assistant. Virtually zero innovation in a decade. I just want to be able to turn on BBC radio while my hands are wet, is that really so hard?!

Google Assistant is pretty decent. But as someone who is pretty much locked into the Apple ecosystem, Siri needs a reboot from scratch.

It's been reportedly rewritten from scratch like five times, during which time people have not stopped posting claims that it's exactly the same as it was in 2010.

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#52

Post by MBCook »

CharlesW wrote: post #39
epaga wrote: post #17

Earlier quoted context omitted.

Mediocrity is far too positive a word for the dumpster fire that is Siri.

The characterization of "mediocre" is fair, but we're transiting a household to Siri from Alexa (because Alexa doesn't work locally, and because of Amazon's track record on privacy), and it's not noticeably worse.

The feeling I’ve heard from people is Alexa was way better than Siri at first.

Over time Siri got better. Not great but better. Alexa had mostly stayed the same or perhaps gotten a touch worse except for adding ads and other annoyances.

I’ve never used anything but Siri. It works decently, definitely has its moods/dumb-as-a-post moments. But I’ve learn what works well and for that it’s proven very useful.

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#53

Post by MBCook »

bestnameever wrote: post #42
erulabs wrote: post #23

If it’s going to take general artificial intelligent to get a voice assistant that can remember not one, but two entirely separate cooking timers, then so be it. Imagine the GPUs required! I’m still baffled at Siri and Google assistant. Virtually zero innovation in a decade. I just want to be able to turn on BBC radio while my hands are wet, is that really so hard?!

You should be able to do this with Siri. You can use a shortcut if it doesn't work out of the box.

It works out of the box as of iOS 16 or 17.

“Hey Siri set an egg timer for 4 minutes”

The interface for switching between multiple timers sucks on the watch, the whole app does now. I don’t know how it’s handled on HomePods, though you can see them somewhere in the home app (yeah that’s discoverable).

But it works fine. And the interface is good on the phone.

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#54

Post by MBCook »

Rinzler89 wrote: post #35
CharlesW wrote: post #34

Earlier quoted context omitted.

https://www.tomsguide.com/how-to/how-to-set-up-and-manage-mu... “How many timers can you have going at one time? […] …I had 26 timers going at once, and the only reason I didn't have more running was because I got bored.”

I wonder if the maximum number of timers is an 8 bit, 16 bit or 32 bit int.

Only one horrifically boring way to find out.

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#55

Post by bionhoward »

Aqua_Geek wrote: post #18
a_vanderbilt wrote: post #2

I wonder if this has anything to do with their acquisition of DarwinAI. After a decade of mediocrity, I'd love to see Siri get smarter. Any improvement would be welcome at this point.

Honest question: what do you (in the general sense, not specifically asking the parent) use Siri for? I think my main (only?) use case is setting a timer. Maybe I find conversational UIs awkward, or maybe I just got jaded REALLY quickly from Siri’s lacking capabilities early on, but I have hardly used it in the decade or whatever that it’s been around.

Since they removed “hey” and I got the latest phone, I’ve noticed many little situations where it’s faster to speak to the device than tap your way around. E.G. when it’s locked you can say, “Siri, open Spotify” and look at it for face unlock, boom. Random stuff. Also Alexa has surprised me lately, like a rational response to, “how many sandwiches is too many?”

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#56

Post by Etheryte »

refibrillator wrote: post #8

Biggest model is 30b MoE trained on 100b tokens, max sequence length 4096. A bit underwhelming compared to recent announcements like the open source Large World Model [1]. Absolutely no benchmarks against GPT4 present in the paper. Notably they used instruction response pairs generated from GPT4 for supervised fine tuning. Which has always felt like an experimental hack to me, but that’s how many folks are bootstrapp…

You seem to be missing what this submission is about. It's not an Apple press release about a competing model, it's a research paper that discusses different tradeoffs in architecture and data and how each part affects the results of the trained model. In an era where training a large model can be cost prohibitive, this insight is key — it tells you where to optimize and where to cut corners to get the most bang for your buck.

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#58

Post by jwells89 »

fauigerzigerk wrote: post #41
zshrc wrote: post #5

Earlier quoted context omitted.

Besides homework, all of these things seem to be professional uses of GPT-4. If they’re trying to bake this into a consumer platform like Siri, I don’t see why they’d need to focus on those use cases. Besides MDM/Enterprise, which will be curious if they try and attack this market or just their army of consumer devices.

They are going to have to focus on the use cases that most of their customers use LLMs for, regardless of whether it falls in the consumer or professional category or somewhere in between. If all it does is improve Siri a bit without massively expanding the range of applications and APIs it will be a big disappointment. I think what Apple presents in June will decide whether on-device AI will be seen as a viable alte…

Many users of Siri would be thrilled if all this did was made it decent at understanding what’s being asked of it and gave it the ability to ask clarifying questions, especially if it does so staying fully local.

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#59

Post by jwells89 »

ribosometronome wrote: post #21
azinman2 wrote: post #19

Earlier quoted context omitted.

I use it almost daily for something that is simple but under appreciated I don’t know why it’s not in every marketing video: “Siri, remind me tomorrow at 10am to do X” I outsource so much of my memory to the phone via Siri ALL THE TIME. It’s so useful. Even for things in 20m. I’ll easily forget if I don’t do this, and it’s reliable so it gives me confidence. It also keeps the notification present until I actually do…

Especially with Shortcuts, Siri can have some pretty useful functionality. My personal big improvement I'd like to see is being able to better able to tap into those actions without having to set things up in advance.

In addition to Shortcuts, being an Apple thing, Siri naturally has native HomeKit integration which is powerful when combined with HomeAssistant.

Re: MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training

#60

Post by nl »

This is an awesome paper, and the somewhat negative sentiment in the discussion here is surprising.

The ablation studies are well done, comprehensive and expensive to do. People will be using the conclusions from this for years, and that is much more impactful than if an upcoming Siri product ourperforms the GPT model at that same point in time.

A few really interesting points:

Synthetic datasets substantially (1%+) increase performance for Image Encoder Pre-training

Architecture of the VisualLanguage model connector doesn't seem to matter.

Interleaving text and image data improves few shot performance, but image captioning data improves zero-shot numbers.

The ideal mix of data types is 5:5:1 for Interleaved:Captions:Plain Text (!)

Synthetic captioning data helps substantially at this point too (up to 4% gain)

The appendices are amazing: lots of details about learning rates tried, batch sizes.

The "explain these figures" are really really good. See page 37.

Post reply on HN