Earlier quoted context omitted.
I don't think the LoRAs are fine-tuned locally at all. It sounds like they use RAG to access data.
Consider a feature from earlier in the keynote: the thing Notes (and Math Notes) does now where it fixes up your handwriting into a facsimile of your handwriting, with the resulting letters then acting semantically as text (snapping to a baseline grid; being reflowable; being interpretable as math equations) but still having the kind of long-distance context-dependent variations that can't be accomplished by just gen…
Apple's On-Device and Server Foundation Models
211–220 of 562 posts
Re: Apple's On-Device and Server Foundation Models
#212It would be interesting to see how these models impact battery life. I’ve tried a few local LLMs on my iPhone 15 Pro via the PrivateLLM app, and the battery charge plummets just after a few minutes of usage.
Re: Apple's On-Device and Server Foundation Models
#213Re: Apple's On-Device and Server Foundation Models
#214Will these smaller on device models lead to a crash in GPU prices?
Not in the short-to-medium-term. Try the local models out, they fall over pretty quickly, even if you have 64GB+ of VRAM.
If you ask it for knowledge, like a comparison of vacuum cleaner models then yes, it's a hallucination fest. They just don't have the parameters for this level of detail. This is where ChatGPT is really king.
But if you give them the data they need with RAG, they're not bad. Acting on commands, looking stuff up in provided context, summarising all perform pretty well. Which seems to be also what Apple is targeting to do with them.
Re: Apple's On-Device and Server Foundation Models
#215Is it me or Apple is really moving fast? I don't think it is easy for a company of this size to concisely put a vision of AI in these short and crazy AI times. BTW, not an Apple fan but an Apple user.
Is that moving fast? Maybe, compared to what, Oracle?
Re: Apple's On-Device and Server Foundation Models
#216Earlier quoted context omitted.
"AI for the rest of us."
Except Apple isn't really for the rest of us. Outside of America and a handful wealthy western countries it's for the top 5-20% earners only.
Ref: https://worldpopulationreview.com/country-rankings/iphone-ma...
Re: Apple's On-Device and Server Foundation Models
#217Earlier quoted context omitted.
Consider a feature from earlier in the keynote: the thing Notes (and Math Notes) does now where it fixes up your handwriting into a facsimile of your handwriting, with the resulting letters then acting semantically as text (snapping to a baseline grid; being reflowable; being interpretable as math equations) but still having the kind of long-distance context-dependent variations that can't be accomplished by just gen…
For everything other than handwriting I don't think the LoRAs are fine-tuned locally.
I can't picture any way to use a RAG to do that.
I can picture a way to do that that doesn't involve any model fine-tuning, but it'd be pretty ridiculous, and the results would probably not be very good either. (Load a static image2text LoRA tuned to describe the subjects of photos; run that once over each photo as it's imported/taken, and save the resulting descriptions. Later, whenever a photo is classified as a particular subject, load up a static LLM fine-tune that summarizes down all the descriptions of photos classified as subject X so far, into a single description of the platonic ideal of subject X's appearance. Finally, when asked for a "memoji", load up a static "memoji" diffusion LoRA, and prompt it with the that subject-platonic-appearance description.)
But really, isn't it easier to just fine-tune a regular diffusion base-model — one that's been pre-trained on photos of people — by feeding it your photos and their corresponding metadata (incl. the names of subjects in each photo); and then load up that LoRA and the (static) memoji-style LoRA, and prompt the model with those same people's names plus the "memoji" DreamBooth-keyword?
(Okay, admittedly, you don't need to do this with a locally-trained LoRA. You could also do it by activating the static memoji-style LoRA, and then training to produce a textual-inversion embedding that locates the subject in the memoji LoRA's latent space. But the "hard part" of that is still the training, and it's just as costly!)
Re: Apple's On-Device and Server Foundation Models
#218For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.
This gives me the vibe of calling high resolution screens as "retina" screens.
Re: Apple's On-Device and Server Foundation Models
#219Earlier quoted context omitted.
It's a huge development in terms of it being a consumer-ready, on-device LLM. And if Karpathy thinks so then I assume it's good enough for HN: https://x.com/karpathy/status/1800242310116262150
[flagged]
Re: Apple's On-Device and Server Foundation Models
#220Earlier quoted context omitted.
The Llama 3 license says: "If, on the Meta Llama 3 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights unde…
They can just run the same tests and cite the results from other websites. That has nothing to do with Meta. No companies can force you to not talk about them.