Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's On-Device and Server Foundation Models

machinelearning.apple.com

Post reply on HN
Print view View original

311–320 of 562 posts

Re: Apple's On-Device and Server Foundation Models

#311

Post by jan3024 »

t-writescode wrote: post #297
jeroenhd wrote: post #287

Earlier quoted context omitted.

I would've thought the same until Microsoft started hiding tank man results in Bing. I'm not so sure if companies will start training different models for every oppressive regime.

Oh? https://www.bing.com/images/search?q=tank%20man Looks visible, to me. Tiananmen Square even shows Tank Man on the first page, 13th and 15th entry, for me. Admittedly, I expected it more quickly on Tiananmen Square; but that might be because I, as a person, forgot that it's also a literal square with more stuff going on at it than a single moment in history.

[dead]

Re: Apple's On-Device and Server Foundation Models

#312

Post by Hugsun »

mixtureoftakes wrote: post #114
teonimesic2 wrote: post #95

Earlier quoted context omitted.

I believe it is because llama 3 8B beats it, which would make it look bad. The phi-3-mini version they used is the 4k which is 3.8B, while LLama 3 8B would be more comparable to phi-3 small (7B) which also considerably better than phi-3-mini. Likely both phi-3 small and llama 3 8B had too good results in comparison to Apple's to be added, since they did add other 7B models for comparison, but only when they won.

llama 3 definitely beats it, but 99% of the users wont care which is actually a good thing... apple totally wins the ai market not by being sota but by sheer amount of devices which will be running their models, we're talking billions

How is any of this good? Apple serves its captive users inferior models without giving them a choice. I don't see how that is winning the AI market either.

Re: Apple's On-Device and Server Foundation Models

#314

Post by franzb »

rishabhjain1198 wrote: post #166

For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.

This isn't about AI research, it's about delivering AI at unimaginable scale.

Re: Apple's On-Device and Server Foundation Models

#315

Post by kanwisher »

ofou wrote: post #171

Quite interesting this was released right after multiple rants from Elon sparked debates on X. "If Apple integrates OpenAI at the OS level, then Apple devices will be banned at my companies. That is an unacceptable security violation." Replying to Tim Cook: "Don’t want it. Either stop this creepy spyware or all Apple devices will be banned from the premises of my companies." "It’s patently absurd that Apple isn’t sma…

Apple made their own AI models, only in certain cases it will ask if you want to send it to OpenAI. Presumably other ai companies later can integrate into API. But this is very privacy safe, if you use an iPhone already it already indexes all your photos and OCRs them for easy searching, on device ...

Re: Apple's On-Device and Server Foundation Models

#316

Post by bt1a »

wpm wrote: post #188
l33t7332273 wrote: post #162

Earlier quoted context omitted.

Why do they not?

AFAIK there is no general purpose, "do this on the ANE" API. You have to be using specific higher level APIs like CoreML or VisionKit in order for it to end up on the ANE.

This, plus metal acceleration works quite well. 7~8B parameter models quantized to 3bpw or so run with good tok/s on my iphone 15 pro

Re: Apple's On-Device and Server Foundation Models

#317

Post by vishnugupta »

ksec wrote: post #124

I hope, this could mean Apple will push the baseline of ALL Macs to have higher than 8GB of Memory. While I wish we all get 16GB M4 as baseline. Apple being Apple may only give us 12GB, and charges extra $100 for the 16GB option. It will still be a lot better than 8GB though.

Does it matter what the baseline memory is as long as they have 16GB M4 as an option?

Re: Apple's On-Device and Server Foundation Models

#318

Post by sofixa »

mrweasel wrote: post #296
throwaway4good wrote: post #290

Earlier quoted context omitted.

The M2 is a chip designed to be in a laptop (and it is quite powerful given its low power consumption). Presumedly they have a different chip or at least completely different configuration (RAM, network, etc.) in their data centers.

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the…

That's probably where Microsoft's "Copilot+ PCs" come in.

Re: Apple's On-Device and Server Foundation Models

#319

Post by manmal »

vishnugupta wrote: post #317
ksec wrote: post #124

I hope, this could mean Apple will push the baseline of ALL Macs to have higher than 8GB of Memory. While I wish we all get 16GB M4 as baseline. Apple being Apple may only give us 12GB, and charges extra $100 for the 16GB option. It will still be a lot better than 8GB though.

Does it matter what the baseline memory is as long as they have 16GB M4 as an option?

Some companies give their employees only base models.

Re: Apple's On-Device and Server Foundation Models

#320

Post by binkethy »

overstay8930 wrote: post #174
tonynator wrote: post #172

Earlier quoted context omitted.

Those who dislike censorship and enjoy hacking avoid iPhones for obvious reasons.

People who understand cybersecurity hygiene use iPhones for obvious reasons

It would seem that integrating a backdoor funnel to OpenAI is a bit of a security issue to those who care about such things.

Yay, we can all train corporate models for free involuntarily.

I guess it's time to check out Lineage OS and Postmarket OS. It was always a matter of time.

Post reply on HN