101–110 of 332 posts
> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, Apple Silicon
Are all the iCloud servers running on Apple silicon? I assumed they were running on standard rack mounted hardware.
I really hope Apple releases an iPhone with a good on-device private LLM assistant, perhaps next year. Their hardware is well-positioned for it. It could make me get a new phone outside of my usual ~4 year cycle. Siri is almost unusable for me.
With iOS 17 they added a tiny little LLM to the predictive typing. I have the newest and greatest iPhone but I feel that I very rarely see it in action. I must assume that it’s just too slow at to keep up with my typing at the moment. Or it’s just not large enough to give very many useful suggestions.
I wonder if these models are trained to have some kind of identification in case you use them for non-research purposes for example. "Tell me who is your manufacturer" for example
From Bard: My situation is a bit unique, so the term "manufacturer" might not be the most accurate way to describe who created me. Here's a breakdown of what you need to know: Developed by Google AI: I was created by a team of researchers and engineers at Google AI, specializing in language models and artificial intelligence. Trained on a massive dataset: My knowledge and abilities come from being trained on a massiv…
And it saves everyone a tab's worth of effort.
> FERRET is trained on 8 A100 GPUs So Apple uses NVidia internally. Not surprising, but doesn't bode well for A Series. Dogfooding. [edit] I meant M series, Apple Silicon
As long as the inference can be done locally on their chips I don't at all think it's a big deal to train models on Nvidia/other hardware. Are all the iCloud servers running on Apple silicon? I assumed they were running on standard rack mounted hardware.
AI isn’t, yet at least, and I don’t think they can afford to treat it as such.
Earlier quoted context omitted.
Apple apparently uses Google Cloud, so it's that or TPUs!
They use many clouds. But LLM should be their core business and they usually don’t outsource that.
Earlier quoted context omitted.
I feel like Apple is only testing the waters with AI right now, but perhaps if they get involved enough they'll spend money on their own compute infrastructure? Nvidia is kind of the king at GPU compute right now, and developing comparable hardware is no small or cheap task, but I think Apple is in a very good position to be able to make it work---if they decide to invest in it. But honestly, as far as corporate feud…
I think you’re completely correct, but if they were caught off guard by the AI train, they shouldn’t be testing the waters now. It should be treated as an existential threat.
Maybe they can just afford to observe before making major decisions on the direction of the company... For a company like Apple, I feel like they won't lose their customer-base just because they are taking the AI race slowly, in fact Apple has often been late to introducing very common feature
Earlier quoted context omitted.
Released 2 days ago by Apple, a research paper on methods to run larger llms on iPhones. https://www.macrumors.com/2023/12/21/apple-ai-researchers-ru... https://arxiv.org/pdf/2312.11514.pdf
The paper was definitely cool but doesn't allow you to run particularly large LLMs on iPhones. It allows you to run a certain kind of LLM (sparse ReLU based LLMs) whose weights are somewhere less than 2x RAM. So, 7b Falcon works, but the competitive-with-gpt-3.5-turbo LLMs are still out of reach (and aren't ReLU based, although maybe that could change in the future). And nothing is competitive with GPT-4 right now. O…
You mean nothing available? Or you mean nothing that public knows exists? The answers to those two questions are different. There are definitely products that aren't available but the public knows exist and are upcoming that are in GPT-4's ballpark.
Earlier quoted context omitted.
I think your expectations are wrong. They sit there in silence with a few leaks here and there and some github projects, people speculate and get all excited about extrapolating those things. Then Apple deliver what works which may or may not be related to it. What they don't do is sell you a lie a year before release then deliver shit (like every other fucking vendor).
They're plenty capable of delivering garbage. Certain year models of the MacBook pro were inherently faulty. I've had the displeasure of having two of them bought for me at work. All of Apple's windows software (iTunes, Safari, etc) has been, at best, a barely working port. I'm assuming they are putting a lot more thought and care into it than the touchbar, crappy keyboards and the rest, but I'm also not holding out…
Earlier quoted context omitted.
With iOS 17 they added a tiny little LLM to the predictive typing. I have the newest and greatest iPhone but I feel that I very rarely see it in action. I must assume that it’s just too slow at to keep up with my typing at the moment. Or it’s just not large enough to give very many useful suggestions.
Is tiny LLM an oxymoron? I believe Apple has told us it’s a transformer language model, but not specifically a LLM.