evals or didn't happen
Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
11–18 of 18 posts
Re: Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
#12Section 7 Conclusion and Future Work
We show that ReaLM outperforms previous approaches, and performs roughly as well as the state of-the-art LLM today, GPT-4, despite consisting of far fewer parameters, even for onscreen references despite being purely in the textual domain. It also outperforms GPT-4 for domain-specific user utterances, thus making ReaLM an ideal choice for a practical reference resolution system that can exist *on-device without compromising on performance.*
-----------------------------
Emphasis mine.
As far as I can tell, the model isn't exactly on-device yet.
Re: Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
#13I recently noticed Apple stock is struggling quite a bit, and then I see things like this pop up more often recently, where the company (or "someone") talks about what's going to happen and what they're working on internally, rather than actual releases and demonstration of work "done". Would love to see a bit more "walk the walk" from Apple.
It reads to me like the Google announcements. Platitudes to calm investors down so they can say "we are doing AI too see!". But I think Apple is really struggling, similar to Google. Dropping sales in China doesn't help either...
It’s not an announcement, and the headline is a lie that has nothing to do with Apple.
Apple published an academic paper, as they have been doing for years, showing a model that objectively outperforms GPT-4 on one narrow test. Nowhere do they claim to outperform GPT-4 in general.
Assuming that this is somehow intended to manipulate the stock price is not a rational analysis of what is going on.
Also, Apple has operated on a predicable release schedule for a decade now. They have already said that they will be announcing their AI work at WWDC. It’s unclear what anyone is expecting them to do differently or why it would make any sense.
As for the stock price - the explanations as to why it’s depressed are twofold - one is regulatory action, and the other is frenzied investors who want to jump on the AI bandwagon.
Dropping sales in China is a headline like a stopped clock. Usually wrong, sometimes right, and we only find out which at earnings time.
Note that nobody’s AI product is doing great yet, so for the most part it’s all performative, something that Apple doesn’t engage in.
Re: Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
#14[deleted]
Re: Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
#15Earlier quoted context omitted.
It reads to me like the Google announcements. Platitudes to calm investors down so they can say "we are doing AI too see!". But I think Apple is really struggling, similar to Google. Dropping sales in China doesn't help either...
It’s not an announcement, and the headline is a lie that has nothing to do with Apple. Apple published an academic paper, as they have been doing for years, showing a model that objectively outperforms GPT-4 on one narrow test. Nowhere do they claim to outperform GPT-4 in general. Assuming that this is somehow intended to manipulate the stock price is not a rational analysis of what is going on. Also, Apple has opera…
Also, unlike most other companies, Apple is really pushing for ML/AI on device. And while there are serious limitations for that, I really don't like to be beholden to opaque profit-at-all-costs companies like Google or OpenAI or...
Re: Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
#16All I want in my Apple product is to be able to choose my provider for AI services.
What? You're more than welcome to download an app, or use any company's website.
And what will that app do? Nothing can answer the queries by default except Siri.
There are extremely awkward workarounds by telling Siri to run a shortcut, but they are workarounds, and due to weird tech choices they don't work well: https://twitter.com/dmitriid/status/1640692625597505537
Re: Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
#17[dupe]
Some more discussion: https://news.ycombinator.com/item?id=39901952
Re: Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4
#18Earlier quoted context omitted.
It’s not an announcement, and the headline is a lie that has nothing to do with Apple. Apple published an academic paper, as they have been doing for years, showing a model that objectively outperforms GPT-4 on one narrow test. Nowhere do they claim to outperform GPT-4 in general. Assuming that this is somehow intended to manipulate the stock price is not a rational analysis of what is going on. Also, Apple has opera…
Also, unlike most other companies, Apple is really pushing for ML/AI on device. And while there are serious limitations for that, I really don't like to be beholden to opaque profit-at-all-costs companies like Google or OpenAI or...
[deleted]