91–100 of 111 posts
Earlier quoted context omitted.
It doesn’t matter either way. Intel is an American company as well, and thus unsuitable as a trust root.
If you’re moving the goalposts from tech implementation to political vibes, it’s just more post-fact nabobism.
It’s not about vibes, but clear proof of a strategy to undermine global information security. Is anyone suppose to believe they don’t do that anymore?
Earlier quoted context omitted.
It doesn’t matter either way. Intel is an American company as well, and thus unsuitable as a trust root.
A company of what country would you prefer? Everyone likes to dunk on the US, but I doubt you could provide a single example of a country that is certainly a better alternative (to be clear I believe many of the west up in the same boat).
Earlier quoted context omitted.
Gimme an actual example instead of downvoting, help me learn. Edit on that too : makes me think of OpenAI Whisper as a service via /e/OS and supposedly anonymous proxying (by mixing), namely running STT remotely. That would be an actual potential usage... but IMHO that's low end enough to be run locally. So I'm still looking for an application here.
Are you looking for a general application of LLMs too large to run locally? Because anything you might use remote inference for, you might want to use privately.
- useful thing (according to someone specific requirements, maybe hallucinations are OK, maybe not) that
- needs privacy (for example generating code that will be open source probably does not need that)
- can't be run locally
- can be trusted to actually process as said it does
Earlier quoted context omitted.
Gimme an actual example instead of downvoting, help me learn. Edit on that too : makes me think of OpenAI Whisper as a service via /e/OS and supposedly anonymous proxying (by mixing), namely running STT remotely. That would be an actual potential usage... but IMHO that's low end enough to be run locally. So I'm still looking for an application here.
> would I actually run on it if I'm not a spammer? > Gimme an actual example instead of downvoting, help me learn. Basically you asked a bunch of people on a privacy minded forum, why should they be allowed to encrypt their data? What are you (they) hiding!? Are you a spammer??? Apple is beloved for their stance on privacy, and you basically called everyone who thinks that's more than marketing, a spammer. And before…
I specifically like the privacy aspect (even though honestly I think most people in this forum claim they do and yet they rely on BigTech with which they their data, so IMHO most people on HN are not as demanding as you describe) the question is precisely about what to run.
FWIW I specifically keep a page on self hosting AI (you can check if you want to see if it's real at https://fabien.benetou.fr/Content/SelfHostingArtificialIntel... ) so again, the privacy aspect is crucial to me.
The question is ... what to actually run. What can't be run locally that would be useful. What model for which tasks. For example coding (which I don't think models are good enough for) typically would NOT need this because one wouldn't share any PII over there, hopefully would even instead publish the resulting code as open source.
So my provocation about spammer is... because they often ARE actual users of LLMs. They use LLM for their language capabilities, namely craft a message that is always slightly different yet convey (roughly) the same meaning (the scam) to avoid detection. A random person though using LLM might NOT be OK with hallucinations when they use it for their own private journal or chat.
So... what for?
Edit: I did use Apple for years, recommending by someone at Mozilla, and I moved away from them earlier this year precisely because even though they are better than others, e.g Google, IMHO it's just not good enough for me. No intermediary is better than one with closed source.
Earlier quoted context omitted.
GCP can and does live migrate confidential VMs between machines. Which of the 50k machines in a cluster were you going to attach your analyzer to?
1) If you were GCP (as they are the attacker in this scenario), you'd attach the analyzer to ANY (!) ONE (!) server and then you migrate the user's workload that you wanted to snoop on (or were required to snoop on by the FBI) to your evil server. Like, you are clearly trying to say this makes it harder (though even if this were true that doesn't make it at all "provable")... but, if you support migration, you actual…
Earlier quoted context omitted.
well, also indefinite time and physical access.
Which is what the provider themselves have, by definition. The people who run these services are literally sitting next to the box day in and day out... this isn't "provably" anything. You can trust them not to take advantage of the fact that they own the hardware, and you can even claim it makes it ever so slightly harder for them to do so, but this isn't something where the word "provably" is anything other than a…
Earlier quoted context omitted.
> I came to understand that basically there's no HW that actually considers physical attacks in scope for their threat model? xbox, playstation, and some smartphone activation locks. Of course, you may note those products have certain things in common...
All things that were compromised with physical attacks? What are mod chips if not physical attack as a service?
Earlier quoted context omitted.
1) If you were GCP (as they are the attacker in this scenario), you'd attach the analyzer to ANY (!) ONE (!) server and then you migrate the user's workload that you wanted to snoop on (or were required to snoop on by the FBI) to your evil server. Like, you are clearly trying to say this makes it harder (though even if this were true that doesn't make it at all "provable")... but, if you support migration, you actual…
Apple’s PCC has an explicit design goal to mitigate your point 1, by designing the protocol such that the load balancer has to decide which physical server to route a request to without knowing which user made the request. If you compromise a single physical server, you will get a random sample of requests, but you can’t target any particular user, not even if you also compromise the load balancer. At least that’s th…
> targeting users should require a wide attack that’s likely to be detected
Regardless, Apple's attacker here doesn't sound like Apple: the "wide attack that's likely to be detected" is going to be detected by them. We even seemingly have to trust them that this magic hardware has the properties they claim it does.
This is way worse than most of these schemes, as if I run one of these on Intel hardware, you inherently are working with multiple parties (me and Intel).
That we trust Apple to not be lying about the entire scheme so they can see the data they are claiming not to be able to see is thereby doing the heavy lifting.
Earlier quoted context omitted.
Are you looking for a general application of LLMs too large to run locally? Because anything you might use remote inference for, you might want to use privately.
Sure that'd do, what - useful thing (according to someone specific requirements, maybe hallucinations are OK, maybe not) that - needs privacy (for example generating code that will be open source probably does not need that) - can't be run locally - can be trusted to actually process as said it does
The key is that these are all things I can verify without much difficulty: read over the script, spot-check the analysis, look at the claimed connection between A and B and see if it's real. And I don't really care about style, quality, maintainability.
You certainly can run this locally, but anything that will fit into reasonable local hardware won't be as good.
I don't need to trust it to process as it says it does, because I'm verifying the output.
And as far as I'm concerned, "needs privacy" is always true. I don't care if the code will be open source. I don't care if it's analyzing existing code that's already open source. Other people have no business seeing what I'm doing unless I explicitly allow it. In any case, I work on a lot of proprietary code as well, and my employer would be most displeased if I were exposing it to others.
Earlier quoted context omitted.
Just a shame they spent so long skimping on iPhone memory. The tail-end of support for 4gb and 6gb handsets is going to push that compute barrier pretty low.
Eh, maybe a bit, but those era devices also have much lower memory bandwidth. I suspect that the utility of client models will rule out those devices for other reasons than memory.
Not really? The A11 Bionic chip that shipped with the iPhone X has 3gb of 30gb/s memory. That's plenty fast for small LLMs if they'll fit in memory, it's only ~1/3rd of the M1's memory speed and it only gets faster on the LPDDR5 handsets.
A big part of Apple's chip design philosophy was investing in memory controller hardware to take advantage of the iOS runtime better. They just didn't foresee any technologies beside GC that could potentially inflate memory consumption.