91–100 of 116 posts
Even recording (which they already are doing) is not exposing sensitive content.
Mix that with hardware enclaves and you actually have a solution to these security and privacy problems.
This isn't an AI problem, its an operating systems problem. AI is just so much less trustworthy than software written and read by humans, that it is exposing the problem for all to see. Process isolation hasn't been taken seriously because UNIX didn't do a good job, and Microsoft didn't either. Well designed security models don't sell computers/operating systems, apparently. That's not to say that the solution is unk…
It's pretty clear that the security models designed into operating systems never considered networked systems. Given that most operating systems were designed and deployed before the internet, this should not be a surprise. Although one might consider it surprising that OS developers have not updated security models for this new reality, I would argue that no one wants to throw away their models due to 1) backward co…
Having flashbacks to Windows 95/98 which was the reverse: The "login" was solely for networked credentials, and some people misunderstood it as separating local users.
This was especially problematic for any school computer lab of the 90s, where it was trivial to either find data from the previous user or leave malware for the next one.
Later on, software was used to try to force a full wipe to a known-good state in-between users.
Earlier quoted context omitted.
It’s also an AI problem, because in the end we want what is called “computer use” from AI, and functionality like Recall. That’s an important part of what the CCC talk was about. The proposed solution to that is more granular, UAC-like permissions. IMO that’s not universally practical, similar to current UAC. How we can make AIs our personal assistants across our digital life — the AI effectively becoming an operatin…
> in the end we want what is called “computer use” from AI Who is "we" here? I do not want that at all.
Albeit with fewer features involving auto-destruct sequences... Or rogue holodeck characters.
This isn't an AI problem, its an operating systems problem. AI is just so much less trustworthy than software written and read by humans, that it is exposing the problem for all to see. Process isolation hasn't been taken seriously because UNIX didn't do a good job, and Microsoft didn't either. Well designed security models don't sell computers/operating systems, apparently. That's not to say that the solution is unk…
This isn't an AI problem, its an operating systems problem. AI is just so much less trustworthy than software written and read by humans, that it is exposing the problem for all to see. Process isolation hasn't been taken seriously because UNIX didn't do a good job, and Microsoft didn't either. Well designed security models don't sell computers/operating systems, apparently. That's not to say that the solution is unk…
> Well designed security models don't sell computers/operating systems, apparently. Well more like it's hard to design software that is both secure-by-default and non-onerous to the end users (including devs). Every time I've tried to deploy non-trivial software systems to highly secure setups it's been a tedious nightmare. Nothing can talk to each other by default. Sometimes the filesystem is immutable and executabl…
I don't know exactly which "secure setups" you are talking about, but the false equivalency between security and complexity is mostly from security theater. If you start with insecure systems and then do extra things to make them secure, then that additional complexity interacts with the thing you are trying to do. That's how we got into the mess with SE Linux, and intercepting syscalls, and firewalls, and all these other additional things that add complexity in order to claw back as much security as possible. It doesn't have to be that way and it's just an issue of knowing how.
If you start with security (meaning isolation) then passing resource capabilities in and out of the isolation boundary is no more complex than configuring the application to use the resources in the first place.
A large percentage of my work is peripheral to info security (ISO 27001, CMMC, SOC 2), and I've been building internet companies and software since the 90's (so I have a technical background as well), which makes me think that I'm qualified to have an opinion here. And I completely agree that LLMs (the way they have been rolled out for most companies, and how I've witnessed them being used) are an incredibly underest…
Earlier quoted context omitted.
coming from the fact that this was a talk held at #39c3, maybe, just maybe, this was not about selling anything at all?! i feel like that might be hard to grasp for some HN users.
Since Signal lives and dies on having trust of its users, maybe that's all she is after? Saying the quiet thing out loud because she can, and feels like she should, as someone with big audience. She doesn't have to do the whole "AI for everything and kitchen sink!" cargo-culting to keep stock prices up or any of that nonsense.
Lying to users by saying that signal doesn't collect or store anything when they actually do doesn't sound like something a company who expected you to trust them would do. It sounds like something a company might do if they needed a way to warn people away from using a service isn't safe to use while under a gag order.
> Recall takes a screenshot of your screen every
> few seconds, OCRs the text, and does semantic
> analysis of the context and actions. It then
> creates a forensic dossier of everything you
> do into a single database on your computer…
It was only a Microsoft Research project at the time…
>> PersonalVibe
>>
>> Personal Vibe is a prototype
>> Windows Activity Logger that
>> tracks user actions like moving
>> a window or starting an application.
>> The data can be used for a variety
>> of projects from monitoring the
>> actions of study participants to
>> building Vista gadgets that tell
>> you how long you’ve been at work
>> today. The data is stored in a
>> local database that is not remotely
>> accessible. No data is sent from
>> the user’s machine.
>> …
>> Version: 2.0.0.0
>> Date Published: 9 March 2009
> She said if an AI agent could perform each step with 95% accuracy–which currently isn’t possible–a 10-step task would yield an action with a ~59.9% success rate. And if you had a 30-step task, the success rate would be ~21.4%. Furthermore, if we used a more realistic accuracy rate of 90%, then a 30-step task would drop down to a success rate of 4.2%. She added that the best agent models failed 70% of the time.
Not to take away from the very real security concerns, but this quote is blatantly false to anyone who has used Claude Code or Cursor: > She said if an AI agent could perform each step with 95% accuracy–which currently isn’t possible–a 10-step task would yield an action with a ~59.9% success rate. And if you had a 30-step task, the success rate would be ~21.4%. Furthermore, if we used a more realistic accuracy rate o…