Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft to delay release of Recall AI feature on security concerns

reuters.com

Post reply on HN
Print view View original

471–480 of 485 posts

Re: Microsoft to delay release of Recall AI feature on security concerns

#471

Post by tedivm »

simonw wrote: post #455
tedivm wrote: post #451

Earlier quoted context omitted.

> Microsoft says Recall lets you find anything you've seen or done on your PC with a simple search query, and it's powered by state-of-the-art large language models, which can understand various content on your PC, like text, images, and videos. It works in any application, so you can search across your computer. https://www.wired.com/story/everything-announced-microsoft-s...

Weird. I got the impression they were using embedding models (which I think of as LLM-adjacent) but not actual LLMs. See notes here: https://simonwillison.net/2024/Jun/5/ai-features-in-microsof...

I haven't actually dug into it myself, so it could just be that marketers or journalists got mixed up somewhere.

Re: Microsoft to delay release of Recall AI feature on security concerns

#472

Post by salawat »

onemoresoop wrote: post #330
red_admiral wrote: post #227

Earlier quoted context omitted.

I imagine MS did a lot of user studies, and found that the average user could gain a lot from being able to ask the computer questions like "where's the word document for the summer anniversary party that I worked on a couple of weeks ago" or "the photo with the waterfall from our holiday in Greece in 2015 that I sent to Mary recently". Whether Recall in 2024 will be good enough to answer queries like that remains to…

> We are, after all, in a world where the youth don't seem to understand file systems and folders [1] and rely on the search feature for everything. Recall could, if done properly, be a great user experience for such people. I think this was done on purpose to disempower the user.

Apps not files. It was a big push back in the 10's most embodied by mobile phone OS's. Instead of designing tools that dumped output to a common site, it all became about passing things around via things like intents.

Re: Microsoft to delay release of Recall AI feature on security concerns

#473

Post by cwillu »

ryandrake wrote: post #120
surfingdino wrote: post #93

Earlier quoted context omitted.

TPM was met with resistance due to privacy concerns and Microsoft quietly re-introduced it anyway. The same will happen to Recall.

It's the same playbook every company uses, who want to feed us something we don't like. They'll try again and again. Maybe they'll add sugar to the medicine, maybe they'll wave the spoon around and make airplane noises, maybe they'll distract us with a toy and jam the spoon in when we aren't expecting it, maybe they'll hold us down and give it as a suppository. One way or another, the baby is going to take the medici…

I remember back in the 90's when MS started making noises about leasing software.

Re: Microsoft to delay release of Recall AI feature on security concerns

#474

Post by cwillu »

panchoop wrote: post #438
ekidd wrote: post #171

Earlier quoted context omitted.

TPM protects against two main threat models: 1. You don't trust people with physical access to the computer. For the average home user, this means you consider the hardware owner a threat. 2. You want to protect against malware that has already taken complete control over the OS at runtime, and that wants to write itself to disk or the BIOS so that it survives a reboot. At this point, the attacker has already won, so…

I think you are missing some parts in the industrial use. The TPM is also used for device authentication. It prevents the leakage of certificates that are used to ensure that you are using the device you claim to be using. This is highly relevant when having remote access from users and one would like to enforce tiering rules together with privileged access workstations. Furthermore, the second example in which "the…

So, like he said, the hardware owner is the threat being protected against.

Re: Microsoft to delay release of Recall AI feature on security concerns

#475

Post by cwillu »

kstrauser wrote: post #317
0xFEE1DEAD wrote: post #275

Earlier quoted context omitted.

This is disgusting. I did not know that Microsoft offers these tools to organizations. I'm honestly shocked that this exists. They'll 100% abuse preview to offer similar features in the future. Over the last years/decade, they worked hard to improve their image in the tech community, and I have to admit, it worked, at least for me. They've just lost all the respect I had for them.

I can't believe I'm saying this, but in Microsoft's defense , those controls are aimed at companies working in regulated industries. They're meant to help those companies prove they they're meeting their legal and/or contractual compliance obligations. For example, if your company works with healthcare information and is a HIPAA "covered entity", your customers will demand to see proof that you're using data loss pre…

They call out a bunch of not-relevant-to-compliance uses in the marketing copy, so they lose any good will they might have otherwise maintained.

It's one thing to say “we offer this sketchy service to verified members of this highly regulated industry”, it's quite another to say “this is what that highly regulated industry uses to do the sketchy things they're required to do, and you can get it too!”

Re: Microsoft to delay release of Recall AI feature on security concerns

#476

Post by hn_throwaway_99 »

hnpolicestate wrote: post #123

You know what would be catastrophically bad? A Recall AI feature being baked into Android. Like most people don't actually use personal computers anymore, even laptops aren't common among demos younger than millennials. I can tolerate switching to Linux or buying a steam deck. But if this became a hard coded feature of android or iOS I'd have to give up smartphones entirely.

> Like most people don't actually use personal computers anymore, even laptops aren't common among demos younger than millennials.

Is that actually true? I'd imagine people actually creating things on their computer would at least need the equivalent of a Chromebook, which I would consider a laptop.

Re: Microsoft to delay release of Recall AI feature on security concerns

#477

Post by ranger_danger »

johnisgood wrote: post #441
ranger_danger wrote: post #329

Earlier quoted context omitted.

That's already true for every desktop application though. All third party programs can spy on all other programs and documents that user has available. This has been a seemingly criminally-overlooked shortcoming of desktop systems and this approach has fallen WAY behind current mobile security practices.

That is why "firejail" exists.

The attack surface of firejail itself scares me enough to never use it.

Re: Microsoft to delay release of Recall AI feature on security concerns

#478

Post by saagarjha »

nofunsir wrote: post #144
neltnerb wrote: post #108

Earlier quoted context omitted.

I do not think it is cynical to assume that Microsoft would sell this to companies as a way to do constant surveillance of their employees with OCR and LLMs used to make it easier for a manager to sift through massive amounts of data. That's just an actual use case that their true customers would pay for, I think it's awful and should be illegal under any reasonable worker protections but why would they not advertise…

It's exactly this. Development of a feature like this surely started during the WFH craze, where managers could no longer casually walk behind people who had to have their monitors facing outwards. A market opened up, and this is not the only tool for this sort of corporate surveillance. Certain Software Engineers will probably get some time without it by claiming they need Admin rights and that the system messes up…

Why do you think Recall was designed for this? This seems like a far-fetched assumption to make.

Re: Microsoft to delay release of Recall AI feature on security concerns

#479

Post by saagarjha »

panchoop wrote: post #438
ekidd wrote: post #171

Earlier quoted context omitted.

TPM protects against two main threat models: 1. You don't trust people with physical access to the computer. For the average home user, this means you consider the hardware owner a threat. 2. You want to protect against malware that has already taken complete control over the OS at runtime, and that wants to write itself to disk or the BIOS so that it survives a reboot. At this point, the attacker has already won, so…

I think you are missing some parts in the industrial use. The TPM is also used for device authentication. It prevents the leakage of certificates that are used to ensure that you are using the device you claim to be using. This is highly relevant when having remote access from users and one would like to enforce tiering rules together with privileged access workstations. Furthermore, the second example in which "the…

Verifying the integrity of the boot process is irrelevant. Nobody actually cares about this.

Re: Microsoft to delay release of Recall AI feature on security concerns

#480

Post by saagarjha »

pjmlp wrote: post #113
digging wrote: post #55

This is confusing and vague to me, which I believe is exactly the intent. It focuses on security, reiterates that security is their top priority (and we know that this is untrue). What were the security problems? They don't even allude to the existence or detection of any specific security problems. It sounds to me like they're figuring out a new marketing approach, or they're softening the blow by "listening to user…

Currently I am still looking forward to when the Secure Future Initiative (SFI) will actually mean more .NET and Rust and less COM and C++ love by Windows team. So until this changes, take with a grain of salt how much secure Recall is actually going to be. Contrast this with Apple Inteligence, where not only are most local APIs made available via Swift, they have created special hardware and a unikernel like OS with…

Focusing on the language used to implement these is missing the forest for the trees
Post reply on HN