Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data accidentally exposed by Microsoft AI researchers

wiz.io

Post reply on HN
Print view View original

191–200 of 238 posts

Re: Data accidentally exposed by Microsoft AI researchers

#191

Post by lawlessone »

wolftickets wrote: post #150
saurik wrote: post #38

A number of replies here are noting (correctly) how this doesn't have much to do with AI (despite some sentences in this article kind of implicating it; the title doesn't really, fwiw) and is more of an issue with cloud providers, confusing ways in which security tokens apply to data being shared publicly, and dealing with big data downloads (which isn't terribly new)... ...but one notable way in which it does implic…

Disclosure I work for the company that released this: https://github.com/protectai/modelscan but we do have a tool to support scanning many models for this kind of problem. That said you should be using something like safe-tensors.

You have me curious now. The models generate text. Could a model hypothetically be trained in such a way that could create a buffer overflow when given certain prompts? I am guessing the way inference works in such a way that cant happen

Re: Data accidentally exposed by Microsoft AI researchers

#192

Post by cosmojg »

sneak wrote: post #179
monkpit wrote: post #22

Earlier quoted context omitted.

I strongly support the “no hello” concept but I also fear being seen as “that guy” so I never mention it. Sigh

Be that guy. In the long run it's better to be right then popular.

But then I might not survive the long run.

Re: Data accidentally exposed by Microsoft AI researchers

#193

Post by cosmojg »

fireflash38 wrote: post #129
monkpit wrote: post #22

Earlier quoted context omitted.

I strongly support the “no hello” concept but I also fear being seen as “that guy” so I never mention it. Sigh

I have seen people never ask their question after multiple days of saying "hello @user", despite having nohello as a status. And despite having asked them in the past to just ask their question and I'll respond when I can. You just can't win.

I'd count that as a win. You avoided wasting your time answering a potentially inane question. If it were important, they would have asked.

Re: Data accidentally exposed by Microsoft AI researchers

#194

Post by LinuxBender »

Kon-Peki wrote: post #189
LinuxBender wrote: post #66

Earlier quoted context omitted.

I should have restricted that statement to include the United States of America. PSTN's are still utilized, deployed and actively sold in most of the US. As a side note I recently tried to get a telco to remove a phone line and two poles and they refused to do it. Their excuse was that they might one day run fiber over it despite there already being a fiber network here. I hope they do as my fiber ISP really does nee…

> As a side note I recently tried to get a telco to remove a phone line and two poles and they refused to do it. You need to align their incentives with yours: wait until it gets windy out, knock the poles down, and demand that they come fix it.

I've been secretly hoping an over-sized big rig would take them out but I would not want anyone to get hurt. They are the only poles within a few miles and are an eye-sore.

Re: Data accidentally exposed by Microsoft AI researchers

#195

Post by wolftickets »

lawlessone wrote: post #191
wolftickets wrote: post #150

Earlier quoted context omitted.

Disclosure I work for the company that released this: https://github.com/protectai/modelscan but we do have a tool to support scanning many models for this kind of problem. That said you should be using something like safe-tensors.

You have me curious now. The models generate text. Could a model hypothetically be trained in such a way that could create a buffer overflow when given certain prompts? I am guessing the way inference works in such a way that cant happen

Absolutely, though that isn't strictly what we're talking about here.

In this case, models themselves are fundamentally files. These files can have malicious code embedded into them that is executed when the model is loaded for further training or inference. When executed it isn't obvious to the user at all. It's a very nasty potential vector.

I wrote a blog about it here: https://protectai.com/blog/announcing-modelscan

Re: Data accidentally exposed by Microsoft AI researchers

#196

Post by pixl97 »

Faelian2 wrote: post #136
j245 wrote: post #75

Earlier quoted context omitted.

From my understanding as a non security expert: Pentest comes across more as checking all the common attack vectors don’t exist. Getting out of bed to do the so-called “real stuff” is typically called a bug bounty program or security researching. Both exist and I don’t see why most companies couldn’t start a bug bounty program if they really cared a lot about the “real stuff”

I work as pentester (as a freelance nowdays). Getting out of bed and "real stuff" is supposed to be part of a pentest. The problem is more the sheer amout of stuff your are supposed to know to be a pentester. Most pentesters come into the field by knowing a bit of XSS, a few thing about PHP, and SQL injections. Then you start to work, and the clients need you to tests things like: - compromise a full Windows Network,…

>As a junior pentester, my biggest fear was always to hand an empty report.

I'm trying to remember the rule where you leave something intentionally misconfigured/wrong for the compliance people to find and that you can fix so they don't look deeper into the system. A fun one with web servers is to get them to report they are some ancient version that runs on a different operating system. Like your IIS server showing it's Apache 2.2 or vice versa.

But at least from your description it sounds like you're attempting to pentest. So many of these pentesting firms are click a button, run a script, send a report and go on to the 5 other tickets you have that day type of firms.

Re: Data accidentally exposed by Microsoft AI researchers

#197

Post by pixl97 »

FridayNightTV wrote: post #51
LinuxBender wrote: post #18

Earlier quoted context omitted.

That modem setup in Wargames is still a thing for many organizations including some banks and telcos. Not naming names but I suspect the modems will be around for a very long time. Some have a password on their modem but they are usually very simple. Their only saving grace is that they are usually in front of a mainframe speaking proprietary MML that only old fuddy duddies like me would remember. There are a few of…

> I suspect the modems will be around for a very long time. No they won't. 'Dial up' modems need a PSTN line to work. The roll out of full fibre networks means analogue PSTN is going the way of the dodo. You cannot get a new PSTN line anymore in Blighty. In Estonia and the Netherlands (IIRC) the PSTN switch off is already complete.

>'Dial up' modems need a PSTN line to work

Cable company here (US) still sells service that has POTS over cable modem. Just plug your modem into the cable modem tele slot and you have a dialton. Now, are you getting super high speed connections, no, but that's not what you need for most hacking like this. Not that I recommend hacking from your own house.

Re: Data accidentally exposed by Microsoft AI researchers

#198

Post by pixl97 »

quickthrower2 wrote: post #81
tremon wrote: post #29

Earlier quoted context omitted.

Encryption at rest does nothing to prevent online access to data. It's only useful if you leave your storage cabinet standing on the side of the road.

Your laptop backup could be encrypted. New problem: where to out the keys. Maybe another storage account with different access controls.

> New problem: where to out the keys.

If it's windows, Active Directory.

Re: Data accidentally exposed by Microsoft AI researchers

#199

Post by xbar »

sillysaurusx wrote: post #20

The article tries to play up the AI angle, but this was a pretty standard misconfiguration of a storage token. This kind of thing happens shockingly often, and it’s why frequent pentests are important.

AI data is highly centralized and not stored in a serially-accessed database, which makes it unusual inasmuch as 40TB of interesting data does not often get put into a single storage bucket.

Re: Data accidentally exposed by Microsoft AI researchers

#200

Post by sillysaurusx »

NikolaNovak wrote: post #26
sillysaurusx wrote: post #12

Earlier quoted context omitted.

How do you have your NAS configured? The more specifics, the better; I’ve wanted one. Do you worry about failure? In your hardware life I mean, not your personal life.

Not the OP, but after a lot of messing with software software and OS RAID, Raid Cards and mother boards, dedicated loud Dell servers, UnRAID, this that and the other thing over years and decades, I just set up a big Synology device 5 years ago. Since then, I've had a NAS that just worked. I have data, it's there. I do online backup to a cloud provider, and a monthly dump to external USB drives that I keep and rotate…

Which model Synology do you have? (Would you still make the same choice today?)

Did you settle on using RAID, or just rely on cloud backups?

Post reply on HN