Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data accidentally exposed by Microsoft AI researchers

wiz.io

Post reply on HN
Print view View original

181–190 of 238 posts

Re: Data accidentally exposed by Microsoft AI researchers

#181

Post by alaxapta7 »

dylan604 wrote: post #74
cj wrote: post #33

Earlier quoted context omitted.

> it’s why frequent pentests are important. Unfortunately a lot of pen testing services have devolved into "We know you need a report for SOC 2, but don't worry, we can do some light security testing and generate a report for you in a few days and you'll be able to check the box for compliance" Which is guess is better than nothing. If anyone works at a company that does pen tests for compliance purposes, I'd recomme…

I recently ran into something along the lines of your devolved pentest concept. I have a public facing webapp, and the report came back with a list of "critical" issues that are solved by yum update. Nothing about vulnerability to session jacking or anything along the lines of requiring actual work. I was a few steps removed from the actual testing, so who knows what was lost in translation and it being the first tim…

I've seen worse. Couple years back, there was an audit that included an internal system I've been working on. It was running on Debian oldstable because of a vital proprietary library I wasn't able to get working on stable at the time, but it had unattended upgrades set up and all that.

The company made some basic port scan and established that we're running outdated and vulnerable version of Apache. I found the act of explaining the concept of backports to a "pentester" to be physically painful.

They didn't get paid and another company was entrusted with the audit.

Re: Data accidentally exposed by Microsoft AI researchers

#182

Post by pixl97 »

anonymousDan wrote: post #141
saurik wrote: post #38

A number of replies here are noting (correctly) how this doesn't have much to do with AI (despite some sentences in this article kind of implicating it; the title doesn't really, fwiw) and is more of an issue with cloud providers, confusing ways in which security tokens apply to data being shared publicly, and dealing with big data downloads (which isn't terribly new)... ...but one notable way in which it does implic…

For me it's also interesting as a potential pathway for data poisoning attacks - if you have control over the data used to train a production model, can you modify the dataset such that it inserts a backdoor to any model trained subsequently trained over it? E.g. what if gpt was biased to insert certain security vulnerabilities as part of its codegen capabilities?

In theory for any AI model that generates code you'll want to have a series of post generation tests, for example something like SAST and/or SCA that ensure the model is not biasing itself to particular flaws.

At least for common languages this should stand out.

Where it gets more tricky is watering hole attacks against specialized languages or certain setups. This said you'd have to ensure that this data is not already there scraped up from the internet.

Re: Data accidentally exposed by Microsoft AI researchers

#183

Post by spullara »

buro9 wrote: post #5

Part of me thought "this is fine as very few could actually download 38TB". But that's not true as it's just so cheap to spin up a machine and some storage on a Cloud provider and deal with it later. It's also not true as I've got a 1Gbps internet connection and 112TB usable in my local NAS. All of a sudden (over a decade) all the numbers got big and massive data exfiltration just looks to be trivial. I mean, obvious…

Not really a sales pitch as it wasn't discovered by their product but rather by their security team doing a bunch of manual work.

Re: Data accidentally exposed by Microsoft AI researchers

#184

Post by LinuxBender »

rft wrote: post #159
LinuxBender wrote: post #18

Earlier quoted context omitted.

That modem setup in Wargames is still a thing for many organizations including some banks and telcos. Not naming names but I suspect the modems will be around for a very long time. Some have a password on their modem but they are usually very simple. Their only saving grace is that they are usually in front of a mainframe speaking proprietary MML that only old fuddy duddies like me would remember. There are a few of…

> proprietary MML that only old fuddy duddies like me would remember. Security through obscurity helps only until someone gets curious/determined. I have a personal anecdote for that. During university I was involved in pentesting an industrial control system (not in an industrial context, but same technology) and implemented a simple mitm attack to change the state of the controls while displaying the operator selec…

Security through obscurity helps only until someone gets curious/determined.

Absolutely. It just weeds out the skiddies and tools like MetaSploit unless they have added mainframe support. I have not kept up with their libraries

The federal agencies I was liaison to knew all the commands better than I did and even taught me a few that were not in my documentation which led to a discussion with the mainframe developers.

Re: Data accidentally exposed by Microsoft AI researchers

#185

Post by pixl97 »

alaxapta7 wrote: post #181
dylan604 wrote: post #74

Earlier quoted context omitted.

I recently ran into something along the lines of your devolved pentest concept. I have a public facing webapp, and the report came back with a list of "critical" issues that are solved by yum update. Nothing about vulnerability to session jacking or anything along the lines of requiring actual work. I was a few steps removed from the actual testing, so who knows what was lost in translation and it being the first tim…

I've seen worse. Couple years back, there was an audit that included an internal system I've been working on. It was running on Debian oldstable because of a vital proprietary library I wasn't able to get working on stable at the time, but it had unattended upgrades set up and all that. The company made some basic port scan and established that we're running outdated and vulnerable version of Apache. I found the act…

This is why I always attempt to turn off as much version information output as possible from any service. Make the pentester do their homework and not just look at "Apache 2.XX"

Hopefully you also have an internal control that looks at actual package versions installed on the server.

Re: Data accidentally exposed by Microsoft AI researchers

#186

Post by pixl97 »

nbk_2000 wrote: post #156
mymac wrote: post #46

Earlier quoted context omitted.

Pentests where people actually get out of bed to do stuff (read code, read API docs etc) and then try to really hack your system are rare. Pentests where people go through the motions, send you report with a few unimportant bits highlit while patting you on the back for your exemplary security so you can check the box on whatever audit you're going through are common.

If you're a large company that's actually serious about security, you'll have a Red Team that is intimately familiar with your tech stacks, procedures, business model, etc. This team will be far better at emulating motivated attackers (as well as providing bespoke mitigation advice, vetting and testing solutions, etc.). Unfortunately, compliance/customer requirements often stipulate having penetration tests performed…

Ugh, in the work I do I run into so much of this kind of stuff.

Customer: "We had a pentest/security scan/whatever find this issue in your software"

Me: "And they realized that mitigations are in place as per the CVE that keep that issue from being an exploitable issue, right"

Customer: "Uhhhh"

Testing group: "Use smaller words please, we only click some buttons and this is the report that gets generated"

Re: Data accidentally exposed by Microsoft AI researchers

#188

Post by EGreg »

alphabetting wrote: post #121
EGreg wrote: post #89

This seems to be a common occurrence with Big Tech and Big Government, so we better get used to it: https://qbix.com/blog/2023/06/12/no-way-to-prevent-this-says... https://qbix.com/blog/2021/01/25/no-way-to-prevent-this-says...

Is this stuff regularly happening to AWS and GCP? This is like the 3rd insane security incident from Microsoft in the past year.

https://www.bleepingcomputer.com/news/security/top-secret-us...

https://www.engadget.com/amp/2018-07-18-robocall-exposes-vot...

Ok so it’s not Microsoft exposing Microsoft, but government exposing its S3 buckets.

The question should be — why is all that data and power concentrated in one place? Because of the capitalist system and Big Tech, or Big Government.

Personally I am rather happy when “top secret information” is exposed, because that I s the type of thing that harms people around the world more than it helps. The government wants to know who is sending you $600 but doesnt want to tell you how they spent trillions on shadowy “defense” contractors.

https://community.qbix.com/t/transparency-in-government/234

Re: Data accidentally exposed by Microsoft AI researchers

#189

Post by Kon-Peki »

LinuxBender wrote: post #66
FridayNightTV wrote: post #51

Earlier quoted context omitted.

> I suspect the modems will be around for a very long time. No they won't. 'Dial up' modems need a PSTN line to work. The roll out of full fibre networks means analogue PSTN is going the way of the dodo. You cannot get a new PSTN line anymore in Blighty. In Estonia and the Netherlands (IIRC) the PSTN switch off is already complete.

I should have restricted that statement to include the United States of America. PSTN's are still utilized, deployed and actively sold in most of the US. As a side note I recently tried to get a telco to remove a phone line and two poles and they refused to do it. Their excuse was that they might one day run fiber over it despite there already being a fiber network here. I hope they do as my fiber ISP really does nee…

> As a side note I recently tried to get a telco to remove a phone line and two poles and they refused to do it.

You need to align their incentives with yours: wait until it gets windy out, knock the poles down, and demand that they come fix it.

Re: Data accidentally exposed by Microsoft AI researchers

#190

Post by dylan604 »

pixl97 wrote: post #185
alaxapta7 wrote: post #181

Earlier quoted context omitted.

I've seen worse. Couple years back, there was an audit that included an internal system I've been working on. It was running on Debian oldstable because of a vital proprietary library I wasn't able to get working on stable at the time, but it had unattended upgrades set up and all that. The company made some basic port scan and established that we're running outdated and vulnerable version of Apache. I found the act…

This is why I always attempt to turn off as much version information output as possible from any service. Make the pentester do their homework and not just look at "Apache 2.XX" Hopefully you also have an internal control that looks at actual package versions installed on the server.

This is actually a take away that I did implement. it's one of those that's not actively a vuln, but might provide info on what other attacks to try.
Post reply on HN