281–290 of 384 posts
Earlier quoted context omitted.
But they did not introduce security vulnerabilities as there prevented the bad patches from being merged
Those measures were insufficient, as according to the developers, a number of the bogus patches were merged and had to be reverted.
Earlier quoted context omitted.
It's an insightful review into some of the issues and regulations around this. I'm not American, so our local rules are different, and I resigned from my institution's IRB several years ago, and these issues have become more fraught in this period. However, the way I was trained to look at these issues were around the concepts of harm, both actual and potential. In this particular case, as Dittrich notes, the questio…
Thank you for such detailed response. Can there be ethical possibility (highly remote one) where an study (assuming it is objectively justified) conducted with deception, but without prior informed consent at all. (eg. human subjects will not know that there's time is used for another purpose)
E.g. I the researcher ask if you consent to spending 30min completing a series of tasks to sort objects by their shape, presumably because I want to study your ability to recognize shapes. However, what I am actually studying is the group dynamics, of how well you and others in the group cooperate or have conflict over your tasks.
Earlier quoted context omitted.
This suggestion, if taken to it's logical conclusion in my mind, I should send an email to every open source project maintainer ever warning them that at some point in the future I might attempt to validate their 2FA settings or try to submit mildly malicious code to them to test their ability to do code review at some unknown date in the future. Maybe this helps to raise awareness, even though in reality I will do t…
> even though in reality I will do tests like this to a tiiiiny percentage of repos over the next decade So...contrary to what you suggest in your first paragraph, all you need to do is send out a handful of warning emails at the appropriate juncture. What’s the problem? For sure you shouldn’t be submitting malicious code to a project without forewarning. That's just a basic ethical no-no. Even if you were right that…
It will make it hard for things to improve, so millions will continue to get hacked due to supply chain attacks that almost no one is doing anything about.
At what point does the ethical fault of negligence become so great that intervening to stop said negligence without consent becomes justified? We don't have much in the way of well defined law in this area yet, and it may take a long time to catch up.
I suggest the lesser evil in the short term is to allow maintainers to suffer occasional mild embarrassment from time to time because they merged bad code. They get a valuable lesson out of the deal with no one getting hurt, no privacy violated, etc.
If a maintainers don't want to review code with the understanding it could be malicious, then they have the right to never review or merge anything.
All code contributions should be considered potentially malicious.
Now, we also don't want to see repos bombarded with bad commits either if we can help it.
Maybe a middle ground could exist where open source authors can signal a willingness to be tested once a year, and can signal if a test for this year has happened already.
I think most projects would not do this, but it might be a step towards wide consent long term if someone cares to build such a system.
In the decades before that though, there are not really any good options.
If someones house is on fire and it has the potential to burn the neighborhood down, I don't imagine you should be faulted for doing what is required to put it out.
Emergencies sometimes warrant dispensing with niceties. I think the supply chain attack risk is approaching emergency levels at this point. Negligence is the norm.
Earlier quoted context omitted.
Most projects have virtually no accountability at all. Hundreds of people have blind push access to brew. Almost no one signs code. One bad commit and you backdoor every company in silicon valley that blindly executes the unsigned code from the brew repo. Pick a favorite Linux distro. All but a handful work the same way. Right now an low skill level malicious party just needs to phish one of the ~90% of maintainers t…
I agree it's insane but this is what the market optimizes for. I pushed for more security and better practices until I was fired from 3 companies. It's quite obvious that business only cares about PR; if a big security incident happens they are more interested in how to reduce liability and not how to prevent the problem in the first place [the next time around]. Until programmers become an elite clique with a huge b…
I feel I am happier and have a wider net impact now that I can choose to invest less time with clients that can't afford to be super interested in security right now and invest more time in those that are.
Weirdly companies seem to listen to security consultants more than their own internal full time security advocates.
Earlier quoted context omitted.
I agree it's insane but this is what the market optimizes for. I pushed for more security and better practices until I was fired from 3 companies. It's quite obvious that business only cares about PR; if a big security incident happens they are more interested in how to reduce liability and not how to prevent the problem in the first place [the next time around]. Until programmers become an elite clique with a huge b…
I know this struggle well. Survivors bias is an epidemic. It is one of the reasons I started my own security consulting firm. I feel I am happier and have a wider net impact now that I can choose to invest less time with clients that can't afford to be super interested in security right now and invest more time in those that are. Weirdly companies seem to listen to security consultants more than their own internal fu…
Earlier quoted context omitted.
This suggestion, if taken to it's logical conclusion in my mind, I should send an email to every open source project maintainer ever warning them that at some point in the future I might attempt to validate their 2FA settings or try to submit mildly malicious code to them to test their ability to do code review at some unknown date in the future. Maybe this helps to raise awareness, even though in reality I will do t…
> Do any security researchers ask permission before finding other vulnerabilities in random open source code? I have never observed this. That's not what happened. Nobody is asking (or is expected to ask) when they look for vulnerabilities. The researchers were trying to introduce vulnerabilities. Had they only looked at the tools that maintainers use and given those a good shake to see whether an attack vector is lu…
There also might be middle ground where people submit benign code that clearly demonstrates highly risky behavior the reviewer should have noticed.
Maybe I introduce code that covertly exfiltrates memory from a non sensitive address, but it is enough to understand I could have chosen a more sensitive one. Or code that simply executes a ping to my server, etc.
Anyway, the point is, intent matters.
Earlier quoted context omitted.
I know this struggle well. Survivors bias is an epidemic. It is one of the reasons I started my own security consulting firm. I feel I am happier and have a wider net impact now that I can choose to invest less time with clients that can't afford to be super interested in security right now and invest more time in those that are. Weirdly companies seem to listen to security consultants more than their own internal fu…
Off-topic but got interested in what you do and how and if you are happy with the financial results (I gathered you are satisfied with your work already). Would you mind an email message?
Earlier quoted context omitted.
For me, it's not learning the lesson the first time around. I completely agree their experiment is unethical. However, it's not actually clear cut to most researchers the ethical bounds of their work, especially for study papers that's never really been explored before. Ethics in of itself is largely a active subtopic for many areas in CS, not only security research. AI is one area where qualifying potential harm to…
I could not disagree more. The ethics around research that involves deception have been pretty well established and are are several good comments here explaining them. Every scientist is personally respsible for the ethics of the research they conducts. Full Stop, no caveats allowed. If your research is in an area where the ethics are controversial or grey, that means your need to spend MORE time considering the ethi…
Ethics in computing research remains an active research area. This incident will be used as a case study in the future, but it's not that well established. Many people have been using anecdotals, which honestly don't fit the scenario because so many variables and parameters distinguish other types of pentesting from this. And disappointingly, not a single post has actually produced the documents that establish this.
Arguably the first set of guidelines for ethics in computer security research [1] was published in 2012 and not yet widely taught in Ethics lectures (I only know about it because I learned Computer Security from one of the authors).
On identifying harms:
> "Challenges identifying harms in ICTR environments stem from the scale and rapidity at which risk can manifest, the difficulty of attributing research risks to specific individuals and/or organizations, and our limited understanding of the causal dynamics between the physical and virtual worlds. As with all exploratory research, it can be challenging to articulate benefits such that subjects can make informed decisions. In ICTR our ability to qualitatively and quantitatively foresee the probable benefits is particularly immature."
On this type of research:
> "Research of criminal activity often involves deception or clandestine research activity, so requests for waivers of both informed consent and post hoc notification and debriefing may be relatively common as compared with research studies of non-criminal activity."
This isn't a huge change from 30 years ago since Moor [2] wrote his thesis on Computer Ethics, see:
> "A typical problem in computer ethics arises because there is a policy vacuum about how computer technology should be used. Computers provide us with new capabilities and these in turn give us new choices for action. Often, either no policies for conduct in these situations exist or existing policies seem inadequate. A central task of computer ethics is to determine what we should do in such cases, i.e., to formulate policies to guide our actions."
Researchers themselves are far from educated on this topic; you won't ever explore this in depth unless you're in this particular sub-field. IRB/REB boards are considered the most qualified but are possibly too outdated to navigate around this. It's a whole mess, there is currently a lot of questionable research in many areas of computing, but the clock moves forward.
[1] https://www.dhs.gov/sites/default/files/publications/CSD-Men...
[2] https://web.cs.ucdavis.edu/~rogaway/classes/188/spring06/pap...
Earlier quoted context omitted.
Just clicking the link is enough to fail? No need to enter private info or execute downloaded files? Either your phising emails are badly crafted or you expect employees to see the future.
Agreed. This is something that made me very annoyed when they did it to us. I know exactly what I'm doing, I know that you can't infect a computer from opening a link unless the attacker possesses a new browser 0-day, and expecting your average employee to worry about new browser 0-days is ridiculous. If it were an obvious phishing URL, like a variation on the company domain, then fine (maybe). But it wasn't.
Earlier quoted context omitted.
Really? I unequivocally think it was unethical research, and thus a mistake. But what’s the appropriate proportional response? I honestly have a hard time thinking firing a bunch of people over this would be a good outcome for anyone. What’s the end goal? I feel like already there’s been enough action to deter this kind of sneaky research in the future. But at some point you’re just going to scare away the ethical re…
Non-academics get fired for doing unethical things entirely unrelated to their jobs all the time, just to avoid bad publicity. Determining how to conduct your research ethically is a core part of being a researcher. Basic research into pen testing and security research would have revealed how unethical this study was. I see firing as completely justifiable given that was a failure in a core part of thier job, that no…
Yes, only if it's unrelated to their jobs. People get hired, not fired, to do unethical research in industry labs. Ethics is breached all the time in industry - rarely even considered. Google tried to make amends, but decided caring too much about ethics was a roadblock to their goals. Tesla markets their "FSD" at the risk of other drivers in the road. The only difference here is that companies have a bigger, better legal and PR team. Comparing the moral compass between academia and industry is absolutely ridiculous.