To all people who ask moral side of this tech and that it can be abused. What do you think about tools like Kali Linux? NMAP? Basically any pentesting software can be abused.
Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
31–40 of 336 posts
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#32Earlier quoted context omitted.
I too would like to imagine whether or not I would release something like this in similar circumstances. Yet this question seems unfairly loaded. At the least - premature. Though we can easily imagine it, afaik there is no evidence of any actual abuse to date.
Tools like these will end up in the hands of those that are trying to harden an installation and in the hands of those that will end up using them to try to break into such installations. I've often wondered whether the net effect of the whole security research community is a net positive or a net negative and I honestly do not know the answer. So yes, it is a loaded question. But asking yourself if what you can do i…
In the end, I decided that one oppressed person using it to improve their situation is morally "worth" many criminals using it. It's kind of like "better ten guilty men go free than put one innocent man behind bars".
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#33from the article: Real-time, controllable deepfakes ready for virtual camera injection. Created for performing penetration testing against e.g. identity verification and video conferencing systems, for the use by security analysts, Red Team members, and biometrics researchers. Reminds one of the vulnerable world hypothesis. It's only a matter of time before a technology comes along that is both destructive and so sim…
In cybersecurity the bad guys always win. There are two very rough "proofs of a kind" I use to explain this to students, both well known in military and counter-terrorism analysis. One is called Blotto analysis. As the number of fronts grows large (>12 is an important point) the game favours the attackers. Defenders have to always win on all fronts, but terrorists only have to win once on any of n fronts. Another sce…
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#34Earlier quoted context omitted.
Tools like these will end up in the hands of those that are trying to harden an installation and in the hands of those that will end up using them to try to break into such installations. I've often wondered whether the net effect of the whole security research community is a net positive or a net negative and I honestly do not know the answer. So yes, it is a loaded question. But asking yourself if what you can do i…
I used to work on secure communications software, and I've often wondered if many more criminals than oppressed people were using it. In the end, I decided that one oppressed person using it to improve their situation is morally "worth" many criminals using it. It's kind of like "better ten guilty men go free than put one innocent man behind bars".
Because the one innocent is usually a defender, whereas the guilty men are usually the attackers and to give the attackers an effective advantage in an arms race is a risky thing with unknown and potentially devastating outcomes.
In my own life (the live video example listed elsewhere) it would have meant that we probably would have had everything that we have today anyway, only maybe a little bit (not even that much, I'm aware of one other individual who was working on a similar concept who contacted me after our release) later.
Video conferencing, live streams in the browser without plug ins and so on all would have happened, for sure. But at least the massive mountain of abuse cases would not rest partially on my shoulders. And because I've been confronted with the direct evidence of the results of my creation for me that link is easy to make. But if you work on secure communications software you are probably not aware of the consequences.
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#35> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#36Earlier quoted context omitted.
Thank you for a considered response. I share those feelings. I just hope the person offering AMA will not be daunted by a good but blunt question from a voice that carries weight around here.
I've been in that position, which is one of the reasons why I'm asking. When I came up with 'live streaming video on the www' I never for one second sat down to think about the abuse potential. Color me hopelessly naive. And when confronted with the various abuses over the years I've always had a problem with that, this was the direct consequence of me just 'scratching my itch' and it caused a huge amount of misery.…
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#37Earlier quoted context omitted.
It's the same as the age old "guns don't kill people" that seems to work fine for the weapons industry in the US.
The world is larger than the US, and even there that argument does not hold water: https://www.nytimes.com/2022/02/15/nyregion/sandy-hook-famil...
That’s not remotely the same thing as holding a creator liable for the misuse of his work by unrelated third-parties.
Frankly, the illiberal push towards “ban everything that could be misused” is disheartening, especially when we see it applied to a speech-based tool like this one.
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#38Earlier quoted context omitted.
I've been in that position, which is one of the reasons why I'm asking. When I came up with 'live streaming video on the www' I never for one second sat down to think about the abuse potential. Color me hopelessly naive. And when confronted with the various abuses over the years I've always had a problem with that, this was the direct consequence of me just 'scratching my itch' and it caused a huge amount of misery.…
May I ask, what proportion of the stuff you did not release on moral grounds were subsequently re-invented in short order?
Which is one of the reasons I'm so completely against software patents and a large number of patents in general. Quite a few of them are simply things that the time is right for.
Here is one of my idea dump lists, you can check for yourself which ones are not yet done (which is probably a really small number by now) and which ones have turned out to be homeruns (and in some cases billion dollar+ companies).
https://jacquesmattheij.com/my-list-of-ideas-for-when-you-ar...
One that wasn't on there eventually led to https://pianojacq.com/, which I'm happy to report to date has not led to any kind of abuse. And no, it did not put any piano teachers out of business either.
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#39> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…
Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)
#40> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…
> If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I don't agree, but people will never reach consensus on this moral topic. So please don't call the other side "at best naive".