Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

211–220 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#212
post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

That's like saying that nuclear weapons exist, and bad actors can potentially get them, so let's lower the bar so that anyone can. Making such tools accessible is reprehensible. It will lead to more bad actors, to less trust in media and in any objective reality, and more erosion of our institutions and society. There is absolutely no reason whatsoever for this. It's unethical and frankly downright evil.

But who are the actors worse that the mafia that is already in control?

Institutions that are corrupt and serve themselves (not the public, despite their lipservice) need to go.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#213
post #55

Earlier quoted context omitted.

Unless we cut off the chain of responsibility somewhere, the creators of the programming language, the computer and its components, as well as the people who have designed the computer and those who mined the required minerals are responsible as well.

> Unless we cut off the chain of responsibility somewhere We can do that. We do it all the time, assigning varying levels of blame to different parties for different things. Determining between proximal and ultimate causes, or assigning more weight to one cause or another, is not some impossible burden.

[deleted]

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#214

Earlier quoted context omitted.

If I stab someone with a knife, who is responsible? The inventor of knives? The knife‘s manufacturer? The store who sold me the knife? I would say the responsibility lays 100% with myself. I do not think it makes sense to pursue inventors for what happens to their creations, unless they actively encourage misuse.

They arrest people for making and selling meth everyday. You have a bias because making deep fakes is not illegal yet. Knives are used for cooking, what alternative good do deep fakes provide?

Cardiovascular diseases are the number of cause of death worldwide BY FAR. Millions of people die every year from consuming fast food. Fast food does not really serve a purpose apart form being tasty/quick/cheap. You could fulfill your dietary needs much better with different food.

Should McDonald's be banned? I do not think so. Anything can be misused.

To answer your question about what "good" uses there are for deep fakes: Cheaper and better special effects, available to a MUCH broader audience than before. I cannot even imagine all of the great use cases creative people will find for this.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#215

Earlier quoted context omitted.

Oh no! We didn't remind our users to be kind to each other! We're software developers, not kindergarten teachers. How do you feel about Sqlite's code of conduct? Because if you don't like that, then you shouldn't be making moral statements in your own repositories.

Kindergarten teachers, as people who can have significant positive or negative impact, are bound by codes of ethics and practice. This is also true of almost every other area of human effort, whether implicit or explicit. As a catch-all baseline, we have the various legal systems. The fact that a significant proportion of the tech industry refuses point-blank to even discuss the idea of ethics is not a sign of superi…

[deleted]

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#216
post #150

Earlier quoted context omitted.

As someone with a stalker, I can't emphasize this enough. A stalker will go to all sorts of lengths to do bizarre shit. People don't believe it. I would guess governments will do some equivalent there-of. Democratizing access to things -- including bad things -- has a preventative effect: 1) I can guard against things I know about 2) People take me seriously if something has been democratized The worst-case scenario…

I want to second this, as there are so many people that just simply can't believe how much time and energy some people will put into destroying someone else's life. And when you ask for help, people think you are the insane one because they simply can't believe your story about the insanity of someone else. I hope you find relief sometime from your stalker. I found it (not a stalker exactly) from letting the person b…

A worse situation: What if you ARE insane, it's just that this time it's not your imagination running wild, it's actually someone stalking you? People that have some mental problem, whatever it is, have a much harder time since no one will believe them. It's always just "I know you are feeling bad right now..." even though the diagnose is ADHD or something else irrelevant.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#217

Earlier quoted context omitted.

Over time the number of single humans that have the power to wipe out a sizeable portion of life on Earth (which is almost certainly non-zero already) will almost certainly increase due to advances in technology and dissemination of information. Do we aim for security through obscurity or work out how to deal with that increasingly terrifying reality? I think at some point we will be forced to do latter.

This trend is clearly real, but what exactly do you mean “work out how to deal with that?” One of the obvious ways to deal with it, albeit imperfectly, is in fact to keep these technologies as obscure as possible for as long as possible. Maybe an interesting norm to try to generate in this community would be, “if you plan on publishing attack X, you should publish immediately alongside it your best guess as to the ef…

Even an extreme example -- The technology to build an automated drone with a camera for targeting and a gun on a gimbal and deploy it to do a mass-shooting -- all exists today. All it would take is for some madman to pick the pieces up and assemble them. In fact, I'd be willing to bet $100 that we'll actually see some kind of tragedy to that effect in the USA within the next 5 years. The technology cats are very quickly getting out of the bags.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#219

Earlier quoted context omitted.

> Isn't the point to prove physical access to the camera. Yes. But since you can't really prove who had and who did not have physical access to the camera (what are you going to use, video evidence ;) ) that carries little weight. Let's say we're talking about a death penalty or a multi billion $ lawsuit, the stakes would be so high that to rely on something as trivial as a signature on a video stream would be way to…

> Which expert would sign on the dotted line to the statement that 'the equipment has not been accessed by someone who wasn't authorized to do so'? You have a lot more trust in the ethics of expert witnesses than I do.

Well, expert witnesses that give false testimony are a lot more rare than expert witnesses that are going to go on the record for things that they know to be false.

Expert witnesses and liability are a very interesting area of the legal system. There are all kinds of variations on this theme ranging from blanket immunity to various forms of liability. Knowingly giving false testimony would likely lead to problems, especially if say a panel of experts would in turn testify that said expert should have known and could have known that what they testified was wrong.

I'm not aware of a case that went that far, but I would love to read more about such conflicts between expert witnesses on two sides of a lawsuit, usually the experts agree in broad terms on their testimony, and may differ in the details.

Maybe some who have served in such a capacity can chime in here?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#220

Earlier quoted context omitted.

Thank you for a considered response. I share those feelings. I just hope the person offering AMA will not be daunted by a good but blunt question from a voice that carries weight around here.

I've been in that position, which is one of the reasons why I'm asking. When I came up with 'live streaming video on the www' I never for one second sat down to think about the abuse potential. Color me hopelessly naive. And when confronted with the various abuses over the years I've always had a problem with that, this was the direct consequence of me just 'scratching my itch' and it caused a huge amount of misery.…

One thing I learned early in my career, and numerous times during it: For every ethical stand you take against writing a bit of software you consider questionable, there's a line of other software engineers out the door willing to do it. I remember when as a junior engineer, I worked up the courage to tell my boss I had a moral problem with writing some code that would help the product cheat at a benchmark. He totally understood, and I didn't get fired or anything--just moved on to a different project. Bob, two cubicles down, was more than happy to write the benchmark-cheating code.

Software engineers and other technology creators don't take a "Do No Harm" oath like doctors. Many of them have never even taken a single Ethics In Technology course at university (it was an optional class when I was in undergrad decades ago). And, even in the alternate universe where ethics was baked into engineering training, all it takes is a single rogue willing to ignore them, and now the world has to deal with it.

Post reply on HN