Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

121–130 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#121

Post by systemvoltage »

Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

This is morally corrupt, dangerous and would lead to an oppressive, violent society. A knife maker killed for murders, a watch maker killed for the tyranny of time.

We should do the exact opposite. Science and reason would cease to exist otherwise. Individuals wouldn't need to be held accountable, innovators/engineers/scientists/entrepreneurs would be. End of a free society.

Have you thought of the chilling effect this might bring?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#122

Post by pid-1 »

nonrandomstring wrote: post #20
isaacfrond wrote: post #2

from the article: Real-time, controllable deepfakes ready for virtual camera injection. Created for performing penetration testing against e.g. identity verification and video conferencing systems, for the use by security analysts, Red Team members, and biometrics researchers. Reminds one of the vulnerable world hypothesis. It's only a matter of time before a technology comes along that is both destructive and so sim…

In cybersecurity the bad guys always win. There are two very rough "proofs of a kind" I use to explain this to students, both well known in military and counter-terrorism analysis. One is called Blotto analysis. As the number of fronts grows large (>12 is an important point) the game favours the attackers. Defenders have to always win on all fronts, but terrorists only have to win once on any of n fronts. Another sce…

> Defenders have to always win on all fronts, but terrorists only have to win once on any of n fronts

It depends on how you design systems and processes.

Let Mn denote the nth defense mechanism of a system. We can build architectures in which the probability of breaches is calculated as:

(M0 is pwned) AND (M1 pwned) AND (M2 is pwned) AND (M3 is pwned).....

You can see the total probability can become small quickly, as it's computed as the product of all probabilites. The collorary is: it's possible to design secure systems from insecure building blocks (as long as those are reasonably uncorrelated!)

Insecure systems are generally designed the way you suggest in your post: if a single defense mechanism is breached, the whole system will be breached. So you basically are as secure as your weakest link.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#123

Post by jacquesm »

imiric wrote: post #110
jacquesm wrote: post #61

Earlier quoted context omitted.

I actually disagree with that. I think software and ethics are closely intertwined and too often we see them as disjoint. It's interesting to try to find a software project that does not have an ethically dubious angle to it, that isn't all that simple.

While it's an interesting discussion, would you ask the same question to a knife or sword maker? Your tone in this and the AMA thread is combative, and wouldn't lead to a productive discussion. It would be much more interesting to discuss the technical merits of this project than any particular moral concerns you might have, especially on this forum.

> Your tone in this and the AMA thread is combative, and wouldn't lead to a productive discussion.

That's mostly your impression. But that may be because you believe that I have a position with regards to this particular software, which I do not, I just see this as an opportunity to gain insight in the position of the author which I find interesting and which may help guide me in similar decisions in the future, because it's something that I've been wrestling with for a long, long time. Since 1995 in fact.

> It would be much more interesting to discuss the technical merits of this project than any particular moral concerns you might have, especially on this forum.

What you find more interesting and what I find more interesting do not necessarily have to be the same things, and you are totally welcome to ask your technical merits questions.

Finally, I don't like to be told what I can and can not discuss, especially not in a thread started by someone who wrote 'AMA' where the third A stands for 'Anything'.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#124

Post by hrgiger »

There is also work going on for deepfake detection challenges since years, I think forensics benchmark was the oldest one and other two placed at 2019 , I dont know if forensics still going on though

http://kaldir.vc.in.tum.de/faceforensics_benchmark/

https://www.kaggle.com/competitions/deepfake-detection-chall...

https://ai.facebook.com/blog/deepfake-detection-challenge-re...

edit*: looks like yes forensics top submission 25 Nov, 2021

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#125

Post by belter »

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

You are right. I saw some guy that looked like Matt Damon trying to sell me some crypto coins...

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#126

Post by Planktonne »

bjt2n3904 wrote: post #113
mensetmanusman wrote: post #107

Earlier quoted context omitted.

Definitely useful for scammers.

Then build a cultural revolution that instills distrust in what you see. We already teach the elderly not to trust a link that says, "you are the millionth visitor, click to win your prize". Teach them also to look past the video call and what looks (and sounds) like their grandson to what is actually being said. He's asking for your bank password, that behavior doesn't match! The cat is already out of the bag. Going…

I don't think anyone so far has called for this to be unpublished, just for it to be more carefully considered and for the ethical aspect to be addressed when publishing. That's part of 'building a culture of distrust'.

Knee-jerk refusal to even consider thinking/talking about the use of technology is far more irresponsible than any other stance.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#127

Post by throw9871928 »

galangalalgol wrote: post #108
jonathanstrange wrote: post #89

Earlier quoted context omitted.

It shouldn't be too hard to film a deepfake movie from a screen or projection that don't make it obvious it was filmed. That way, the cryptographic signature will even lend extra authenticity to the deepfake!

A screen with double the resolution and twice the framrate should be indistinguishable. Moreover if you pop the case on the camera and replace the sensor with something fed by display port (probably need an fpga to convert display port to lvds, spi,ic2 or whatever those sensors use, at speed) that should work too.

This is still a lot harder than just using the deepfake application in the OP. But I’ll admit the arms race might not be over yet.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#128

Post by jacquesm »

VMG wrote: post #116
jacquesm wrote: post #67

Earlier quoted context omitted.

Indeed it is, but I'm interested in this particular author's stance on this, prompted by their disclaimer which clearly indicates that they realize that there is the risk of abuse, and an AMA seems to be an excellent opportunity to gain some insight.

I agree. I'm not OP but I would answer like this: abuses of this technology are inevitable and can only be mitigated by counter-software (which leads to an arms race). The release of this source code could kickstart the development of deepfake detection software. Or maybe in general people need to put less weight on video evidence.

Interesting, I will wait for the OP to answer before responding to you.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#129

Post by mgdlbp »

galangalalgol wrote: post #108
jonathanstrange wrote: post #89

Earlier quoted context omitted.

It shouldn't be too hard to film a deepfake movie from a screen or projection that don't make it obvious it was filmed. That way, the cryptographic signature will even lend extra authenticity to the deepfake!

A screen with double the resolution and twice the framrate should be indistinguishable. Moreover if you pop the case on the camera and replace the sensor with something fed by display port (probably need an fpga to convert display port to lvds, spi,ic2 or whatever those sensors use, at speed) that should work too.

Not if the camera includes metadata like focus, shutter speed, accelerometer, GPS, etc. I don't really know, but I imagine the hardware security required wouldn't be too far from what's common now. Cameras are already unrepairable, so I suppose the arguments would have to be more from the privacy and who-controls-the-chipmakers perspectives.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#130

Post by tinus_hn »

throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

The surveillance states wet dream, where you can just look up who took that ‘unfair’ video of your agent breaking the rules.
Post reply on HN