Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

181–190 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#181

Post by ethanbond »

FollowingTheDao wrote: post #138
bjt2n3904 wrote: post #113

Earlier quoted context omitted.

Then build a cultural revolution that instills distrust in what you see. We already teach the elderly not to trust a link that says, "you are the millionth visitor, click to win your prize". Teach them also to look past the video call and what looks (and sounds) like their grandson to what is actually being said. He's asking for your bank password, that behavior doesn't match! The cat is already out of the bag. Going…

But you are making the argument that releasing it is protective, and there is no basis for that argument. > Society must unlearn that seeing is believing. A total lack of trust of our shared perception? Let me know how that works out. Might as well have everyone on acid 24/7.

Yeah I can’t believe that people think it’s possibly a desirable future for there to be no such thing as trust, and furthermore that we should accelerate our march in that direction. What an insane way to live.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#182

Post by Workaccount2 »

Tepix wrote: post #80
throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

> it will pretty much solve the trust issues I'm not so sure. How will you verify a signature if you see a video on TV or on social media? Do you believe these devices are 100% secure and the keys will never be extracted?

It will be 6 months before hardware bypass devices show up on alibaba.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#183

Post by Workaccount2 »

pbhjpbhj wrote: post #131
jacquesm wrote: post #93

Earlier quoted context omitted.

That will just move the hack one level further down and will create even more confusion because then you'll have a 'properly signed video stream' as a kind of certificate that the video wasn't manipulated. But you don't really know that, because the sensor input itself could be computer generated and I can think off the bat of at least two ways in which I could do just that.

Isn't the point to prove physical access to the camera. As in, this stream originated at this TPM which was sold as part of this camera. So, the best you get is that the stream shows it wasn't produced with access to a particular camera. Then impersonating a YouTuber, say, requires access to their physical camera.

Just like this post about making a software bypass to a security check, there are hardware guys making bypasses to security checks.

Now you create a situation where hardware bypassed cameras have immense value to bad actors.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#184

Post by RobertRoberts »

blagie wrote: post #150
sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

As someone with a stalker, I can't emphasize this enough. A stalker will go to all sorts of lengths to do bizarre shit. People don't believe it. I would guess governments will do some equivalent there-of. Democratizing access to things -- including bad things -- has a preventative effect: 1) I can guard against things I know about 2) People take me seriously if something has been democratized The worst-case scenario…

I want to second this, as there are so many people that just simply can't believe how much time and energy some people will put into destroying someone else's life.

And when you ask for help, people think you are the insane one because they simply can't believe your story about the insanity of someone else.

I hope you find relief sometime from your stalker. I found it (not a stalker exactly) from letting the person burn themselves with their behavior so many times without me doing or saying anything in return to them (my strategy of non-direct conflict, and it worked for me), that eventually they ran out of people to manipulate and fool.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#185

Post by Scea91 »

barnabee wrote: post #169
bko wrote: post #148

Earlier quoted context omitted.

Honest question, do you apply the same reasoning to gun control?

Good question. I would apply that reasoning to the information about how a gun works for sure, but IMO there is some (admittedly hard to define) amount of danger or perhaps destructive power beyond which we might or perhaps should choose as a society to restrict the ability to wield a tool and certainly at least some guns exceed that limit for me. I’m not sure that any software, on its own, does though, at least I ca…

In this analogy information about how gun works would be an equivalent of Deepfake research paper. This tool is analogous to an actual gun.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#186

Post by galangalalgol »

throw9871928 wrote: post #127
galangalalgol wrote: post #108

Earlier quoted context omitted.

A screen with double the resolution and twice the framrate should be indistinguishable. Moreover if you pop the case on the camera and replace the sensor with something fed by display port (probably need an fpga to convert display port to lvds, spi,ic2 or whatever those sensors use, at speed) that should work too.

This is still a lot harder than just using the deepfake application in the OP. But I’ll admit the arms race might not be over yet.

You can raise the bar a bit. No one checks to see if a $5 bill is fake. There is no real upper limit on what the payout for a deepfake could be. TPM isnt going to save facial recognition ID like the IRS from being obsoleted by deepfakes. But, for things that go to trial a TPM dashcam with tamper evident packaging (that can be inspected during trial) is probably good enough for small claims court. You could add GPS spoof detection, put as much as you can on the tpm chip (like the accelerometer), and sign all sorts of available data along with the video, but that will up the unit price a lot, and for the kind of fraudulent payouts you'd be trying to stop, you wouldn't make it enough harder to keep it from being cost effective for the fakers.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#187

Post by Karliss »

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

Even without deepfakes any kind of system relying on a person (or computer) not being tricked by webcam video seems quite questionable. People could still be tricked with a spliced video fragments of the real person or makeup especially if the set of face expressions used during "liveness check" is known ahead of time.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#188

Post by ridgered4 »

ethanbond wrote: post #181
FollowingTheDao wrote: post #138

Earlier quoted context omitted.

But you are making the argument that releasing it is protective, and there is no basis for that argument. > Society must unlearn that seeing is believing. A total lack of trust of our shared perception? Let me know how that works out. Might as well have everyone on acid 24/7.

Yeah I can’t believe that people think it’s possibly a desirable future for there to be no such thing as trust, and furthermore that we should accelerate our march in that direction. What an insane way to live.

It's not desirable at all, but it is inevitable and in many ways already here. While I'm not sure I agree with the argument, I think the idea is holding this stuff back just means it's more powerful and nefarious for those that do have it because the population at large will remain unaware for longer that trust is already gone. And that means innocent people are going to get hurt while that knowledge asymmetry in the population is delayed from catching up.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#190

Post by leereeves »

ethanbond wrote: post #178
barnabee wrote: post #174

Earlier quoted context omitted.

Over time the number of single humans that have the power to wipe out a sizeable portion of life on Earth (which is almost certainly non-zero already) will almost certainly increase due to advances in technology and dissemination of information. Do we aim for security through obscurity or work out how to deal with that increasingly terrifying reality? I think at some point we will be forced to do latter.

This trend is clearly real, but what exactly do you mean “work out how to deal with that?” One of the obvious ways to deal with it, albeit imperfectly, is in fact to keep these technologies as obscure as possible for as long as possible. Maybe an interesting norm to try to generate in this community would be, “if you plan on publishing attack X, you should publish immediately alongside it your best guess as to the ef…

Keeping these technologies in the hands of a few has its own set of severe negative consequences.

In the case of nuclear weapons, one of those consequences is that poor countries are denied the ability to defend themselves.

This led to the bullying and invasion of countries without nuclear weapons by countries with nuclear weapons (most recently Russia invading Ukraine, but China and the US are equally guilty).

I'm reminded of a line from the TV show The West Wing, something like: "India must and will be a nuclear power. That way, we'll never be dictated to again."

Post reply on HN