Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Resistance: some recent anti-AI stuff that’s worth discussing

stephvee.ca

Post reply on HN
Print view View original

361–370 of 439 posts

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#361

Post by vidarh »

Ar-Curunir wrote: post #296
vidarh wrote: post #156

Earlier quoted context omitted.

> the ability to poison models, if it can be made to work reliably Ultimately, it comes down to the halting problem: If there's a mechanism that can be used to alter the measured behaviour, then the system can change behaviour to take into account the mechanism. In other words, unless you keep the poisoning attack strictly inaccessible to the public, the mechanism used to poison will also be possible to use to train…

> Ultimately, it comes down to the halting problem: If there's a mechanism that can be used to alter the measured behaviour, then the system can change behaviour to take into account the mechanism. No, that’s the opposite of the halting problem…

It's the very core of the halting problem.

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#362

Post by vidarh »

GTP wrote: post #193
vidarh wrote: post #156

Earlier quoted context omitted.

> the ability to poison models, if it can be made to work reliably Ultimately, it comes down to the halting problem: If there's a mechanism that can be used to alter the measured behaviour, then the system can change behaviour to take into account the mechanism. In other words, unless you keep the poisoning attack strictly inaccessible to the public, the mechanism used to poison will also be possible to use to train…

This reduction to the halting problem looks too handwawy to me. I don't see as a given that the possibility of the system taking into account the attack follows from the existence of the attack.

This is why I pointed out that the only way poisoning has a chance of working other than over very short timelines is if the tools to do so remains private and inaccessible to the public.

It's a bit of a leap, but the halting problem can be generalized to:

It is impossible in the general case to produce a detector function f(x), that will decide if program x behaves according to rule y if x can include f(x) as part of the itself.

The reason is that if a program x can make use of the detector, it can effectively do if f(x) { do the opposite of what f(x) predicts}

The leap from that to poisoning might be a bit unintuitive, but it boils down to the poisoner having a mechanism that would alter model behaviour.

If you have access to that mechanism, you can produce a detector by using the mechanism to induce the unwanted behaviour, and train a model on that.

Once you have a detector, you can behave differently based on the signal from the detector, and by extension avoid the effects of the original mechanism.

And that is the core of the halting problem.

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#363

Post by intended »

ofjcihen wrote: post #337
tptacek wrote: post #9

I'm glad this person found community, but I think they've been a bit starstruck by concentrated interest. At no point in the next 30 years will there not be an active community of people who "loathe" AI and work to obstruct it. There are those people about smart phones, the Internet itself, even television. Meanwhile: the ability to poison models, if it can be made to work reliably, is a genuinely interesting CS ques…

> At no point in the next 30 years will there not be an active community of people who "loathe" AI and work to obstruct it. On the one hand I agree with you but on the other sometimes I wonder just how insulated we are in the tech community and especially in sites like this. At some point in the last few months I realized that my friend group is basically a bubble of people making mid 6 figures that all work in tech…

Very insulated.

Stanford released a report[1] in April, that shows a wide gap between AI Insiders and everyone else.

> 5. AI experts and the U.S. public have very different perspectives on AI's future, except on elections and personal relationships.

>On how people do their jobs, 73% of experts expect a positive impact compared to just 23% of the public, a 50-point gap. Similar divides appear for the economy (69% vs. 21%) and medical care (84% vs. 44%)

> 6. Nearly two-thirds of Americans (64%) expect AI to lead to fewer jobs over the next 20 years, while only 5% expect more.

>Experts were less pessimistic (39% fewer, 19% more) but forecast far faster adoption, expecting generative AI to assist 18% of U.S. work hours by 2030 versus the public's estimate of 10%.

[1] https://hai.stanford.edu/ai-index/2026-ai-index-report

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#364

Post by vidarh »

mswphd wrote: post #221
GTP wrote: post #193

Earlier quoted context omitted.

This reduction to the halting problem looks too handwawy to me. I don't see as a given that the possibility of the system taking into account the attack follows from the existence of the attack.

They might be trying to talk about Rice's theorem? https://en.wikipedia.org/wiki/Rice%27s_theorem Formally, any non-trivial semantic property of a Turing machine is undecidable. Semantic here (roughly) means "behavioral" questions of the turing machine. E.g. if you only look at the "language" it defines (viewing it as a black box), then it is undecidable to answer any question about that language (including things li…

Rice's theorem is a close corollary, but I did mean the halting problem. Pointing to the halting problem was a bit of a throwaway quip because the "general shape" of it is an easy smell test for whether something is likely to be possible:

If you have access to run a transform on data, you can use it to train a model that acts as a detector of whether that transform has been applied to the data.

When you have a detector for a given property, you can use that detector to alter behaviour to exclude that property.

And that is the abstract core of why the halting problem is unsolvable.

In this case, if you have access to a mechanism for poisoning data, you can use that to train a detector. Once you have a detector, you can either exclude poisoned data, or use it for adversarial training.

Either way: The existence of the poisoning mechanism can be directly used to derive the tools to create its own antidote.

And that's back to the core of the halting problem.

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#366

Post by nkrisc »

nemo44x wrote: post #180
nkrisc wrote: post #99

Earlier quoted context omitted.

All I’ve been hearing is how AI will replace human workers with no mention of what those humans are supposed to do when they get replaced. I think people are rightfully concerned about that. We’re automating the interesting work with AI and leaving the drudge work for humans.

> We’re automating the interesting work with AI and leaving the drudge work for humans. I think you have that backwards.

Ask artists and designers what they think about that.

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#367

Post by emil-lp »

neop1x wrote: post #365

Funny, I can't even access his site: " Sorry, you have been blocked You are unable to access stephvee.ca " - CloudFlare Anti-AI but pro-MITM and pro-centralisation, limiting human visitors to his site...

Same here.

Wayback Machine

https://web.archive.org/web/20260420203809/https://stephvee....

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#368

Post by vidarh »

thfuran wrote: post #317
GTP wrote: post #193

Earlier quoted context omitted.

This reduction to the halting problem looks too handwawy to me. I don't see as a given that the possibility of the system taking into account the attack follows from the existence of the attack.

And even if it is provably possible to do, that doesn't mean it's easy. That's kind of the basis of encryption.

It doesn't need to be easy, but unlike with encryption it also doesn't need to be particularly precise. E.g. it's okay to exclude non-poisoned training data because you didn't manage to create a precise enough detector, as long as you don't exclude too much.

Basically any poisoning attack is also fundamentally limited because it needs to be non-invasive enough for humans not to be adversely affected, and that limits the problem space severely - the poisoning mechanism basically becomes reduced to a training mechanism to train out places where the models act different to humans.

Re: AI Resistance: some recent anti-AI stuff that’s worth discussing

#370

Post by vidarh »

lepus wrote: post #209
vidarh wrote: post #156

Earlier quoted context omitted.

> the ability to poison models, if it can be made to work reliably Ultimately, it comes down to the halting problem: If there's a mechanism that can be used to alter the measured behaviour, then the system can change behaviour to take into account the mechanism. In other words, unless you keep the poisoning attack strictly inaccessible to the public, the mechanism used to poison will also be possible to use to train…

It's a very comparable game of cat and mouse to spam email filtering. People also tried to claim that spam was over because for a time companies like Google cared enough to invest a lot in preventing as much as possible from getting through. If you've noticed in recent years the motivation to keep up that level of filtering has greatly diminished. Whether model poisoning becomes a bigger issue depends on the incentiv…

Indeed, the existence of public access to the spam detection mechanism (up to and including spammers having tons of Gmail accounts etc. to test deliverability with) provides the same mechanism: The spammers can use the detector meant to stop them, to alter behaviour to evade the detector.
Post reply on HN