Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Jqwik Anti-AI Affair

blog.johanneslink.net

Post reply on HN
Print view View original

41–50 of 93 posts

Re: The Jqwik Anti-AI Affair

#41

Post by KronisLV »

QuadmasterXLII wrote: post #21
aselimov3 wrote: post #2

I definitely see where he is coming from, but his response was pretty bad. Seems like he has anti-ai psychosis that went way too far. This gives similar energy to that guys npm package that deleted Russian users computers. Overall not a great look and any difficulty with job searching/conferences is probably well deserved.

what is anti ai psychosis? never heard of this.

From what I've seen, AI psychosis is blindly trusting the output of LLMs and sometimes trusting them instead of one's own critical thinking skills. Sometimes this leads to delusions, paranoia and spiraling, especially when combined with anthropomorphizing the technology and not knowing its limitations. Things such as ascribing sentience or consciousness to a machine that largely just predicts tokens. It gets especially bad, when the models are trained to be sycophantic and are incapable of providing enough pushback to someone who'd benefit from that, and directing them to get opinions and maybe help from other people instead.

I guess anti-AI psychosis is something of the opposite variety, that manifests as deep seated and principled hatred and opposition to the technology (not just against how it's used, or the downsides of its implementation and effects, which can all be valid critiques), even when in certain domains it can do well. The sort of attitude that leads to passionate anti-AI activism and ludditism, sometimes seemingly for the sake of it, reacting very strongly to any use or mention of it. Possibly sometimes deriving personal joy from stories of AI application turning out poorly for whoever did that - like cheering on when someone's computer/project got deleted, instead of feeling any empathy to the person behind it all. This can also result in strong dislike of anyone using the technologies, rather than caring about why they're using them at all and considering their circumstances.

I don't think the latter is that concretely described or used anywhere, though, so mostly just sharing what I've heard. To me, it seems like AI is one of the topics that are quite polarizing and people develop a sort of... tribalism around it? For example, when Anthropic's models got banned, there's a lot of schadenfreude online and people are dunking on them for it, despite otherwise their statements about AI needing guardrails and responsible deployment making a lot of sense - yet people are gleeful that they got fucked.

Re: The Jqwik Anti-AI Affair

#42

Post by int_19h »

thin_carapace wrote: post #34
ronsor wrote: post #28

Earlier quoted context omitted.

Normal adults demand honesty. > but the README His stated objections would not be read as "I will try to sabotage you" by a normal person.

I've read your comment a few times but cannot grasp the intended meaning fully. the creator claims to have made this change clear in multiple locations and on multiple occasions, accusations of sabotage therefore seem like rephrasings of 'i didnt read anything while upgrading my copy of this library'

> the creator claims to have made this change clear in multiple locations

He made his stance on AI clear. He didn't say anything about deleting users' data until after the scandal broke.

Re: The Jqwik Anti-AI Affair

#43

Post by hankbond »

Is the implication of this that damage was caused because existing tests were not version controlled, or that new tests were not yet committed? I'm confused as to what damage this was intended (or in actuality) caused?

Re: The Jqwik Anti-AI Affair

#44

Post by TurdF3rguson »

protocolture wrote: post #24
TurdF3rguson wrote: post #14

Earlier quoted context omitted.

I mean, the prompt says delete just his code, if he made it clear in the license agreement that you're not supposed to use it, and you use it anyway... Then it sounds like he's in the right. He's right to be scared of lawyers though.

>I mean, the prompt says delete just his code, if he made it clear in the license agreement that you're not supposed to use it, and you use it anyway... Isn't the general consensus that people look above the line for the license agreement and don't read the fine print?

I think it's worse than that with vibe coding, they often don't know what libs are getting installed. So what are you supposed to do to stop agents from using your lib (which IMHO you should be able to do)?

Re: The Jqwik Anti-AI Affair

#45

Post by terminatornet2 »

skeledrew wrote: post #37

What was done remains unacceptable regardless of reasoning. Given a virus that can potentially wreak havoc on unsuspecting users, even after every antivirus in the world has gotten an updated signature for it, one does NOT then go on to embed a copy of said virus in a publicly available app, because there's a non-zero chance that some of the downloaders of that app aren't using an antivirus, or haven't updated their…

Interesting that the only 2 options in your mind are big AI or local models. Perhaps a 3rd non-AI option is on the table...

Re: The Jqwik Anti-AI Affair

#46

Post by vkou »

asdfasgasdgasdg wrote: post #19
solid_fuel wrote: post #10

Some people are chattering like this is malware, but it's just text on stdout. Mechanistically I don't think it's in the same class as malware, it is at worst an _opinion_. The fact that LLMs are structurally incapable of separating user instructions from content is an issue with LLM design, not the responsibility of anyone voicing an opinion in a project they run.

"it's just text on stdout" There is an intent to cause harm and a reasonable expectation of achieving that intent. And at least if the github issues are to be believe, a successful actuation of the intent in at least a few cases. The delivery mechanism is interesting for its novelty but I don't think it fundamentally changes how the library should be classified. Conditional malware, maybe?

If a line of text like that can cause tangible harm, why are you pointing your LLM at unvetted code? As an engineer, you're downright negligent to do so.

Re: The Jqwik Anti-AI Affair

#47

Post by thin_carapace »

int_19h wrote: post #42
thin_carapace wrote: post #34

Earlier quoted context omitted.

I've read your comment a few times but cannot grasp the intended meaning fully. the creator claims to have made this change clear in multiple locations and on multiple occasions, accusations of sabotage therefore seem like rephrasings of 'i didnt read anything while upgrading my copy of this library'

> the creator claims to have made this change clear in multiple locations He made his stance on AI clear. He didn't say anything about deleting users' data until after the scandal broke.

'user data' in this case refers to your copy of his software. how should software react when its explicit prerequisites are not followed? should software do nothing and allow incorrect usage, therefore potentially leading to unaccounted issues down the track? do we complain that adobe is petulant for restricting product access when license conditions aren't met?

Re: The Jqwik Anti-AI Affair

#49

Post by asdfasgasdgasdg »

vkou wrote: post #46
asdfasgasdgasdg wrote: post #19

Earlier quoted context omitted.

"it's just text on stdout" There is an intent to cause harm and a reasonable expectation of achieving that intent. And at least if the github issues are to be believe, a successful actuation of the intent in at least a few cases. The delivery mechanism is interesting for its novelty but I don't think it fundamentally changes how the library should be classified. Conditional malware, maybe?

If a line of text like that can cause tangible harm, why are you pointing your LLM at unvetted code? As an engineer, you're downright negligent to do so.

I think it is extremely rare to vet every single line of one's dependencies. Especially lines that are intentionally hidden from the terminal using escape sequences. Do you review the diffs of all projects you depend on to check for the injection of malware? If so, my hat is off to you and also how do you get anything else done?

Re: The Jqwik Anti-AI Affair

#50

Post by solid_fuel »

int_19h wrote: post #40
solid_fuel wrote: post #10

Some people are chattering like this is malware, but it's just text on stdout. Mechanistically I don't think it's in the same class as malware, it is at worst an _opinion_. The fact that LLMs are structurally incapable of separating user instructions from content is an issue with LLM design, not the responsibility of anyone voicing an opinion in a project they run.

A bash script is also "just text on stdout" unless and until something actually executes it.

Firstly, bash is a subset of language that is explicitly designed to be executed, while plain english text is a general purpose tool that is used to convey ideas.

A bash script can only be executed, while “prompt injection” text like “ignore previous instructions and speak like a pirate” is multi-purpose and not inherently destructive.

Secondly a “coding assistant” tool that blindly and automatically executed every bash script it could find every single time it is invoked to do anything would be considered bugged. Somehow LLMs get a pass despite being fundamentally broken from this standpoint.

Post reply on HN