121–130 of 162 posts
Earlier quoted context omitted.
The backup part of that is that you are sending a copy of your code to a separate server (github). It has nothing to do with git. Making a copy on a separate server would still be a backup even if you weren't using git. Using git without pushing your repo somewhere else would not be a backup.
In this case, using Git would have helped even if it was local-only.
I feel like there is a line somewhere here. Just because you dont like someone or what they are doing doesn't mean its ok to intentionally screw with them.
It only screwed with people who violated the licence. Just like those taint chips in clothing stores only screw with people who steal clothes.
> Just like those taint chips in clothing stores only screw with people who steal clothes.
If we are going to extend the metaphor to the physical, i'd point out that probably the most equivalent is putting a bomb in a package on your porch in order to target people who steal packages. Which is illegal pretty much everywhere.
Regardless, even if you are of the opinion that the maintainer of jqwik was wronged, just because someone wrongs you does not give you the right to wrong them in turn. There is a reason why we as a society developed a court system instead of just settling disputes by vengence.
Earlier quoted context omitted.
But in this case the author of the project didn't execute the injection code... it's more analagous in some ways to pulling in a project with an example file containing a bunch of useful SQL stuff and then an example of an injection at the bottom, and just (in this case the agent) copy/pasting the whole thing in without reviewing it. If we're slicing on technicalities, there's a lot of ways to decide. "PROSECUTE THEM…
This is like saying I can slip malware into a project and so long as the user is the one who executed the code I'm free and clear.. which we both know isn't true.
Say I lay a log on a road which you can clearly see and avoid but choose to drive over and crash your car, that’s prompt injection.
One is way worse than the other.
I feel like such prompt injections are really just another variant of the supply chain attack. Instead of selecting for bitcoin afficionados, this one hits AI fans. This will be fashionable for a little while but if AI continues to gain mindshare it will eventually be project suicide (at least to the extent the project exists in any part to serve third parties) to pull tricks like this. I'm not sure it's anything to…
It was already YEARS ago that they found that certain things such as the time of year (December vs. start of January) had an impact on reasoning effort.
Until we're training models such that the undesirable human patterns aren't picked up from training data there will always be a way to prompt it to be smarter. Also look at anthropic's "assistant axis" research from a short while ago - because intelligence in a domain is relative, if I prompt it with language connected to a particular domain, use the appropriate jargon that achieves far better results.
I feel like such prompt injections are really just another variant of the supply chain attack. Instead of selecting for bitcoin afficionados, this one hits AI fans. This will be fashionable for a little while but if AI continues to gain mindshare it will eventually be project suicide (at least to the extent the project exists in any part to serve third parties) to pull tricks like this. I'm not sure it's anything to…
The most controversial part is probably that something like jqwik could be a dependency you're not even aware of. You could be asking your agent to fully analyze a project to isolate a bug and in the right situation that would probably trigger the exploit.
Earlier quoted context omitted.
This is like saying I can slip malware into a project and so long as the user is the one who executed the code I'm free and clear.. which we both know isn't true.
Say I loosen the bolts of your car tires which causes a crash, that’s malware. Say I lay a log on a road which you can clearly see and avoid but choose to drive over and crash your car, that’s prompt injection. One is way worse than the other.
The jqwik trick is how to prevent AI crap into your pull requests and issues, btw, I hope it gets adopted widely
The jqwik trick wouldn't work in practice because modern LLMs aren't that stupid, which makes the whole thing pointlessly performative. If someone else tried to do the same thing again with a more popular/widely-used software, a) the software would just get pulled as a supply-chain risk and b) the developer would likely be blacklisted. Again, accomplishing nothing.
Performative for which side you mean? The author described it in the context of them expressing their opinion, thus imo the performative part describes all these extreme, unwarranted reactions and canceling against them.
Earlier quoted context omitted.
In this case, using Git would have helped even if it was local-only.
What did the AI actually do? Because `rm -rf .` is not something git can help you with.