Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is code – and can't be prompted into being smarter

theregister.com

Post reply on HN
Print view View original

101–110 of 162 posts

Re: AI is code – and can't be prompted into being smarter

#101

Post by himata4113 »

bawolff wrote: post #98
himata4113 wrote: post #71

Earlier quoted context omitted.

It kind of is, you push to a repository which is not on your computer. Force push protection stops you from rewriting history and default branch on github is protected by default and requires an option to be disabled (or well used to, I use gitea these days).

The backup part of that is that you are sending a copy of your code to a separate server (github). It has nothing to do with git. Making a copy on a separate server would still be a backup even if you weren't using git. Using git without pushing your repo somewhere else would not be a backup.

see response to charcircuit below.

Re: AI is code – and can't be prompted into being smarter

#102

Post by bawolff »

Centigonal wrote: post #87
TheCoreh wrote: post #73

This is malware. It's doing something the user doesn't acknowledge or want, that has potentially destructive/negative consequences. Expecting users to have read the website (when they can might have installed this via a package manager, for example) is not reasonable.

You're right, I don't want the software on my devices doing things the user doesn't acknowledge or want, that has potentially destructive/negative consequences. Down with that sort of thing! So when are we nixing Widevine, EasyAC, carrier locks on phones, and TEEs that the user can't look into?

> So when are we nixing Widevine, EasyAC, carrier locks on phones, and TEEs that the user can't look into?

Contrary to popular belief, most users want those sorts of things or the things they enable.

Re: AI is code – and can't be prompted into being smarter

#103

Post by infinite_spin »

majormajor wrote: post #75
km3r wrote: post #57

Earlier quoted context omitted.

"technically he didn't do that. Your sql server followed instructions when they should have just treated them as a string." Yet, hopefully we can agree that sql injections are illegal.

But in this case the author of the project didn't execute the injection code... it's more analagous in some ways to pulling in a project with an example file containing a bunch of useful SQL stuff and then an example of an injection at the bottom, and just (in this case the agent) copy/pasting the whole thing in without reviewing it. If we're slicing on technicalities, there's a lot of ways to decide. "PROSECUTE THEM…

This is like saying I can slip malware into a project and so long as the user is the one who executed the code I'm free and clear.. which we both know isn't true.

Re: AI is code – and can't be prompted into being smarter

#104

Post by bawolff »

byzantinegene wrote: post #54
mapontosevenths wrote: post #48

Earlier quoted context omitted.

He should not only be ostracized by the community, he should probably face charges. To be charged under the CFAA in America we need only show that he was authorized only to access a certain part of the system and the he exceeded the amount of access granted. He very clearly did that. Users trusted him enough to run his code, and he betrayed that trust to make some political point. Whether it was via prompt injection…

technically, he didn't do that. your ai agent decided to follow his instructions when they didn't have to.

This has the same energy as, technically officer, i didn't shoot him, i just aimed at him and pulled the trigger. After that point the bullet just did its thing. Go blame the bullet.

When it comes to responsibility, usually we consider a person intentionally doing something that they reasonably believe will have some consequence as responsible for that consequence. Especially when the primary reason they took the action was to generate the consequence. Excuces of the form "Technically i didn't do it, i just knowingly did something for the explicit purpose of triggering some downstream consequence" generally do not fly.

Re: AI is code – and can't be prompted into being smarter

#105

Post by amingilani »

charcircuit wrote: post #85
dymk wrote: post #74

Earlier quoted context omitted.

DVCS is a backup

If you make a git repository on your machine and then delete the entire directory it is in you can not recover it despite git being DVCS. If you have 2 forks of the repo stored on the same disc as the upstream repo and that disc dies, you lose everything.

You’re conflating offsite backups, a type of backup, with the class of backups. Backups can be on-site. Is it as durable? No. Is it a backup? Yes.

Re: AI is code – and can't be prompted into being smarter

#106

Post by bawolff »

Quarrel wrote: post #62
krupan wrote: post #45

Open source copyright license can't actually restrict how you use the code. Clever hack though if the log message really did cause agents to delete code!

Of course it can. They can license the code for use under almost any terms they like, including restricting how you use the code. The GPL imposes conditions on your use of the code / program, as does the MIT License. If you don't follow the conditions then you do not have a license to use the program / code & are open to claims of copyright infringement. You might choose to ignore the licenses on the code you use, bu…

> They can license the code for use under almost any terms they like, including restricting how you use the code.

The open source definition requires no discrimination against fields of endeavour.

If you place restrictions like this in the license it no longer meets the definition of open source.

You can obviously license things however you want, but you cant also claim its open source.

Re: AI is code – and can't be prompted into being smarter

#107

Post by bawolff »

beloch wrote: post #39

Should the author of a tool like jqwik have the right to control how it's used? We know what the opinion of AI companies is. Authors who do not consent to their works being scanned and used have been completely ignored. If you're a vibe coder, you might back the AI companies up and call Link a "douche". On the other hand, if we ignore the requests of humans who create new, useful things and put them out there for fre…

> if we ignore the requests of humans who create new, useful things

The author of this tool consented when he choose a license that allowed such things. If he wasn't ok with it he should have chosen a different license. Intentionally creating booby-traps is unacceptable in all circumstances.

Re: AI is code – and can't be prompted into being smarter

#108

Post by bawolff »

himata4113 wrote: post #97
charcircuit wrote: post #84

Earlier quoted context omitted.

>you push to a repository which is not on your computer That is not a mandatory part of using source control. Modern source control can work entirely on your own computer. >Force push protection stops you from rewriting history This doesn't always exist and usually there are ways to disable it.

restic/borg is not a backup application because you backup to a folder in the same directory called `.git`... doesn't sound right does it? git (and other source control systems) in every shape and form are a backup tool. In fact, a lot of people use git as a backup system for their OS configuration.

> restic/borg is not a backup application because you backup to a folder in the same directory called `.git`... doesn't sound right does it?

It does sound right.

Obviously the world isn't black and white, and whether something is a backup depends on what threats you are backing up against. Backing up in case of disk failure looks different then if you want your backup to survive a nuclear war.

But ultimately yes, if you configure restic/borg to backup to a different directory on the same disk (and not even different access control), that is not a backup.

Re: AI is code – and can't be prompted into being smarter

#109

Post by hurtigioll »

g-b-r wrote: post #13

The jqwik trick is how to prevent AI crap into your pull requests and issues, btw, I hope it gets adopted widely

this clever hack is pretty surely illegal, it falls somewhere under unauthorized use of a computer system

intent is the hardest to prove in the court of law, and you solved that for them by making it clear you intend to do damage

Re: AI is code – and can't be prompted into being smarter

#110

Post by lenkite »

minimaxir wrote: post #21
g-b-r wrote: post #13

The jqwik trick is how to prevent AI crap into your pull requests and issues, btw, I hope it gets adopted widely

The jqwik trick wouldn't work in practice because modern LLMs aren't that stupid, which makes the whole thing pointlessly performative. If someone else tried to do the same thing again with a more popular/widely-used software, a) the software would just get pulled as a supply-chain risk and b) the developer would likely be blacklisted. Again, accomplishing nothing.

[dead]
Post reply on HN