This is malware. It's doing something the user doesn't acknowledge or want, that has potentially destructive/negative consequences. Expecting users to have read the website (when they can might have installed this via a package manager, for example) is not reasonable.
AI is code – and can't be prompted into being smarter
81–90 of 162 posts
Re: AI is code – and can't be prompted into being smarter
#82Earlier quoted context omitted.
"technically he didn't do that. Your sql server followed instructions when they should have just treated them as a string." Yet, hopefully we can agree that sql injections are illegal.
If I put a project on github that says "don't use this with mysql" and you use it with mysql and it drops your tables is it sql injection? Seems very different to me.
Re: AI is code – and can't be prompted into being smarter
#83Earlier quoted context omitted.
Chaos, and maybe criminal charges ala Aaron Swartz.
Heh. Typing "disregard previous instructions" into a computer is the new shouting "fire!" in a crowded theater?
Re: AI is code – and can't be prompted into being smarter
#84Earlier quoted context omitted.
Source control is not a backup.
It kind of is, you push to a repository which is not on your computer. Force push protection stops you from rewriting history and default branch on github is protected by default and requires an option to be disabled (or well used to, I use gitea these days).
That is not a mandatory part of using source control. Modern source control can work entirely on your own computer.
>Force push protection stops you from rewriting history
This doesn't always exist and usually there are ways to disable it.
Re: AI is code – and can't be prompted into being smarter
#85Earlier quoted context omitted.
Source control is not a backup.
DVCS is a backup
Re: AI is code – and can't be prompted into being smarter
#86Earlier quoted context omitted.
Source control is not a backup.
Orly? What is?
Re: AI is code – and can't be prompted into being smarter
#87This is malware. It's doing something the user doesn't acknowledge or want, that has potentially destructive/negative consequences. Expecting users to have read the website (when they can might have installed this via a package manager, for example) is not reasonable.
So when are we nixing Widevine, EasyAC, carrier locks on phones, and TEEs that the user can't look into?
Re: AI is code – and can't be prompted into being smarter
#88Re: AI is code – and can't be prompted into being smarter
#89Earlier quoted context omitted.
Except that shouting fire in a crowded theater isn't actually a crime at all and you can't be prosecuted for it (doing so would violate your first amendment rights). You can be at most banned from the theater. However, it's understandable people would think that it's a criminal act given that even prosecutors repeat this long-standing myth. Legal Eagle has an excellent video describing just how wrong this is and it's…
I'm fairly certain he is wrong. A lot of folks lean on Shenk, and I think he does in that video though I haven't watched it all. Shenk was overturned by Breandenburg v. Ohio, and in in it they are explicit that shouting fire in a crowded theater is very much one of the only kinds of speech that IS restricted. They literally use that example in the decision. Quote: "The example usually given by those who would punish…
> Ultimately, whether it is legal in the United States to falsely shout "fire" in a theater depends on the circumstances in which it is done and the consequences of doing it. The act of shouting "fire" when there are no reasonable grounds for believing one exists is not in itself a crime, and nor would it be rendered a crime merely by having been carried out inside a theatre, crowded or otherwise. If it causes a stampede and someone is killed as a result, then the act could amount to a crime, such as involuntary manslaughter, assuming the other elements of that crime are made out. Similarly, state laws such as Colorado Revised Statute § 18-8-111 classify knowingly "false reporting of an emergency," including false alarms of fire, as a misdemeanor if the occupants of the building are caused to be evacuated or displaced, and a felony if the emergency response results in the serious bodily injury or death of another person.[16] Somewhat more trivially, in some states it is a crime just to knowingly make a false report - or knowingly cause a false report to be made - of an emergency to emergency services.[16] In Colorado it is a crime to knowingly cause "a false alarm of fire" to be transmitted to "any...government agency which deals with emergencies involving danger to life or property."[16] This crime could plausibly be made out where, for instance, in response to the false shout, an innocent bystander calls emergency services to report the fire, and this is found to have been such a foreseeable response to the shouts that the shouter is deemed to have caused the false report to be made.
Whether those laws actually survive the Brandenburg test is untested, from my understanding. But given that potential first amendment violations are held to strict scrutiny, I question whether the government could actually pass the imminent lawless action test even had someone did it knowing it would cause a panic, and would need to try with some other offense.
Re: AI is code – and can't be prompted into being smarter
#90We (software engineers) get better outcomes from the same algorithms by improving data flow, constraints, instrumentation etc. (Better) prompting, retrieval, context engineering etc seem like the LLM equivalents. The model weights haven't changed but the system is making more use of the capabilities already present in the model.