81–90 of 124 posts
Here's a proposal: app.md. A structured text file with everything you want your app to do. That way we can have entire projects with nothing but Markdown files. And we can run apps with just `claude run app.md`. Who needs silly code anyway?
I'm seeing a lot of negativity in the comments. Here's why I think this is actually a Good Idea. Many command line tools rely on something like this for installation: $ curl -fsSL https://bun.com/install | bash This install script is hundreds of lines long and difficult for a human to audit. You can ask a coding agent to do that for you, but you still need to trust that the authors haven't hidden some nefarious instr…
This seems less auditable though, because now there is more variability in the way something is installed. Now there are two layers to audit: - What the agent is told to do in prose - How the agent interprets those instructions with the particular weights/contexts/temperature at the moment. I’m all for the prose idea, but wouldn’t want to trade determinism for it. Shell scripts can be statically analyzed. And also re…
Earlier quoted context omitted.
>Installing software is a task which should be left to AI. This is such an insane statement. Is this satire?
Ok, I've toned that bit down for you!
I think the subtext here is actually revealing a deeper issue. Installing software sucks. It’s error prone and every project does it a slightly different way. What we need is standardization, and I can see why prose could be an attractive middle ground. Easier to understand but less precise may result in marginally better outcomes.
I’m concerned that this approach serves to fix the obvious problems while simultaneously introducing subtler problems.
Tangentially, I’ve been thinking about this a lot lately. There are projects like nix that are excellent at fixing a lot of problems in the software packaging and installation space that are great from a security perspective but are famously difficult to use. I’d personally like to see more work leveraging AI to increase the accessibility of these paradigms and not throw the bathwater out with the baby, so to speak.
Earlier quoted context omitted.
This seems less auditable though, because now there is more variability in the way something is installed. Now there are two layers to audit: - What the agent is told to do in prose - How the agent interprets those instructions with the particular weights/contexts/temperature at the moment. I’m all for the prose idea, but wouldn’t want to trade determinism for it. Shell scripts can be statically analyzed. And also re…
Yes, this approach (substituting a markdown prompt for a shell script) introduces an interesting trade-off between "do I trust the programmer?" and "do I trust the LLM?" I wouldn't be surprised to see prompt-sharing become the norm as LLMs get better at following instructions and people get more comfortable using them.
Earlier quoted context omitted.
This seems less auditable though, because now there is more variability in the way something is installed. Now there are two layers to audit: - What the agent is told to do in prose - How the agent interprets those instructions with the particular weights/contexts/temperature at the moment. I’m all for the prose idea, but wouldn’t want to trade determinism for it. Shell scripts can be statically analyzed. And also re…
Yes, this approach (substituting a markdown prompt for a shell script) introduces an interesting trade-off between "do I trust the programmer?" and "do I trust the LLM?" I wouldn't be surprised to see prompt-sharing become the norm as LLMs get better at following instructions and people get more comfortable using them.
I hear you, and I can see the pragmatism of your approach. I’m just not convinced that it’s better.
Earlier quoted context omitted.
IMO it's completely the other way around. Shell scripts can be audited. The average user may not do it due to laziness and/or ignorance, but it is perfectly doable. On the other hand, how do you make sure your LLM, a non-deterministic black box, will not misinterpret the instructions in some freak accident?
How about both worlds? Instead of asking the agent to execute it for you, you ask the agent to write an install.sh based on the install.md? Then you can both audit whatever you want before running or not.
Sorry but what the heck?
We should NOT standardize irresponsible behavior, in particular for repeatable tasks. This is particularly maddening when solutions like dependency resolution, containers, distribution of self-contained and binaries DO exist.
I understand that the hype machine must feed on yet another idea to keep its momentum but this is just ridiculous.
Earlier quoted context omitted.
My point is not that LLMs are inherently trustworthy. It is that a prompt can make the intentions of the programmer clear in a way that is difficult to do with code because code is hard to read, especially in large volumes.
I’m not sure I agree with you that code is hard to read. I usually tend to go straight to the source code as it communicates precisely how something will behave. Well written code, like well written prose can also communicate intent effectively.
That is why we have programming languages, they, coupled with a specific interpreter/compiler, are pretty clear on what they do. If someone misunderstands some specific code segment, they can just test their assumptions easily.
You cannot do that with just written prose, you would need to ask the writer of that prose to clarify.
And with programming languages, the context is contained, and clearly stated, otherwise it couldn't be executed. Even undefined behavior is part of that, if you use the same interpreter/compiler.
Also humans often just read something wrong, or skip important parts. That is why we have computers.
Now, I wouldn't trust a LLM to execute prose any better then I trust a random human of reading some how-to guide and doing that.
The whole idea that we now add more documentation to our source code projects, so that dumb AI can make sense of it, is interesting... Maybe generally useful for humans as well... But I would instead target humans, not LLMs. If the LLMs finds it useful as well, great. But I wouldn't try to 'optimize' my instructions so that every LLM doesn't just fall flat on its face. That seems like a futile effort.
Earlier quoted context omitted.
IMO it's completely the other way around. Shell scripts can be audited. The average user may not do it due to laziness and/or ignorance, but it is perfectly doable. On the other hand, how do you make sure your LLM, a non-deterministic black box, will not misinterpret the instructions in some freak accident?
How about both worlds? Instead of asking the agent to execute it for you, you ask the agent to write an install.sh based on the install.md? Then you can both audit whatever you want before running or not.
Good idea. That seems sensible.
Bonus: LLM is only used once, not every time anyone wants to install some software. With some risks of having to regenerate, because the output was nonsensical.