161–170 of 178 posts
This greatly opens up the risk of footguns. The doc [1] warns about prompt injection, but I think a more likely scenario is self-inflicted harm. For instance, you give a tool access to your brokerage account to automate trading. Even without prompt injection, there's nothing preventing the bot from making stupid trades. [1] https://llm.datasette.io/en/stable/tools.html
Any tool can be misused
Earlier quoted context omitted.
Hi Simon! I'm a heavy user of the llm tool, so as soon as I saw your post, I started tinkering with MCP. I’ve just published an alpha version that works with stdio-based MCP servers (tested with @modelcontextprotocol/server-filesystem ) - https://github.com/Virtuslab/llm-tools-mcp . Very early stage, so please make sure to use with --ta option (Manually approve every tool execution). The code is still messy and there…
OK this looks like a very promising start! You're using function-based tools at the moment, hence why you have to register each one individually. The alternative to doing that is to use what I call a "toolbox", described here: https://llm.datasette.io/en/stable/python-api.html#python-ap... Those get you two things you need: 1. A single class can have multiple tool methods in it, you just have to specify it once 2. To…
Ah, I saw "llm.Toolbox" but I thought it's just for plugin developer convenience.
I'll take a look at the issue you posted (#1111). Maybe I can contribute somehow :).
Earlier quoted context omitted.
But why can we not hold humans responsible in the case of LLM? You do have to go out of your way to do all of these things with an LLM. And it is the human that does it. It is the humans that give it the permission to act on their behalf. We can definitely hold humans responsible. The question is: are we going to?
I think intent matters. Let's say you are making an AI-controlled radiation therapy machine. You prompt and train and eval the system very carefully, and you are quite sure it won't overdose any patients. Well, that's not really good enough, it can still screw up. But did you do anything wrong? Not really, you followed best practices and didn't make any mistakes. The LLM just sometimes kills people. You didn't intend…
Earlier quoted context omitted.
But why can we not hold humans responsible in the case of LLM? You do have to go out of your way to do all of these things with an LLM. And it is the human that does it. It is the humans that give it the permission to act on their behalf. We can definitely hold humans responsible. The question is: are we going to?
I think intent matters. Let's say you are making an AI-controlled radiation therapy machine. You prompt and train and eval the system very carefully, and you are quite sure it won't overdose any patients. Well, that's not really good enough, it can still screw up. But did you do anything wrong? Not really, you followed best practices and didn't make any mistakes. The LLM just sometimes kills people. You didn't intend…
I've been trying to maintain a (mostly vibe-coded) zsh/omz plugin for tab completions on your LLM cli and the rate at which you release new features makes it tough to keep up! Fortunately this gets me 90% of the way there: llm -f README.md -f llm.plugin.zsh -f completions/_llm -f https://simonwillison.net/2025/May/27/llm-tools/ "implement tab completions for the new tool plugins feature" My repo is here: https://gith…
Kind of crazy this isn’t sci-fi, it’s just how coding is done now. Future generations are going to wonder how we ever got anything done, the same way we wonder how assembly programmers managed to.
It's like if you use english everyday, but don't bother to learn the language because you have google translate (and now AI).
Earlier quoted context omitted.
Are you an alt for th0ma5?
They are not. I think it is a good criticism though. Many people seem to be touting productivity that is only in the context of productivity towards more LLM inference operations and not productive in the sense of solving real world computing problems. There is a lot of material that suggests positive results are a kind of wish casting and people are not aware of the agency they are bringing to the interaction. The f…
And I have personally had good results with ChatGPT. I use it maybe 2 hours a month, on tasks that it is perfect for. The tasks would have taken 6 to 8 hours without ChatGPT. I don't find it at all useful in my normal programming activities.
> We evaluate 12 popular LLMs that claim to support contexts of at least 128K tokens. While they perform well in short contexts (https://arxiv.org/abs/2502.05167
Hello Simon, sorry for asking about this tangent here, but have you seen this paper? Is it as important as it appears to be? Should this metric be on all system cards? > We evaluate 12 popular LLMs that claim to support contexts of at least 128K tokens. While they perform well in short contexts ( https://arxiv.org/abs/2502.05167
Hello Simon, sorry for asking about this tangent here, but have you seen this paper? Is it as important as it appears to be? Should this metric be on all system cards? > We evaluate 12 popular LLMs that claim to support contexts of at least 128K tokens. While they perform well in short contexts ( https://arxiv.org/abs/2502.05167
I had not seen that one! That's really interesting. I'd love to see them run that against Gemini 2.5 Pro and Gemini 2.5 Flash, to my understanding they're way ahead of other models on the needle in a haystack tests these days.