Do local multimodal llms have low latency? If it can plug into monitor video stream and the accessibility representation of UI (like Shortcat), it could answer if a printer is connected to the computer and preview a full page print and wait for us to hit print.
Show HN: My LLM CLI tool can run tools now, from Python code or plugins
141–150 of 178 posts
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#142[deleted]
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#143[dead]
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#144This greatly opens up the risk of footguns. The doc [1] warns about prompt injection, but I think a more likely scenario is self-inflicted harm. For instance, you give a tool access to your brokerage account to automate trading. Even without prompt injection, there's nothing preventing the bot from making stupid trades. [1] https://llm.datasette.io/en/stable/tools.html
I think the whole footgun discussion misses the point. Yes, you can shoot yourself in the foot (and probably will), but not evaluating the possibilities is also a risk. Regular people tend to underestimate the footgun potential (probably driven by fear of missing out) and technical people tend to underestimate the risk of not learning the new possibilities.
Even a year ago I let LLMs execute local commands on my laptop. I think it is somewhat risky, but nothing harmful happened. You also have to consider what you are prompting. So when I prompt 'find out where I am and what weather it is going to be', it is possible that it will execute rm -rf / but very unlikely.
However, speaking of letting an LLMs trade stocks without understanding how the LLM will come to a decision... too risky for my taste ;-)
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#145Earlier quoted context omitted.
That's why I said "mostly". I know they're just statistical models, and that having conversations with them is like having a conversation with a stack of dice. But if the simulation is good enough to be useful, the fact that they don't genuinely "understand" doesn't really matter to me. I've had tens of thousands of "conversations" with these things now (I know because I log them all). Whether or not they understand…
I guess I respect that you're stating it honestly, but this is a statement of belief or faith. I think it is something that you should disclose perhaps more often because it doesn't stem from other first principles and is I guess actually just tautological. I guess this is also getting more precise with our fundamental disagreement, I guess I just wouldn't blog about things that are beliefs as if they are the technol…
I don't need belief or faith to get use and entertainment out of the transformers. As Simon said, good enough.
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#146Earlier quoted context omitted.
You're missing the point. Most tools are deployed by humans. If they do something bad, we can blame the human for using the tool badly. And we can predict when a bad choice by the human operator will lead to a bad outcome. Letting the LLM run the tool unsupervised is another thing entirely. We do not understand the choices the machines are making. They are unpredictable and you can't root-cause their decisions. LLM t…
But why can we not hold humans responsible in the case of LLM? You do have to go out of your way to do all of these things with an LLM. And it is the human that does it. It is the humans that give it the permission to act on their behalf. We can definitely hold humans responsible. The question is: are we going to?
I think intent matters.
Let's say you are making an AI-controlled radiation therapy machine. You prompt and train and eval the system very carefully, and you are quite sure it won't overdose any patients. Well, that's not really good enough, it can still screw up. But did you do anything wrong? Not really, you followed best practices and didn't make any mistakes. The LLM just sometimes kills people. You didn't intend that at all.
I make this point because this is already how these systems work today. But instead of giving you a lethal dose of radiation, it uses slurs or promotes genocide or something else. The builders of those bots didn't intend that, and in all likelihood tried very hard to prevent it. It's not very fair to blame them.
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#147Thank you for this release. I believe your library is a key component to unlocking the potential of LLMs without the limitations/restricitions of existing clients. Since you released version 0.26 alpha, I’ve been trying to create a plugin to interact with a some MCP server, but it’s a bit too challenging for me. So far, I’ve managed to connect and dynamically retrieve and use tools, but I’m not yet able to pass param…
Yeah I had a bit of an experiment with MCP this morning, to see if I could get a quick plugin demo out for it. It's a bit tricky! The official mcp Python library really wants you to run asyncio and connect to the server and introspect the available tools.
Hi Simon!
I'm a heavy user of the llm tool, so as soon as I saw your post, I started tinkering with MCP.
I’ve just published an alpha version that works with stdio-based MCP servers (tested with @modelcontextprotocol/server-filesystem) - https://github.com/Virtuslab/llm-tools-mcp. Very early stage, so please make sure to use with --ta option (Manually approve every tool execution).
The code is still messy and there are a couple of TODOs in the README.md, but I plan to work on it full-time until the end of the week.
Some questions:
Where do you think mcp.json should be stored? Also, it might be a bit inconvenient to specify tools one by one with -T. Do you think adding a --all-tools flag or supporting glob patterns like -T name-prefix* in llm would be a good idea?
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#148Earlier quoted context omitted.
Kind of crazy this isn’t sci-fi, it’s just how coding is done now. Future generations are going to wonder how we ever got anything done, the same way we wonder how assembly programmers managed to.
The transition from assembly to C was to a different layer of abstraction within the same context of deterministic computation. The transition from programming to LLM prompting is to a qualitatively different context, because the process is no longer deterministic, nor debuggable. So your analogy fails to apply in a meaningful way to this situation.
Ultimately it’s about being able to create features within a certain period of time, not just to write code. And in that context the shift from assembly to C and the shift from deterministic programming to LLM prompting seem to have similar magnitudes of impact.
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#149Earlier quoted context omitted.
I guess I'm having trouble following this, what is the non-productive quagmire? It's someone's Show HN and you started grumping at them out of nowhere - that's what it looks like externally. There is no right way to do that on HN just like there isn't in most typical social contexts. You can just offer your critique without the overwrought bombast, it's what most other people do. I imagine you don't begin work emails…
I guess I see the overton window in the other direction from this? If I had a coworker who made a random insult machine that emailed other employees random, but deeply personal insults, and some people thought it was funny, and the CEO said that we should reflect on those insults and learning something, I would most definitely begin my emails with probably a stronger tone. And that is the reality of the fatigue that…
If I had a coworker who made a random insult machine
Nothing of the sort happened, though? Why even make this gigantically escalatory analogy? You don't have to like the work presented and you still have the option of non-yelly critique or saying nothing. You're acting like you've done the work of persuasion for your position and everyone is or should be as incensed as you are. And that's clearly not the case.
Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins
#150Almost like RiveScript from Perl running code from within a chatbot :)
Things now come in full circle :D