Cool, can you make it use local free models because I'm broke and can't afford AI's crazy costs.
You should write an agent
341–350 of 409 posts
Re: You should write an agent
#342> I’m not even going to bother explaining what an agent is. Does anyone actually know what exactly an agent is?
Yes, and the post says what it is about 100 words later. It's an LLM running in a loop that can access tool calls.
Re: You should write an agent
#343Earlier quoted context omitted.
You and I clearly have a different idea of what "very very simple stuff" involves. Even the small models are very capable of stringing together a short sequence of simple tool calls these days - and if you have 32GB of RAM (eg a ~$1500 laptop) you can run models like gpt-oss:20b which are capable of operating tools like bash in a reasonably useful way. This wasn't true even six months ago - the local models released…
You mean like a demo for simple stuff? Something like hello world type tasks? The small models you mentioned earlier are incapable of doing anything genuinely useful for daily use. The few tasks they can handle are easier and faster to just write yourself with the added assurance that no mistakes will be made. I’d love to have small local models capable of running tools like current SOTA models, but the reality is th…
Yes, I mean a demo for simple stuff. This whole conversation is attached to an article about building the simplest possible tool-in-a-loop agent as a learning exercise for how they work.
Re: You should write an agent
#344Do we need an agent? I get the point of this post: have fun building one because it's easy. But every time I see one of these takes, I keep wondering why do we encourage a tool that would potentially replace us. Why help it build better that could eventually take away what was fun and sustainable income-wise?
Interesting to think that this question could have been asked of almost all software work up until this point, except the "us" was always "someone else "
It’s generally been true, but not close to the scale we’re looking at now. The implied/assumed hypocrisy also doesn’t stop it from it sucking, or make it immune to criticism.
Re: You should write an agent
#345If you want to play with this stuff without spending a lot of money, what are your best options?
Gemini has a generous free tier (2500 prompts per day), all you need is a Google account to get an API key.
Re: You should write an agent
#346I've been having so much fun writing the agent loop for https://revise.io, most fun I've had programming in a long time.
Re: You should write an agent
#347Earlier quoted context omitted.
If you can get a specialized agent to work in its domain at 10% parameters of a foundation model, you can feasibly run locally, which opens up e.g. offline use cases. Personally I’d absolutely buy an LLM in a box which I could connect to my home assistant via usb.
What use cases do you imagine for LLMs in home automation? I have HA and a mini PC capable of running decently sized LLMs but all my home automation is super deterministic (e.g. close window covers 30 minutes after sunset, turn X light on if Y condition, etc.).
maybe not direct automation, but ask-respond loop of your HA data. How are you optimizing your electricity, heating/cooling with respect to local rates, etc
Re: You should write an agent
#348Cool, can you make it use local free models because I'm broke and can't afford AI's crazy costs.
Yep, change nothing in the code in the article but spin up an Ollama server and use the OpenAI API https://docs.ollama.com/api/openai-compatibility.
Re: You should write an agent
#349Earlier quoted context omitted.
The post is just about playing around with the tech for fun. Why does monetization come into it? It feels like saying you don't want to use Python because Astral, the company that makes uv, is operating at a loss. What?
Agents use Apis that I will need to pay for and generally software dev is a job for me that needs to generate income. If the Apis I call are not profitable for the provider then they won't be for me either. This post is a fly.io advertisement
Because if you build an agent you'll need to host it in a cloud virtual machine...? I don't follow.
Re: You should write an agent
#350Earlier quoted context omitted.
> They know all the flags and are generally better at interpreting tool output than I am. In the toy example, you explicitly restrict the agent to supply just a `host`, and hard-code the rest of the command. Is the idea that you'd instead give a `description` something like "invoke the UNIX `ping` command", and a parameter described as constituting all the arguments to `ping`?
Honestly, I didn't think very hard about how to make `ping` do something interesting here, and in serious code I'd give it all the `ping` options (and also run it in a Fly Machine or Sprite where I don't have to bother checking to make sure none of those options gives code exec). It's possible the post would have been better had I done that; it might have come up with an even better test. I was telling a friend onlin…
Or have the agent strace a process and describe what's going on as if you're a 5 year old (because I actually need that to understand strace output)