Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You should write an agent

fly.io

Post reply on HN
Print view View original

231–240 of 409 posts

Re: You should write an agent

#231

Post by aenis »

ht96 wrote: post #110
furyofantares wrote: post #95

Earlier quoted context omitted.

That's why you build extensive tooling to run your change hundreds of times in parallel against the context you're trying to fix, and then re-run hundreds of past scenarios in parallel to verify none of them breaks.

Do you use a tool for this? Is there some sort of tool which collects evals from live inferences (especially those which fail)

For sure, for instance Google has ADK Eval framework. You write tests, and you can easily run them against given input. I'd say its a bit unpolished, as is the rest of the rapidly developing ADK framework, but it does exist.

Re: You should write an agent

#232

Post by sumedh »

andai wrote: post #204
foobarian wrote: post #42

Earlier quoted context omitted.

Honestly the top AI use case for me right now is personal throwaway dev tools. Where I used to write shell oneliners with dozen pipes including greps and seds and jq and other stuff, now I get an AI to write me a node script and throw in a nice Web UI to boot. Edit: reflecting on what the lesson is here, in either case I suppose we're avoiding the pain of dealing with Unix CLI tools :-D

Could you give some examples? I'm having the AI write the shell scripts, wondering if I'm missing out on some comfy UIs...

It can be anything. It depends on what you want to do with the output.

You can have a simple dashboard site which collects the data from our shell scripts and shows your a summary or red/green signals so that you can focus on things which are interested in.

Re: You should write an agent

#233

Post by sumedh »

sibeliuss wrote: post #159

Its easy to create a toy, but much harder to make something right! Like anything, so much weird polish stuff creeps in at the 90% mark.

> so much weird polish stuff creeps in at the 90% mark.

That is where the human in the loop needs to focus on for now :)

Re: You should write an agent

#234

Post by simonw »

cyberpunk wrote: post #229
crystal_revenge wrote: post #225

Earlier quoted context omitted.

The problem with statements like these is that I work with people who make the same claims, but are slowly building useless, buggy monstrosities that for various reasons nobody can/will call out. Obviously I’m reasonably willing to believe that you are an exception. However every person I’ve interacted with who makes this same claim has presented me with a dumpster fire and expected me to marvel at it.

We have gpt-5 and gemini 2.5 pro at work, and both of them produce huge amounts of basically shit code that doesn’t work. Every time i reach for them recently I end up spending more time refactoring the bad code out or in deep hostage negotiations with the chatbot of the day that I would have been faster writing it myself. That and for some reason they occasionally make me really angry. Oh a bunch of prompts in and t…

Are you using them via a coding agent harness such as Codex CLI or Gemini CLI?

Re: You should write an agent

#235

Post by sumedh »

behnamoh wrote: post #6

> nobody knows anything yet that sums up my experience in AI over the past three years. so many projects reinvent the same thing, so much spaghetti thrown at the wall to see what sticks, so much excitement followed by disappointment when a new model drops, so many people grifting, and so many hacks and workarounds like RAG with no evidence of them actually working other than "trust me bro" and trial and error.

That is because for the people for whom AI is actually working/making money they would prefer to keep it a secret on what and how they are doing it, why attract competition?

Re: You should write an agent

#236

Post by sumedh »

simonw wrote: post #148
8cvor6j844qw_d6 wrote: post #101

Question, how hard is it for someone new to agents to dip their toes into writing a simple agent to get data? (e.g., getting reviews from sites for sentiment analysis?) Forgive if I get someting wrong: From what I see, it seems fundamentally it is a LLM being ran each loop with information about tools provided to it. On each loop the LLM evaluates inputs/context (from tool calls, inputs, etc.) and decided which tool…

You can prototype this without writing any code at all. Fire up "claude --dangerously-skip-permissions" in a fresh directory (ideally in a Docker container if you want to limit the chance of it breaking anything else) and prompt this: > Use Playwright to fetch ten reviews from http://www.example.com/ then run sentiment analysis on them and write the results out as JSON files. Install any missing dependencies. Watch w…

Dont you need to setup Playwright MCP first?

Re: You should write an agent

#237

Post by sumedh »

vkou wrote: post #33

> It’s Incredibly Easy client = OpenAI() context_good, context_bad = [{ "role": "system", "content": "you're Alph and you only tell the truth" }], [{ "role": "system", "content": "you're Ralph and you only tell lies" }] ... And this will work great until next week's update when Ralph responses will consist of "I'm sorry, it would be unethical for me to respond with lies, unless you pay for the Premium-Super-Deluxe su…

Its easy to switch to an open source model

Re: You should write an agent

#238

Post by simonw »

sumedh wrote: post #236
simonw wrote: post #148

Earlier quoted context omitted.

You can prototype this without writing any code at all. Fire up "claude --dangerously-skip-permissions" in a fresh directory (ideally in a Docker container if you want to limit the chance of it breaking anything else) and prompt this: > Use Playwright to fetch ten reviews from http://www.example.com/ then run sentiment analysis on them and write the results out as JSON files. Install any missing dependencies. Watch w…

Dont you need to setup Playwright MCP first?

No. I don't use Playwright MCP at all - if the coding agent can run Python code it can use the Playwright Python library directly, if Node.js it can use the Playwright Node library.

Re: You should write an agent

#239

Post by crystal_revenge »

simonw wrote: post #228
crystal_revenge wrote: post #225

Earlier quoted context omitted.

The problem with statements like these is that I work with people who make the same claims, but are slowly building useless, buggy monstrosities that for various reasons nobody can/will call out. Obviously I’m reasonably willing to believe that you are an exception. However every person I’ve interacted with who makes this same claim has presented me with a dumpster fire and expected me to marvel at it.

I'm not going to dispute your own experience with people who aren't using this stuff effectively, but the great thing about the internet is that you can use it to track the people who are making the very best use of any piece of technology.

This line of reasoning is smelling pretty "no true Scotsman" to me. I'm sure there were amazing ColdFusion devs, but that hardly justifies the use of the technology. Likewise "This tool works great on the condition that you need to hire a Simon Willison level dev" is almost a fault. I'm pretty confident you could squeeze some juice out of a Markov Chain (ignoring, of course, that decoder-only LLMs are basically fancy MCs).

In a weird way it sort of reminds me of Common Lisp. When I was younger I thought it was the most beautiful language and a shame that it wasn't more widely adopted. After a few decades in the field I've realized it's probably for the best since the average dev would only use it to create elaborate foot guns.

Post reply on HN