Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Token anxiety", a slot machine by any other name

jkap.io

Post reply on HN
Print view View original

211–220 of 249 posts

Re: "Token anxiety", a slot machine by any other name

#211

Post by jonahrd »

QuadrupleA wrote: post #188
jonahrd wrote: post #167

Earlier quoted context omitted.

I might be able to shine a little light on this. I came from embedded, where I wasn't able to use agents very effectively for anything other than quick round trip iterative stuff. They were still really useful, but I definitely could never envision just letting an agent run unattended. But I recently switched domains into vaguely "fullstack web" using very popular frameworks. If I spend a good portion of my day going…

The amount of boilerplate people talk about seems like the fault of these big modern frameworks honestly. A good system design shouldn't HAVE so much boilerplate. Think people would be better off simplifying and eliminating it deterministically before reaching for the LLM slot machine.

I'm not so sure I agree. To me it's somewhat magical that I can write even this amount of code and have this stuff just magically work on pretty much every platform via docker, the web platform, etc. Maybe this again is me having started with embedded, but I am blown away at the ratio of actual code to portability we currently have.

Re: "Token anxiety", a slot machine by any other name

#212

Post by phplovesong »

ctoth wrote: post #11

The gambling analogy completely falls apart on inspection. Slot machines have variable reward schedules by design — every element is optimized to maximize time on device. Social media optimizes for engagement, and compulsive behavior is the predictable output. The optimization target produces the addiction. What's Anthropic's optimization target??? Getting you the right answer as fast as possible! The variability in…

Claude RARELY get it right on the fifth time. Usually i write the damn thing when my account is on "cooldown".

Re: "Token anxiety", a slot machine by any other name

#213

Post by mrnotcrazy »

HolyLampshade wrote: post #151

I know I'm running a bit late to the party here, but maybe someone can provide some color that I (on the slightly older end of the spectrum when it comes to this) don't fully understand. When people talk about leaving their agents to run overnight, what are those agents actually doing? The limited utility I've had using agent-supported software development requires a significant amount of hand holding, maybe because…

I have agents run at night to work through complicated TTRPG campaigns. For example I have a script that runs all night simulating NPCs before a session. The NPCs have character sheets + motivations and the LLMs do one prompt per NPC in stages so combat can happen after social interactions. IF you run enough of these and make the prompts well written you can save a lot of time. You can't like... simulate the start of a campaign and then jump in. Its more like you know there is a big event, you already have characters, you can throw them in a folder to see how things would cook all else being equal and then use that to riff off of when you actually write your notes.

I think of my agents like golems from disc world, they are defined by their script. Adding texture to them improves the results so I usually keep a running tally of what they have worked on and add that to the header. They are a prompt in a folder that a script loops over and sends to gemeni(spawning an agent and moving to the next golem script)

I also was curious to see if it could be used it for developing some small games, whenever I would run into a problem I couldn't be bothered to solve or needed a variety of something I would let a few llms work on it so in the morning I had something to bounce off. I had pretty good success with this for RTS games and shooting games where variety is something well documented and creativity is allowed. I imagine there could be a use here, I've been calling it dredging cause I imagine myself casting a net down into the slop to find valuables.

I did have an idea where all my sites and UI would be checked against some UI heuristic like Oregon State's inclusivity heuristic but results have been mixed so far. The initial reports are fine, the implementation plans are ok but it seems like the loop of examine, fix, examine... has too much drift? That does seem solvable but I have a concern that this is like two lines that never touch but get closer as you approach infinity.

There is some usefulness in running these guys all night but I'm still figuring out when its useful and when its a waste of resources.

Re: "Token anxiety", a slot machine by any other name

#214

Post by mmasu »

HolyLampshade wrote: post #151

I know I'm running a bit late to the party here, but maybe someone can provide some color that I (on the slightly older end of the spectrum when it comes to this) don't fully understand. When people talk about leaving their agents to run overnight, what are those agents actually doing? The limited utility I've had using agent-supported software development requires a significant amount of hand holding, maybe because…

I will give you an example I heard from an acquaintance yesterday - this person is very smart but not strictly “technical”.

He is building a trading automation for personal use. In his design he gets a message on whatsapp/signal/telegram and approves/rejects the trade suggestion.

To define specifications for this, he defined multiple agents (a quant, a data scientist, a principal engineer, and trading experts - “warren buffett”, “ray dalio”) and let the agents run until they reached a consensus on what the design should be. He said this ran for a couple of hours (so not strictly overnight) after he went to sleep; in the morning he read and amended the output (10s of pages equivalent) and let it build.

This is not a strictly-defined coding task, but there are now many examples of emerging patterns where you have multiple agents supporting each other, running tasks in parallel, correcting/criticising/challenging each other, until some definition of “done” has been satisfied.

That said, personally my usage is much like yours - I run agents one at a time and closely monitor output before proceeding, to avoid finding a clusterfuck of bad choices built on top of each other. So you are not alone my friend :-)

Re: "Token anxiety", a slot machine by any other name

#215

Post by ASalazarMX »

jonahrd wrote: post #211
QuadrupleA wrote: post #188

Earlier quoted context omitted.

The amount of boilerplate people talk about seems like the fault of these big modern frameworks honestly. A good system design shouldn't HAVE so much boilerplate. Think people would be better off simplifying and eliminating it deterministically before reaching for the LLM slot machine.

I'm not so sure I agree. To me it's somewhat magical that I can write even this amount of code and have this stuff just magically work on pretty much every platform via docker, the web platform, etc. Maybe this again is me having started with embedded, but I am blown away at the ratio of actual code to portability we currently have.

> To me it's somewhat magical that I can write even this amount of code

It's because you're not writing it, you adopted the role of Project Manager or Chief Engineer. How much cognitive debt are you accumulating?

Re: "Token anxiety", a slot machine by any other name

#216

Post by ASalazarMX »

evmaki wrote: post #67
ctoth wrote: post #11

The gambling analogy completely falls apart on inspection. Slot machines have variable reward schedules by design — every element is optimized to maximize time on device. Social media optimizes for engagement, and compulsive behavior is the predictable output. The optimization target produces the addiction. What's Anthropic's optimization target??? Getting you the right answer as fast as possible! The variability in…

The LLM is not the slot machine. The LLM is the lever of the slot machine, and the slot machine itself is capitalism. Pull the lever, see if it generates a marketable product or moment of virality, get rich if you hit the jackpot. If not, pull again.

I don't know why you were downvoted. This is the FOMO that encourages agent gambling, automated experimentation in the hopes of accidentally striking digital gold before your peers do. A million monkeys racing 24/7 to create the next Harry Potter first.

Ideas are a dime a dozen, now proofs of concept are a load of tokens a dozen.

Re: "Token anxiety", a slot machine by any other name

#217

Post by thwarted »

LeonidBugaev wrote: post #187
HolyLampshade wrote: post #151

I know I'm running a bit late to the party here, but maybe someone can provide some color that I (on the slightly older end of the spectrum when it comes to this) don't fully understand. When people talk about leaving their agents to run overnight, what are those agents actually doing? The limited utility I've had using agent-supported software development requires a significant amount of hand holding, maybe because…

I had a few useful examples of this. In order to make it work you need to define your quality gates, and rather complex spec. I personally use https://github.com/probelabs/visor for creating the gates. It can be a code-review gate, or how well implementation align with the spec and etc. And basically it makes agent loop until it pass it. One of the tips, especially when using Claude Code, is explictly ask to create a…

> One of the tips, especially when using Claude Code, is explictly ask to create a "tasks", and also use subagents. For example I want to validate and re-structure all my documentation - I would ask it to create a task to research state of my docs, then after create a task per specific detail, then create a task to re-validate quality after it has finished task.

This is definitely a way to keep those who wear Program and Project manager hats busy.

Re: "Token anxiety", a slot machine by any other name

#218

Post by BatteryMountain »

hattmall wrote: post #210
BatteryMountain wrote: post #204

Earlier quoted context omitted.

Spin up a mid sized linux vm (or any machine with 8 or 12 cores will do with at least 16GB RAM with nmve). Add 10 users. Install claude 10 times (one per user). Clone repo 10 times (one per user). Have a centralized place to get tasks from (db, trello, txt, etc) - this is the memory. Have a cron wake up every 10 minutes and call your script. Your script calls claude in non-interactive mode + auto accept. It grabs a n…

So is this something you do with a monthly subscription or is this using API tokens?

At first used Claude Max x5, but we are using the api now.

We only give it very targeted tasks, no broad strokes. We have a couple of "prompt" templates, which we select when creating tasks. The new opus model one shots about 90% of tasks we throw at it. Getting a ton of value from diagnostic tasks, it can troubleshoot really quickly (by ingesting logs, exceptions, some db rows).

Re: "Token anxiety", a slot machine by any other name

#219

Post by nerdsniper »

jascha_eng wrote: post #195
nerdsniper wrote: post #186

Earlier quoted context omitted.

This is exactly my experience as well. The feature interaction matrix is growing as models get better, and I tend to build "prompt library components" for each project which saves time on "you prompt very precisely but this also takes time". But so far that doesn't change the reality - I can't find any opportunities to let an agent run for more than 30 minutes at best, and parallel agents just seem to confuse each ot…

idk I haven't really hit the point with any llm that it comes up with useful abstractions on its own unless those abstractions have been in the training data. E.g. imagine building a google docs clone where you have different formatting options. Claude would happily build bold and italic for you but if afterwards you add headings, tables, colors, font size, etc. It would just produce a huge if/else tree instead of bu…

FWIW I’ve used LLMs to invent new things. Not super groundbreaking fundamental research, but they were able to use physics to design a device that didn’t exist yet, from first principles.

Re: "Token anxiety", a slot machine by any other name

#220

Post by flerchin »

HolyLampshade wrote: post #184
flerchin wrote: post #154

Earlier quoted context omitted.

This is my experience of it too. Perhaps if it was chunking through a large task like upgrading all of our repos to the latest engine supported by our cloud provider, I could leave it overnight. Even then it would just result in a large daylight backlog of "not quite right" to review and redo.

I think that's the issue I have with using these tools so far (definitely professionally, but even in pet projects for embedded systems). The mental load of having to go back through and make sure all of the lines of code do what the agent claims they do, even with tests, is significantly more than it would take to learn the implementation myself. I can see the utility in creating very simple web-based tools where th…

What I tell my kids is: You know how when you ask AI about something you know very well, how its answers are always somewhat wrong? It's like that for things you do not know very well too.
Post reply on HN