Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents that run while I sleep

claudecodecamp.com

Post reply on HN
Print view View original

511–520 of 532 posts

Re: Agents that run while I sleep

#511

Post by silentsvn »

girvo wrote: post #504
silentsvn wrote: post #395

Earlier quoted context omitted.

Human-driven compaction is interesting — you sidestep the "what's worth keeping" problem by putting a person in the loop. The tradeoff I've hit is that agents running autonomously need it to happen automatically or coherence degrades fast between sessions. For pruning we landed on a last-touched timestamp + recall frequency counter per memory. Things not accessed in N sessions that were weakly formed to begin with ge…

> The tradeoff I've hit is that agents running autonomously need it to happen automatically or coherence degrades fast between sessions. Yeah that makes total sense. I wonder (and am sure the labs are doing so) if the HitL output would be good to fine tune the models used to do it autonomously? I’m sticking with humans for the moment because I’m not sure where the boundaries lie: what actually makes it better and wha…

> I’m sticking with humans for the moment Haha totally get this statement.

The HitL fine-tuning angle is exactly right. The labeled dataset you're building (good/bad/stylistically-wrong memory events) is probably worth more than the compaction itself. Coherence preferences are surprisingly personal — what reads as "not correct based on my style" is hard to spec without examples.

The loop-pruning maps really cleanly to the contradiction detection in our setup. A model circling the same state N times is often because it stored an inconclusive result with the same confidence as a resolved one they look identical at recall time. Tagging memory entries with a status [open, resolved, or contradicted] before they go in cuts a lot of that.

On the autonomy question: we ended up treating certainty as continuous rather than binary. Low-certainty memories stay soft, high-certainty ones get promoted. Automatic compaction only operates on the low end, higher certainty entries are off-limits without explicit override. That lets you keep the autonomy without the coherence risk. The failure mode shifts from "deleted something important" to "kept something stale too long," which feels more recoverable.

Would be curious what your pruning signal looks like at the turn level — are you scoring relevance per-turn retroactively, or flagging at write time?

Re: Agents that run while I sleep

#512

Post by pepoluan »

9wzYQbTYsAIc wrote: post #508
ruszki wrote: post #507

Earlier quoted context omitted.

For that you need to create something which you know exactly how you want to code, or what architecture is needed. In other words, you would win basically nothing, because typing was never the real bottleneck (no matter what VIM and Emacs people would tell you). LLMs also make mistakes even way lower level than those one pagers allow you to control with the planning mode. Which I use all the time btw. And anyway, the…

Ah, this is true - for my purposes, I've been directing the design and deliberating on the constraints and specifications for a larger system in tandem with smaller planning sessions. That has worked well so far, but yes, you are totally right, there are still quite a few pain points and it is still rather far from being fire-and-forget "build me a fancy landing page for a turnkey business" and getting enterprise qua…

You know, with all the babysitting needed, I wonder if effort is not better spent in just, you know, writing code.

Can you actually quantify the time & effort 'saved' letting LLM generate code for you?

Re: Agents that run while I sleep

#513

Post by pepoluan »

rurban wrote: post #260
jack_pp wrote: post #240

Earlier quoted context omitted.

I worked at a startup, they built their API in PHP because it was easy and fast. Now they're successful, app doesn't scale, high latency etc. What does their php code do? 95% of it is calling a DB. You're telling me today with LLM power multiplier it's THAT much faster to write in PHP compared to something that can actually have a future?

PHP did better than python and perl. Python is doomed. PHP got a good jit already, a good OO lately, good frameworks, stable extensions. It has a company behind. Unlike python or ruby which break right and left all the time on updates. you have to use bunkers of venvs, without any security updates. A nightmare. PHP can scale and has a future.

What's so bad about Python's virtualenv? It's a good way to have an LKGC (Last Known Good Configuration). When a CVE happens, you spin up a new venv and do smoke test, canary test, blue/green deployment, and so on.

If the update breaks things, you go back to the old venv. If the update goes well, then you just delete the old venv.

Not implementing security updates the right way is just human laziness and has nothing to do with the language in use.

Re: Agents that run while I sleep

#514

Post by 9wzYQbTYsAIc »

pepoluan wrote: post #512
9wzYQbTYsAIc wrote: post #508

Earlier quoted context omitted.

Ah, this is true - for my purposes, I've been directing the design and deliberating on the constraints and specifications for a larger system in tandem with smaller planning sessions. That has worked well so far, but yes, you are totally right, there are still quite a few pain points and it is still rather far from being fire-and-forget "build me a fancy landing page for a turnkey business" and getting enterprise qua…

You know, with all the babysitting needed, I wonder if effort is not better spent in just, you know, writing code. Can you actually quantify the time & effort 'saved' letting LLM generate code for you?

For me, personally, I'm building things that would have been impractical for me to do as cleanly within the same amount of time - prototypes in languages that I don't have the muscle memory for, using algorithms i have a surface level understanding of but would need time to deeply understand and implement by hand, and, at my pace, as a retired dev, is probably quantified in terms of years worth of time and effort saved.

edit: also, would I take the time to implement LCARS by hand? No. But with an LLM, sure, took it about 3 minutes or less to implement a pretty decent LCARS interface for me.

Re: Agents that run while I sleep

#515

Post by woah »

krater23 wrote: post #250
woah wrote: post #145

Earlier quoted context omitted.

Build features faster. Granted, this exposes the difference between people who like to finish projects and people who like to get paid a lot of money for typing on a keyboard.

Bullshit! You project isn't finished as long as there are obvious major bugs that you can't fix because you don't unterstand the code.

Why does understanding computer science principles and software architecture and instructing a person or an ai on how to fix them require typing every line yourself?

Re: Agents that run while I sleep

#516

Post by godelski »

josephg wrote: post #509
godelski wrote: post #489

Earlier quoted context omitted.

> Tests only pass if both implementations of your software behave the same way. That's not true. I even addressed this in my comment as did Dijkstra

What is untrue about this statement you quoted?

You can have software behave differently while passing the same tests.

Idk man, this is pretty easy to demonstrate. Start with a trivial example: test is that input (2,2) -> 4. Function 1 does multiplication, function 2 does exponentiation. Both functions pass the test.

Sure, simple example but illustrative examples should be simple. But add more complexity and I'll add more examples of functions where the outputs are the same for a given set of inputs. (There's a whole area of mathematics dedicated to this!) It's simple, but you also confidently claimed something that was trivial to disprove.

Your claim is true if and only if your tests have complete coverage. So, your claim is only true if you've done formal verification of your code. Which was what I said in the beginning and is what Dijkstra claimed as well.

Re: Agents that run while I sleep

#517

Post by godelski »

UK-Al05 wrote: post #372
godelski wrote: post #259

Earlier quoted context omitted.

TDD has always been flawed. Tests can't give you complete coverage, they are always incomplete. Though every time I say this people think I'm against tests. I'm just saying tests can't prove correctness. You'd have to be a lunatic to think they are proofs. Even crazier is having the LLMs write their own tests and think that that's proof. I'm sure it improves things, but proofs are a different beast all together. Seem…

It's not meant to give you complete coverage. It's meant to guide to meeting the acceptance criteria.

Then you agree. But the existence of unit tests does not mean you're doing TDD.

Re: Agents that run while I sleep

#518

Post by godelski »

rurban wrote: post #283
godelski wrote: post #259

Earlier quoted context omitted.

TDD has always been flawed. Tests can't give you complete coverage, they are always incomplete. Though every time I say this people think I'm against tests. I'm just saying tests can't prove correctness. You'd have to be a lunatic to think they are proofs. Even crazier is having the LLMs write their own tests and think that that's proof. I'm sure it improves things, but proofs are a different beast all together. Seem…

Of course tests are not proofs. For proofs I do 'make verify' :) Tests just catch the most simple mistakes, edge cases and some regressions.

So you're agreeing? The problem with TDD isn't the T it's the (first) D

Re: Agents that run while I sleep

#519

Post by girvo »

silentsvn wrote: post #511
girvo wrote: post #504

Earlier quoted context omitted.

> The tradeoff I've hit is that agents running autonomously need it to happen automatically or coherence degrades fast between sessions. Yeah that makes total sense. I wonder (and am sure the labs are doing so) if the HitL output would be good to fine tune the models used to do it autonomously? I’m sticking with humans for the moment because I’m not sure where the boundaries lie: what actually makes it better and wha…

> I’m sticking with humans for the moment Haha totally get this statement. The HitL fine-tuning angle is exactly right. The labeled dataset you're building (good/bad/stylistically-wrong memory events) is probably worth more than the compaction itself. Coherence preferences are surprisingly personal — what reads as "not correct based on my style" is hard to spec without examples. The loop-pruning maps really cleanly t…

Semi-retroactively: my agent has a command to /compact and its then that I pop the interface. It gets opened automatically if the context is full, too, and then I've gone back and fed some recorded sessions into it as well days later too, to test things out. Still getting the hang of it, but I won't be surprised to see much bigger teams/companies do something similar (I assume they are already, really)

Re: Agents that run while I sleep

#520

Post by wahnfrieden »

zarzavat wrote: post #301
aprdm wrote: post #249

Earlier quoted context omitted.

Why read code when you are getting results fast ? See https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16d... I am not kidding. People don't seem to understand what's actually happening in our industry. See https://www.linkedin.com/posts/johubbard_github-eleutherailm...

I'm not getting results. That's the point. Claude doesn't fucking work without human intervention. When left to its own devices it makes bad decisions. It writes bad code. It needs constant supervision to stop it from going off the rails and replacing working code with broken code. It doesn't know what it's doing! It's about as far as you can get from being able to work independently. Yegge is an entertainer. Gas Tow…

Use Codex
Post reply on HN