261–270 of 409 posts
Earlier quoted context omitted.
> catch you eventually if you try to extract and reuse access tokens What does that mean?
How do they know your requests come from Claude Code?
Earlier quoted context omitted.
That's why you build extensive tooling to run your change hundreds of times in parallel against the context you're trying to fix, and then re-run hundreds of past scenarios in parallel to verify none of them breaks.
Do you use a tool for this? Is there some sort of tool which collects evals from live inferences (especially those which fail)
darin@mcptesting.com
(gist: evals as a service)
Earlier quoted context omitted.
Some of us have been happily using agentic coding tools (Claude Code etc) since February and we're still not abandoning them for their inherent flaws.
I abandoned Claude Code pretty quickly, I find generic tools give generic answers, but since I do Elixir I’m ”blessed” with Tidewave which gives a much better experience . I hope more people get to experience framework built tooling instead of just generic stuff. It still wants to build an airplane to go out with the trash sometimes and will happily tell you wrong is right. However I much prefer it trying to figure i…
Earlier quoted context omitted.
There are MCP configured virtualization solutions that is supposed to be safe for letting LLM go wild. Like this one: https://github.com/zerocore-ai/microsandbox I haven't tried it.
You can build your agent into a docker image then easily limit both networking and file system scope. docker run -it --rm \ -e SOME_API_KEY="$(SOME_API_KEY)" \ -v "$(shell pwd):/app" \ /dev/null | awk '{printf " --add-host=llm-provider.com:%s", $$0}') \ Probably could be a bit cleaner, but it worked for me.
If you want your agent to pull untrusted code from the internet and go wild while you're doing other stuff it might not be a good choice.
Absolutely, especially the part about just rolling your own alternative to Claude Code - build your own lightsaber. Having your coding agent improve itself is a pretty magical experience. And then you can trivially swap in whatever model you want (Cerebras is crazy fast, for example, which makes a big difference for these many-turn tool call conversations with big lumps of context, though gpt-oss 120b is obviously no…
>build your own lightsaber I think this is the best way of putting it I've heard to date. I started building one just to know what's happening under the hood when I use an off-the-shelf one, but it's actually so straightforward that now I'm adding features I want. I can add them faster than a whole team of developers on a "real" product can add them - because they have a bigger audience. The other takeaway is that ag…
I'm now experimenting with letting the agent generate its own source code from a specification (currently generating 9K lines of Python code (3K of implementation, 6K of tests) from 1.5K lines in specifications (https://alejo.ch/3hi).
I feel like one small piece is missing to call it an agent? The ability to iterate in multiple steps until it feels like it's "done". What is the canonical way to do that? I suspect that implementing that in the wrong way could make it spiral.