Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tools: Code Is All You Need

lucumr.pocoo.org

Post reply on HN
Print view View original

71–80 of 234 posts

Re: Tools: Code Is All You Need

#71

Post by rapind »

antirez wrote: post #48

I have the feeling that's not really MCP specifically VS other ways, but it is pretty simply: at the current state of AI, to have a human in the loop is much better. LLMs are great at certain tasks but they often get trapped into local minima, if you do the back and forth via the web interface of LLMs, ask it to write a program, look at it, provide hints to improve it, test it, ..., you get much better results and yo…

So I run my own business (product), I code everything, and I use claude-code. I also wear all the other hats and so I'd be happy to let Claude handle all of the coding if / when it can. I can confirm we're certainly not there yet.

It's definitely useful, but you have to read everything. I'm working in a type-safe functional compiled language too. I'd be scared to try this flow in a less "correctness enforced" language.

That being said, I do find that it works well. It's not living up to the hype, but most of that hype was obvious nonsense. It continues to surprise me with it's grasp on concepts and is definitely saving me some time, and more importantly making some larger tasks more approachable since I can split my time better.

Re: Tools: Code Is All You Need

#72

Post by dingnuts »

mritchie712 wrote: post #42
jayd16 wrote: post #27

Earlier quoted context omitted.

I feel like I'm taking crazy pills sometimes. You have a file with a set of snippets and you prefer to ask the AI to hopefully run them instead of just running it yourself?

the snippets are examples. You can ask hundreds of variations of similar, but different, complex questions and the LLM can adjust the example for that need. I don't have a snippet for, "find all 500's for the meltano service for duckdb syntax errors", but it'd easily nail that given the existing examples.

but if I know enough about the service to write examples, most of the time I will know the command I want, which is less typing, faster, costs less, and doesn't waste a ton of electricity.

In the other cases I see what the computer outputs, LEARN, and then the functionality of finding what I need just isn't useful next time. Next time I just type the command.

I don't get it.

Re: Tools: Code Is All You Need

#73

Post by nico »

simonw wrote: post #25

Something I've realized about LLM tool use is that it means that if you can reduce a problem to something that can be solved by an LLM in a sandbox using tools in a loop, you can brute force that problem. The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide and how to define the success criteria for the model. That still takes significant skill an…

> LLM in a sandbox using tools in a loop, you can brute force that problem

Does this require using big models through their APIs and spending a lot of tokens?

Or can this be done either with local models (probably very slow), or with subscriptions like Claude Code with Pro (without hitting the rate/usage limits)?

I saw the Mandelbrot experiment, it was very cool, but still a rather small project, not really comparable to a complex/bigger/older code base for a platform used in production

Re: Tools: Code Is All You Need

#74

Post by simonw »

nico wrote: post #73
simonw wrote: post #25

Something I've realized about LLM tool use is that it means that if you can reduce a problem to something that can be solved by an LLM in a sandbox using tools in a loop, you can brute force that problem. The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide and how to define the success criteria for the model. That still takes significant skill an…

> LLM in a sandbox using tools in a loop, you can brute force that problem Does this require using big models through their APIs and spending a lot of tokens? Or can this be done either with local models (probably very slow), or with subscriptions like Claude Code with Pro (without hitting the rate/usage limits)? I saw the Mandelbrot experiment, it was very cool, but still a rather small project, not really comparabl…

The local models aren't quite good enough for this yet in my experience - the big hosted models (o3, Gemini 2.5, Claude 4) only just crossed the capability threshold for this to start working well.

I think it's possible we'll see a local model that can do this well within the next few months though - it needs good tool calling, not an encyclopedic knowledge of the world. Might be possible to fit that in a model that runs locally.

Re: Tools: Code Is All You Need

#75

Post by simonw »

dist-epoch wrote: post #49
simonw wrote: post #25

Something I've realized about LLM tool use is that it means that if you can reduce a problem to something that can be solved by an LLM in a sandbox using tools in a loop, you can brute force that problem. The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide and how to define the success criteria for the model. That still takes significant skill an…

I've been using a VM for a sandbox, just to make sure it won't delete my files if it goes insane. With some host data directories mounted read only inside the VM. This creates some friction though. Feels like a tool which runs the AI agent in a VM, but then copies it's output to the host machine after some checks would help, so that it would feel that you are running it natively on the host.

Have you tried giving the model a fresh checkout in a read-write volume?

Re: Tools: Code Is All You Need

#76

Post by chamomeal »

simonw wrote: post #25

Something I've realized about LLM tool use is that it means that if you can reduce a problem to something that can be solved by an LLM in a sandbox using tools in a loop, you can brute force that problem. The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide and how to define the success criteria for the model. That still takes significant skill an…

That’s super cool, I’m glad you shared this!

I’ve been thinking about using LLMs for brute forcing problems too.

Like LLMs kinda suck at typescript generics. They’re surprisingly bad at them. Probably because it’s easy to write generics that look correct, but are then screwy in many scenarios. Which is also why generics are hard for humans.

If you could have any LLM actually use TSC, it could run tests, make sure things are inferring correctly, etc. it could just keep trying until it works. I’m not sure this is a way to produce understandable or maintainable generics, but it would be pretty neat.

Also while typing this is realized that cursor can see typescript errors. All I need are some utility testing types, and I could have cursor write the tests and then brute force the problem!

If I ever actually do this I’ll update this comment lol

Re: Tools: Code Is All You Need

#77

Post by luckystarr »

I always dreamed of a tool which would know the intent, semantic and constraints of all inputs and outputs of any piece of code and thus could combine these code pieces automatically. It was always a fuzzy idea in my head, but this piece now made it a bit more clear. While LLMs could generate those adapters between distinct pieces automatically, it's a expensive (latency, tokens) process. Having a system with which not only to type the variables, but also to type the types (intents, semantic meaning, etc.) would be helpful but likely not sufficient. There has been so much work on ontologies, semantic networks, logical inference, etc. but all of it is spread all over the place. I'd like to have something like this integrated into a programming language and see what it feels like.

Re: Tools: Code Is All You Need

#79

Post by datameta »

whiplash451 wrote: post #32
pclowes wrote: post #5

Directionally I think this is right. Most LLM usage at scale tends to be filling the gaps between two hardened interfaces. The reliability comes not from the LLM inference and generation but the interfaces themselves only allowing certain configuration to work with them. LLM output is often coerced back into something more deterministic such as types, or DB primary keys. The value of the LLM is determined by how well…

Interesting take but too bearish on LLMs in my opinion. LLMs have already found large-scale usage (deep research, translation) which makes them more ubiquitous today than 3D printers ever will or could have been.

Without trying to take away from your assertion, I think it is worthwhile to mention that part of this phenomenon is the unavoidable matter of meatspace being expensive and dataspace being intangibly present everywhere.
Post reply on HN