Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tools: Code Is All You Need

lucumr.pocoo.org

Post reply on HN
Print view View original

41–50 of 234 posts

Re: Tools: Code Is All You Need

#41

Post by light_hue_1 »

jayd16 wrote: post #27
mritchie712 wrote: post #2

> try completing a GitHub task with the GitHub MCP, then repeat it with the gh CLI tool. You'll almost certainly find the latter uses context far more efficiently and you get to your intended results quicker. This is spot on. I have a "devops" folder with a CLAUDE.md with bash commands for common tasks (e.g. find prod / staging logs with this integration ID). When I complete a novel task (e.g. count all the rows that…

I feel like I'm taking crazy pills sometimes. You have a file with a set of snippets and you prefer to ask the AI to hopefully run them instead of just running it yourself?

Yes. I'm not the poster but I do something similar.

Because now the capabilities of the model grow over time. And I can ask questions that involve a handful of those snippets. When we get to something new that requires some doing, it becomes another snippet.

I can offload everything I used to know about an API and never have to think about it again.

Re: Tools: Code Is All You Need

#42

Post by mritchie712 »

jayd16 wrote: post #27
mritchie712 wrote: post #2

> try completing a GitHub task with the GitHub MCP, then repeat it with the gh CLI tool. You'll almost certainly find the latter uses context far more efficiently and you get to your intended results quicker. This is spot on. I have a "devops" folder with a CLAUDE.md with bash commands for common tasks (e.g. find prod / staging logs with this integration ID). When I complete a novel task (e.g. count all the rows that…

I feel like I'm taking crazy pills sometimes. You have a file with a set of snippets and you prefer to ask the AI to hopefully run them instead of just running it yourself?

the snippets are examples. You can ask hundreds of variations of similar, but different, complex questions and the LLM can adjust the example for that need.

I don't have a snippet for, "find all 500's for the meltano service for duckdb syntax errors", but it'd easily nail that given the existing examples.

Re: Tools: Code Is All You Need

#43

Post by lsaferite »

forrestthewoods wrote: post #31

Unpopular Opinion: I hate Bash. Hate it. And hate the ecosystem of Unix CLIs that are from the 80s and have the most obtuse, inscrutable APIs ever designed. Also this ecosystem doesn’t work on Windows — which, as a game dev, is my primary environment. And no, WSL does not count. I don’t think the world needs yet another shell scripting language. They’re all pretty mediocre at best. But maybe this is an opportunity to…

Python CAN be a "shell script" in this case though...

Tool composition over stdio will get you very very far. That's what an interface "from the 80s" does for you 45 years later. That same stdio composability is easily pipe into/through any number of cli tools written in any number of languages, compiled and interpreted.

Re: Tools: Code Is All You Need

#44

Post by jumploops »

We’re playing an endless cat and mouse game of capabilities between old and new right now.

Claude Code shows that the models can excel at using “old” programmatic interfaces (CLIs) to do Real Work™.

MCP is a way to dynamically provide “new” programmatic interfaces to the models.

At some point this will start to converge, or at least appear to do so, as the majority of tools a model needs will be in its pre-training set.

Then we’ll argue about MPPP (model pre-training protocol pipeline), and how to reduce knowledge pollution of all the LLM-generated tools we’re passing to the model.

Eventually we’ll publish the Merrium-Webster Model Tool Dictionary (MWMTD), surfacing all of the approved tools hidden in the pre-training set.

Then the kids will come up with Model Context Slang (MCS), in an attempt to use the models to dynamically choose unapproved tools, for much fun and enjoyment.

Ad infinitum.

Re: Tools: Code Is All You Need

#45

Post by dingnuts »

whiplash451 wrote: post #32
pclowes wrote: post #5

Directionally I think this is right. Most LLM usage at scale tends to be filling the gaps between two hardened interfaces. The reliability comes not from the LLM inference and generation but the interfaces themselves only allowing certain configuration to work with them. LLM output is often coerced back into something more deterministic such as types, or DB primary keys. The value of the LLM is determined by how well…

Interesting take but too bearish on LLMs in my opinion. LLMs have already found large-scale usage (deep research, translation) which makes them more ubiquitous today than 3D printers ever will or could have been.

And yet you didn't provide a single reference link! Every case of LLM usage that I've seen claimed about those things has been largely a lie -- guess you won't take the opportunity to be the first to present a real example. Just another rumor.

Re: Tools: Code Is All You Need

#46

Post by JyB »

> It demands too much context.

This is solved trivially by having default initial prompts. All majors tools like Claude Code or Gemini CLI have ways to set them up.

> You pass all your tools to an LLM and ask it to filter it down based on the task at hand. So far, there hasn't been much better approaches proposed.

Why is a "better" approach needed? If modern LLMs can properly figure it out? It's not like LLMs don't keep getting better with larger and larger context length. I never had a problem with an LLM struggling to use the appropriate MCP function on it's own.

> But you run into three problems: cost, speed, and general reliability

- cost: They keep getting cheaper and cheaper. It's ridiculously inexpensive for what those tools provide.

- speed: That seem extremely short sighted. No one is sitting idle looking at Claude Code in their terminal. And you can have more than one working on unrelated topics. That defeats the purpose. No matter how long it takes the time spent is purely bonus. You don't have to spend time in the loop when asking well defined tasks.

- reliability: Seem very prompt correlated ATM. I guess some people don't know what to ask which is the main issue.

Having LLMS being able to complete tedious tasks involving so many external tools at once is simply amazing thanks to MCP. Anecdotal but just today it did a task flawlessly involving: Notion pages, Linear Ticket, git, GitHub PR, GitHub CI logs. Being in the loop was just submitting one review on the PR. All the while I was busy doing something else. And for what, ~1$?

Re: Tools: Code Is All You Need

#47

Post by benreesman »

whiplash451 wrote: post #32
pclowes wrote: post #5

Directionally I think this is right. Most LLM usage at scale tends to be filling the gaps between two hardened interfaces. The reliability comes not from the LLM inference and generation but the interfaces themselves only allowing certain configuration to work with them. LLM output is often coerced back into something more deterministic such as types, or DB primary keys. The value of the LLM is determined by how well…

Interesting take but too bearish on LLMs in my opinion. LLMs have already found large-scale usage (deep research, translation) which makes them more ubiquitous today than 3D printers ever will or could have been.

What we call an LLM today (by which almost everyone means an autogressive language model from the Generative Pretrained Transformer family tree, and BERTs are still doing important eork, believe that) is actually an offshoot of neural machine translation.

This isn't (intentionally at least) mere HN pedantry: they really do act like translation tools in a bunch of observable ways.

And while they have recently crossed the threshold into "yeah, I'm always going to have a gptel buffer open now" territory at the extreme high end, their utility outside of the really specific, totally non-generalizing code lookup gizmo usecase remains a claim unsupported by robust profits.

There is a hole in the ground where something between 100 billion and a trillion dollars in the ground that so far has about 20B in revenue (not profit) going into it annually.

AI is going to be big (it was big ten years ago).

LLMs? Look more and more like the Metaverse every day as concerns the economics.

Re: Tools: Code Is All You Need

#48

Post by antirez »

I have the feeling that's not really MCP specifically VS other ways, but it is pretty simply: at the current state of AI, to have a human in the loop is much better. LLMs are great at certain tasks but they often get trapped into local minima, if you do the back and forth via the web interface of LLMs, ask it to write a program, look at it, provide hints to improve it, test it, ..., you get much better results and you don't cut yourself out to find a 10k lines of code mess that could be 400 lines of clear code. That's the current state of affairs, but of course many will try very hard to replace programmers that is currently not possible. What it is possible is to accelerate the work of a programmer several times (but they must be good both at programming and LLM usage), or take a smart person that has a relatively low skill in some technology, and thanks to LLM make this person productive in this field without the long training otherwise needed. And many other things. But "agentic coding" right now does not work well. This will change, but right now the real gain is to use the LLM as a colleague.

It is not MCP: it is autonomous agents that don't get feedbacks from smart humans.

Re: Tools: Code Is All You Need

#49

Post by dist-epoch »

simonw wrote: post #25

Something I've realized about LLM tool use is that it means that if you can reduce a problem to something that can be solved by an LLM in a sandbox using tools in a loop, you can brute force that problem. The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide and how to define the success criteria for the model. That still takes significant skill an…

I've been using a VM for a sandbox, just to make sure it won't delete my files if it goes insane.

With some host data directories mounted read only inside the VM.

This creates some friction though. Feels like a tool which runs the AI agent in a VM, but then copies it's output to the host machine after some checks would help, so that it would feel that you are running it natively on the host.

Re: Tools: Code Is All You Need

#50

Post by the_mitsuhiko »

JyB wrote: post #46

> It demands too much context. This is solved trivially by having default initial prompts. All majors tools like Claude Code or Gemini CLI have ways to set them up. > You pass all your tools to an LLM and ask it to filter it down based on the task at hand. So far, there hasn't been much better approaches proposed. Why is a "better" approach needed? If modern LLMs can properly figure it out? It's not like LLMs don't k…

> This is solved trivially by having default initial prompts. All majors tools like Claude Code or Gemini CLI have ways to set them up.

That only makes it worse. The MCP tools available all add to the initial context. The more tools, the more of the context is populated by MCP tool definitions.

Post reply on HN