Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tools: Code Is All You Need

lucumr.pocoo.org

Post reply on HN
Print view View original

111–120 of 234 posts

Re: Tools: Code Is All You Need

#111

Post by loudmax »

dingnuts wrote: post #72
mritchie712 wrote: post #42

Earlier quoted context omitted.

the snippets are examples. You can ask hundreds of variations of similar, but different, complex questions and the LLM can adjust the example for that need. I don't have a snippet for, "find all 500's for the meltano service for duckdb syntax errors", but it'd easily nail that given the existing examples.

but if I know enough about the service to write examples, most of the time I will know the command I want, which is less typing, faster, costs less, and doesn't waste a ton of electricity. In the other cases I see what the computer outputs, LEARN, and then the functionality of finding what I need just isn't useful next time. Next time I just type the command. I don't get it.

LLMs are really good at processing vague descriptions of problems and offering a solution that's reasonably close to the mark. They can be a great guide for unfamiliar tools.

For example, I have a pretty good grasp of regular expressions because I'm an old Perl programmer, but I find processing json using `jq` utterly baffling. LLMs are great at coming up with useful examples, and sometimes they'll even get it perfect the first time. I've learned more about properly using `jq` with the help of LLMs than I ever did on my own. Same goes for `ffmpeg`.

LLMs are not a substitute for learning. When used properly, they're an enhancement to learning.

Likewise, never mind the idiot CEOs of failing companies looking forward to laying off half their workforce and replacing them with AI. When properly used, AI is a tool to help people become more productive, not replace human understanding.

Re: Tools: Code Is All You Need

#112

Post by forrestthewoods »

nativeit wrote: post #108
forrestthewoods wrote: post #69

Earlier quoted context omitted.

> because it is everywhere Except for the fact that actually it is not everywhere.

I see your point, but bear with me here--it kind of is. I suppose if one wanted to be pedantically literal, then you are indeed correct. In every other meaningful consideration, the parent comment is. Maybe not Bash specifically, but #!/bin/sh is broadly available on nearly every connected device on the planet, in some capacity. From the perspective of how we could automate nearly anything, you'd be hard-pressed to f…

> you'd be hard-pressed to find something more universal than a shell script.

99.9% of my 20-year career has been spent on Windows. So bash scripts are entirely worthless and dead to me.

Re: Tools: Code Is All You Need

#113

Post by benreesman »

sebzim4500 wrote: post #101
benreesman wrote: post #47

Earlier quoted context omitted.

What we call an LLM today (by which almost everyone means an autogressive language model from the Generative Pretrained Transformer family tree, and BERTs are still doing important eork, believe that) is actually an offshoot of neural machine translation. This isn't (intentionally at least) mere HN pedantry: they really do act like translation tools in a bunch of observable ways. And while they have recently crossed…

>LLMs? Look more and more like the Metaverse every day as concerns the economics. ChatGPT has 800M+ weekly active users how is that comparable to the Metaverse in any way?

I said as concerns the economics. It's clearly more popular than the Oculus or whatever, but it's still a money bonfire and shows no signs of changing on that front.

Re: Tools: Code Is All You Need

#114

Post by JyB »

tristanz wrote: post #84

You can combine MCPs within composable LLM generated code if you put in a little work. At Continual ( https://continual.ai ), we have many workflows that require bulk actions, e.g. iterating over all issues, files, customers, etc. We inject MCP tools into a sandboxed code interpreter and have the agent generate both direct MCP tool calls and composable scripts that leverage MCP tools depending on the task complexity.…

That autocompounding aspect of constantly refining initial prompts with more and more knowledge is so interesting. Gut feeling says it’s something that will be “standardized” in some way, exactly like what MCP did.

Re: Tools: Code Is All You Need

#115

Post by vunderba »

simonw wrote: post #25

Something I've realized about LLM tool use is that it means that if you can reduce a problem to something that can be solved by an LLM in a sandbox using tools in a loop, you can brute force that problem. The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide and how to define the success criteria for the model. That still takes significant skill an…

> The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide, and how to define the success criteria for the model.

Your test case seems like a quintessential example where you're missing that last step.

Since it is unlikely that you understand the math behind fractals or x86 assembly (apologies if I'm wrong on this), your only means for verifying the accuracy of your solution is a superficial visual inspection, e.g. "Does it look like the Mandelbrot series?"

Ideally, your evaluation criteria would be expressed as a continuous function, but at the very least, it should take the form of a sufficiently diverse quantifiable set of discrete inputs and their expected outputs.

Re: Tools: Code Is All You Need

#116

Post by hollerith »

nativeit wrote: post #109
hollerith wrote: post #56

Earlier quoted context omitted.

Me, too. Also, Unix as a whole is overrated. One reason it won was an agreement mediated by a Federal judge presiding over an anti-trust trial that AT&T would not enter the computer market while IBM would not enter the telecommunications market, so Unix was distributed at zero cost rather than sold. Want to get me talking reverentially about the pioneers of our industry? Talk to me about Doug Engelbart, Xerox PARC an…

> Also, Unix as a whole is overrated. One reason it won was an agreement mediated by a Federal judge presiding over an anti-trust trial that AT&T would not enter the computer market while IBM would not enter the telecommunications market, so Unix was distributed at zero cost rather than sold. What did Unix win?

Mind share of the basic design. Unix's design decisions are important parts of MacOS and Linux.

Multics would be an example of a more innovative OS than Unix, but its influence on the OSes we use today has been a lot less.

Re: Tools: Code Is All You Need

#117

Post by JyB »

the_mitsuhiko wrote: post #36
victorbjorklund wrote: post #30

I think the GitHub CLI example isn't entirely fair to MCP. Yes, GitHub's CLI is extensively documented online, so of course LLMs will excel at generating code for well-known tools. But MCP shines in different scenarios. Consider internal company tools or niche APIs with minimal online documentation. Sure, you could dump all the documentation into context for code generation, but that often requires more context than…

> Sure, you could dump all the documentation into context for code generation, but that often requires more context than interacting with an MCP tool. MCP works exactly that way: you dump documentation into the context. That's how the LLM knows how to call your tool. Even for custom stuff I noticed that giving the LLM things to work with that it knows (eg: python, javascript, bash) beats it using MCP tool calling, an…

Ive never used that many. The LLM performances collapse/degrade significantly because of too much initial context? It seems like MCP implems updates could easily solve that. Like only injecting relevant servers for the given task based on initial user prompt.

Re: Tools: Code Is All You Need

#118

Post by pramodbiligiri »

Wouldn't the sweet spot for MCP be where the LLM is able to do most of the heavy lifting on its own (outputting some kind of structured or unstructured output), but needs a bit of external/dynamic data that it can't do without? The list of MCP servers/tools it can use should nail that external lookup in a (mostly) deterministic way.

This would work best if a human is the end consumer of this output, or will receive manual vetting eventually. I'm not sure I'd leave such a system running unsupervised in production ("the Automation at Scale" part mentioned by the OP).

Re: Tools: Code Is All You Need

#120

Post by juped »

rapind wrote: post #64
benreesman wrote: post #47

Earlier quoted context omitted.

What we call an LLM today (by which almost everyone means an autogressive language model from the Generative Pretrained Transformer family tree, and BERTs are still doing important eork, believe that) is actually an offshoot of neural machine translation. This isn't (intentionally at least) mere HN pedantry: they really do act like translation tools in a bunch of observable ways. And while they have recently crossed…

> There is a hole in the ground where something between 100 billion and a trillion dollars in the ground that so far has about 20B in revenue (not profit) going into it annually. This is a concern for me. I'm using claude-code daily and find it very useful, but I'm expecting the price to continue getting jacked up. I do want to support Anthropic, but they might eventually need to cross a price threshold where I bail.…

I'm just taking advantage and burning VCs' money on useful but not world-changing tools while I still can. We'll come out of it with consumer-level okay tools even if they don't reach the levels of Claude today, though.
Post reply on HN