Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coding with LLMs in the summer of 2025 – an update

antirez.com

Post reply on HN
Print view View original

411–420 of 434 posts

Re: Coding with LLMs in the summer of 2025 – an update

#411

Post by stillsut »

Overall strong piece of writing. This part resonated with me as aptly described:

> more/better in the same time used in the past — which is what I do), when left alone with nontrivial goals they tend to produce fragile code bases that are larger than needed, complex, full of local minima choices, suboptimal in many ways.

And this part felt like a "bitter lesson" anti-pattern:

> Avoid any RAG that will show only part of the code / context to the LLM. This destroys LLMs performance. You must be in control of what the LLM can see when providing a reply.

Ultimately I think cli agents like claude-code and gemini-cli and aider will be controlling the context dynamically, and the human should not be spending premature optimization time on this activity.

If anyone's interested I've got some very exact stats on prompts and accepted solution linked in my LLM proof of concept repo: https://github.com/sutt/agro/blob/master/docs/dev-summary-v1...

Re: Coding with LLMs in the summer of 2025 – an update

#412

Post by schneehertz »

thefourthchime wrote: post #259
wg0 wrote: post #258

I don't understand. Is author suggesting manually pasting redis C files into Gemini Pro chat window on the web?

I was mostly nodding my head until he got to this part. The fundamental requirement for the LLM to be used is: don’t use agents or things like editor with integrated coding agents. So right, is he like actually copying and pasting stuff into a chat window? I did this before Co-Pilot, but with cursor I would never think of doing that. He never mentioned Cursor or Claude Code so I wonder if he's even experienced it.

The edit mode of GitHub Copilot requires manually providing context files and does not have RAG or other agent tools. I think this mode is much easier to use than the agent mode.

Re: Coding with LLMs in the summer of 2025 – an update

#413

Post by schneehertz »

nlh wrote: post #33

Can anyone recommend a workflow / tools that accomplishes a slightly more augmented version of antirez’ workflow & suggestions minus the copy-pasting? I am on board to agree that pure LLM + pure original full code as context is the best path at the moment, but I’d love to be able to use some shortcuts like quickly applying changes, checkpoints, etc. My persistent (and not unfounded?) worry is that all the major tools…

Github Copilot's Edit mode allows you to manually specify the context, and it runs only once each time to write code with diff checking, without entering a agent loop.

Re: Coding with LLMs in the summer of 2025 – an update

#414

Post by theshrike79 »

simonw wrote: post #141
dakiol wrote: post #25

> Gemini 2.5 PRO | Claude Opus 4 Whether it's vibe coding, agentic coding, or copy pasting from the web interface to your editor, it's still sad to see the normalization of private (i.e., paid) LLM models. I like the progress that LLMs introduce and I see them as a powerful tool, but I cannot understand how programmers (whether complete nobodies or popular figures) dont mind adding a strong dependency on a third part…

The models I can run locally aren't as good yet, and are way more expensive to operate. Once it becomes economical to run a Claude 4 class model locally you'll see a lot more people doing that. The closest you can get right now might be Kimi K2 on a pair of 512GB Mac Studios, at a cost of about $20,000.

This is the thing. I'm waiting for the equivalent of Google Coral[0], but powerful enough for AI workloads.

You can plug in the $60 Coral to a Raspberry Pi and get real-time image recognition running in Frigate.

When I can have:

1) Something similar inside my computer/laptop

2) Something I can plug in to my computer via USB-C

3) Something I can buy and install to my LAN so all devices in my home can connect to it

I'll buy it instantly.

What I don't want is a massive generic GPU that just happens to be good at AI workloads, I want custom hardware that's more efficient and cheaper.

(Off topic, but my guess is that Apple is aiming for #3 with an Apple TV variant so you can have more power than your phone, but still keep it 100% local)

[0] https://coral.ai/products/accelerator/

Re: Coding with LLMs in the summer of 2025 – an update

#415

Post by theshrike79 »

jmb99 wrote: post #313
simonw wrote: post #141

Earlier quoted context omitted.

The models I can run locally aren't as good yet, and are way more expensive to operate. Once it becomes economical to run a Claude 4 class model locally you'll see a lot more people doing that. The closest you can get right now might be Kimi K2 on a pair of 512GB Mac Studios, at a cost of about $20,000.

What’s your budget and speed requirement? A quad-CPU Xeon E7 v4 server (Supermicro X10QBI, for example) with 1TB of RAM gives you ~340GB/s memory bandwidth and enough actual memory to host a full DeepSeek instance, but it will be relatively slow (a few tokens/s max in my experience). Up front cost a bit under $1k, less if you can source cheap 32GB DDR3 RAM. Power consumption is relatively high, ~1kW under load. But I…

I think we're at early 2000's bitcoin markets here.

People were buying stores empty of GPUs to mine for BTC.

Then people built custom ASICs that couldn't do anything but mine BTC, but did it a lot cheaper and with a lot less electricity required -> nobody GPU mines anymore pretty much.

I'm waiting for a similar thing to happen to local AI.

Re: Coding with LLMs in the summer of 2025 – an update

#416

Post by theshrike79 »

amelius wrote: post #399
loudmax wrote: post #388

Earlier quoted context omitted.

It probably depends what your objective is. One of the benefits you get from running less capable models is that it's easier to understand what their limitations are. The shortcomings of more powerful models are harder to see and understand, because the models themselves are so much more capable. If you have no interest in the inner workings of LLMs and you just want the machine to spit out some end result while putt…

Is it really possible to learn from the mistakes of an LLM? It sounds like psychology or even alchemy 2.0, to be honest.

You can kinda get a feel for what they're good at, if you get what I mean?

Even the big online models have very specific styles and preferences for similar tasks. You can easily test this by giving them all some generic task without too many limits, each of them will gravitate towards a different solution to the same problem.

Re: Coding with LLMs in the summer of 2025 – an update

#417

Post by manmal »

InterviewFrog wrote: post #349
Abishek_Muthian wrote: post #344

Earlier quoted context omitted.

What type of code you write for which the opensource models aren't good enough? I use Qwen2.5 coder for auto complete and occasional chat. I don't want AI to edit my code and so this works well for me. I agree that the hardware investment for local AI is steep but IMO the local models are good enough for most experienced coders who just want a better autocomplete than the one provided by the IDE by default.

Using AI for autocomplete is like using a racecar to pick up groceries. This is exactly what the author says about avoiding LLMs for some ideological or psychological refusal.

Nothing's wrong with using autocomplete in addition to agents.

Re: Coding with LLMs in the summer of 2025 – an update

#418

Post by otabdeveloper4 »

simonw wrote: post #364
otabdeveloper4 wrote: post #326

Earlier quoted context omitted.

a) Rent a GPU server. b) Learn to finetune your models. You're a programmer, right? Whatever happened to knowing your tools? OP is right, these people are posers and fakers, not programmers.

Have you had any success finetuning models? What did you do?

Not yet. That day will come though.

Re: Coding with LLMs in the summer of 2025 – an update

#419

Post by cratermoon »

keeda wrote: post #283
cratermoon wrote: post #67

Earlier quoted context omitted.

They should also share their prompts and discuss exactly how much effort went into checking the output and re-prompting to get the desired result. The post hints at how much work it takes for the human, "If you are able to describe problems in a clear way and, if you are able to accept the back and forth needed in order to work with LLMs ... you need to provide extensive information to the LLM: papers, big parts of t…

> After all the effort getting to the point where the generated code is acceptable, one has to wonder, why not just write it yourself? Because it is still way, way, way faster and easier. You're absolutely right that the hard part is figuring out the solution. But the time spent typing is in no way trivial or cognitively simple, especially for more complex tasks. A single prompt can easily generate 5 - 10x the amount…

> A single prompt can easily generate 5 - 10x the amount of code in a few seconds

That doesn't sound like a great outcome. Every line of code is a potential bug and a liability. The least important skill in programming is typing speed.

Re: Coding with LLMs in the summer of 2025 – an update

#420

Post by airspresso »

oblio wrote: post #250
airspresso wrote: post #231

Earlier quoted context omitted.

Many reasons, one being that LLMs are essentially compressing the training data to unbelievably small data volumes (the weights). When doing so, they can only afford to keep the general principles and semantic meaning of the training data. Bigger models can memorize more than smaller ones of course, but are still heavily storage limited. Through this process they become really good at semantic understanding of code a…

Yeah, I just asked Gemini and apparently some older estimates put a relatively filtered dataset of Github source code at around 21TB in 2018, and some more recent estimates could put it in the low hundreds of TB. Considering as you said, that LLMs are doing a form of compression, and assuming generously that you add extra compression on top, yeah, now I understand a bit more. Even if you focus on non-similar code to…

Yes, for inference the main bottleneck is GPU VRAM and the bandwidth between the GPU cores and VRAM. Ideally you want enough GPU VRAM to be able to load the entire model into VRAM + have room for caching the already-produced output in VRAM when you're generating output tokens. And fast enough VRAM bandwidth that you can copy the weights from VRAM to GPU compute cores as fast as possible to do the calculations for each token. This determines the tokens/sec speed you get for the output. So yes, more and faster VRAM is essential.
Post reply on HN