Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current state of LLM-driven development

blog.tolki.dev

Post reply on HN
Print view View original

61–70 of 244 posts

Re: The current state of LLM-driven development

#61

Post by SkyPuncher »

uvdn7 wrote: post #55
SkyPuncher wrote: post #44

Earlier quoted context omitted.

> Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. That's a wild statement. I'm now extremely productive with LLMs in my core codebases, but it took a lot of practice to get it right and repeatable. There's a lot of little contextual details you need to learn how to control so the LLM makes the right choice…

Is the non-trivial amount of time significantly less than you trying to ramp up yourself? I am still hesitant using AI for solving problems for me. Either it hallucinates and misleads me. Or it does a great job and I worry that my ability of reasoning through complex problems with rigor will degenerate. When my ability of solving complex problems degenerated, patience diminished, attention span destroyed, I will beco…

The ramp-up time with AI is absolutely lower than trying to ramp up without AI.

My comment is specifically in contrast to working in a codebase where I'm at "max AI productivity". In a new codebase, it just takes a bit of time to work out kinks and figure out tendencies of the LLMs in those codebases. It's not that I'm slower than I'd be without AI, I'm just not at my "usual" AI-driven productivity levels.

Re: The current state of LLM-driven development

#63

Post by antonvs »

SadErn wrote: post #29
rustybolt wrote: post #19

Earlier quoted context omitted.

> "There's no learning curve" just means this guy didn't get very far up Not everyone with a different opinion is dumber than you.

This is all just ignorance. We've all worked with LLMs and know that creating an effective workflow is not trivial and it varies based on the tool.

Define "not trivial". Obviously, experience helps, as with any tool. But it's hardly rocket science.

It seems to me the biggest barrier is that the person driving the tool needs to be experienced enough to recognize and assist when it runs into issues. But that's little different from any sophisticated tool.

It seems to me a lot of the criticism comes from placing completely unrealistic expectations on an LLM. "It's not perfect, therefore it sucks."

Re: The current state of LLM-driven development

#64

Post by rocqua »

tptacek wrote: post #8

Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. You can safely ignore them if they don’t fit your workflows at the moment. I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. It's a really weird way to open up an article concluding that LLMs make one a worse prog…

The OPs point seems to be: it's very quick for LLMs to be a net benefit to your skills, if it is a benefit at all. That is, he's only speaking of the very beginning part of the learning curve.

Re: The current state of LLM-driven development

#65

Post by troupo »

simonw wrote: post #48
troupo wrote: post #23

Earlier quoted context omitted.

> I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. Because for all our posturing about being skeptical and data driven we all believe in magic. Those "counterintuitive non-trivial workflows"? They work about as well as just prompting "implement X" with no rules, agents.md, careful lists etc. Becau…

The problem with your argument here is that you're effectively saying that developers (like myself) who put effort into figuring out good workflows for coding with LLMs are deceiving themselves, and are effectively wasting their time. Either I've wasted significant chunks of the past ~3 years of my life or you're missing something here. Up to you to decide which you believe. I agree that it's hard to take solid measu…

> who put effort into figuring out good workflows for coding with LLMs are deceiving themselves, and are effectively wasting their time.

It's quite possible you do. Do you have any hard data justifying the claims of "this works better", or is it just a soft fuzzy feeling?

> The same goes for managing people, and yet somehow many good engineering managers can judge if their team is performing well

It's actually really easy to judge if a team is performing well.

What is hard is finding what actually makes the team perform well. And that is just as much magic as "if you just write the correct prompt everything will just work"

---

wait. why are we fighting again? :) https://dmitriid.com/everything-around-llms-is-still-magical...

Re: The current state of LLM-driven development

#66

Post by troupo »

tptacek wrote: post #54
georgeburdell wrote: post #52

Earlier quoted context omitted.

Working efficiently as a team, perhaps, but during solo development this is unnecessary beyond the extent that is necessary to document the code

Even this opens up a whole field of weird subtle workflow tricks people have, because people run parallel asynchronous agents that step on each other in git. Solo developers run teams now! Really wild to hear someone say out loud "there's no learning curve to using this stuff".

The "learning curve" is reading "experts opinion" on the ever-changing set of magical rituals that may or may not work but trust us it works.

Re: The current state of LLM-driven development

#68

Post by donperignon »

LLM’s are basically glorified slot machines. Some people try very hard to come up with techniques or theories about when the slot machine is hot, it’s only an illusion, let me tell you, it’s random and arbitrary, maybe today is your lucky day maybe not. Same with AI, learning the “skill” is as difficult as learning how to google or how to check stackoverflow, trivial. All the rest is luck and how many coins do you have in your pocket.

Re: The current state of LLM-driven development

#69

Post by jamboca »

Have built many pipelines integrating LLMs to drive real $ results. I think this article boils it down too simply. But i always remember, if the LLM is the most interesting part of your work, something is severely wrong and you probably aren’t adding much value. Context management based on some aspects of your input is where LLMs get good, but you need to do lots of experimentation to tune something. Most cases i have seen are about developing one pipeline to fit 100s of extremely different cases; LLM does not solve this problem but basically serves as an approximator for you to discretize previously large problems in to some information sub space where you can treat the infinite set of inputs as something you know. LLMs are like a lasso (and a better/worse one than traditional lassos depending on use case) but once you get your catch you still need to process it, deal with it progammatically to solve some greater problem. I hate how so many LLM related articles/comments say “ai is useless throw it away dont use it” or “ai is the future if we dont do it now we’re doomed lets integrate it everywhere it can solve all our problems” like can anyone pick a happy medium? Maybe thats what being in a bubble looks like

Re: The current state of LLM-driven development

#70

Post by jstummbillig »

donperignon wrote: post #68

LLM’s are basically glorified slot machines. Some people try very hard to come up with techniques or theories about when the slot machine is hot, it’s only an illusion, let me tell you, it’s random and arbitrary, maybe today is your lucky day maybe not. Same with AI, learning the “skill” is as difficult as learning how to google or how to check stackoverflow, trivial. All the rest is luck and how many coins do you ha…

We know what random* looks like: a coin toss, the roll of a die. Token generation is neither.
Post reply on HN