Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

271–280 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#271
post #67
post #60

Earlier quoted context omitted.

Go look at the latest YC batch. 30% of them were built with 90% of the code being written by LLMs.

Where did you see that stat? And why just 30%, shouldn't almost all of them be built by LLM if LLM is that much better? Edit: This says a quarter, where did you get the 30% figure from? https://techcrunch.com/2025/03/06/a-quarter-of-startups-in-y...

wow 25% not 30% ya sure got me

> shouldn't almost all of them be built by LLM if LLM is that much better

a) not every yc company is a software company

b) these things take time, and not everyone knows how to use these tools

this stuff should be pretty self evident.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#272
I speculate, from this, that you are giving the LLMs too big of a task. Not quite vibe coding, but along that path.

> "You've refactored most of the file but forgot a single function"). It would take many many iterations on trivial issues, and because these iterations are slow that just meant I had to context switch a lot, which is also exhausting.

Try prompts like this:

"Given these data structures: (Code of structs and enums), please implement X algorithm in this function signature. (Post exact function signature)."

Or: "This code is repetitive. Please write a macro to simplify the syntax. Here's what calling it should look like (Show macro use syntax)"

Or: "I get X error on this function call. Please correct it."

Or: "I'm copying these tables into native data structures/match arms etc. Here's the full table. Here's the first few lines of the native structures: ...""

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#273
post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

I find LLMs 100x more productive for greenfield work. If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results. But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. T…

I have not seen much discussion on how to properly work with legacy code using LLMs.

Michael Feathers book comes to mind when thinking about the topic. One gets the idea that you have to write a lot of tests. But what happens when there are no tests, comments, documents etc?

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#274
post #246

Earlier quoted context omitted.

I couldn’t agree more and I’ve said the same thing many times. I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. It gets stuck in loops (removing and adding the same code/concept), it gets hung up on simple errors, etc. For greenfield it’s amazing, no doubt, but unless you are watching it very closely and approving/reviewing the code along the way it will go of…

> For greenfield it’s amazing I'll preface this comment with: I am a recent startup owner (so only dev, which is important) and my entire codebase has been generated via Sonnet (mostly 3.7, now using 4.0). If you actually looked at the work I'm (personally) producing, I guess I'm more of a product-owner/project-manager as I'm really just overseeing the development. > I have yet to see an LLM-generated app not collaps…

Do you have an example of a rule file? Or the MCPs you use?

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#275
post #274
post #246

Earlier quoted context omitted.

> For greenfield it’s amazing I'll preface this comment with: I am a recent startup owner (so only dev, which is important) and my entire codebase has been generated via Sonnet (mostly 3.7, now using 4.0). If you actually looked at the work I'm (personally) producing, I guess I'm more of a product-owner/project-manager as I'm really just overseeing the development. > I have yet to see an LLM-generated app not collaps…

Do you have an example of a rule file? Or the MCPs you use?

MCPs:

  - `server-sequential-thinking` (MVP)
  - `memory` (2nd MVP, needs custom rules for config)
  - `context7`
  - `filesystem`
  - `fetch`
  - `postgres`
  - `git`
  - `time`

Example rules file for ticketing system:

```

# Ticket Management Guidelines

This document outlines the standardized approach to ticket management in the project. All team members should follow these guidelines when creating, updating, or completing tickets.

## Ticket Organization

Tickets are organized by status and area in the following structure:

TICKETS/ COMPLETED/ - Finished tickets BACKEND/ - Backend-related tickets FRONTEND/ - Frontend-related tickets IN_PROGRESS/ - Tickets currently being worked on BACKEND/ FRONTEND/ BACKLOG/ - Tickets planned but not yet started BACKEND/ FRONTEND/

## Ticket Status Indicators

All tickets must use consistent status indicators:

- *BACKLOG* - Planned but not yet started - *IN_PROGRESS* - Currently being implemented - *COMPLETED* - Implementation is finished - *ABANDONED* - Work was stopped and will not continue

## Required Ticket Files

Each ticket directory must contain these files:

1. *Main Ticket File* (TICKET_.md): - Problem statement and background - Detailed analysis - Implementation plan - Acceptance criteria

2. *Implementation Plan* (IMPLEMENTATION_PLAN.md): - Detailed breakdown of tasks - Timeline estimates - Success metrics

3. *Implementation Progress* (IMPLEMENTATION_PROGRESS.md): - Status updates - Issues encountered - Decisions made

4. *Design Documentation* (DESIGN_RECOMMENDATIONS.md), when relevant: - Architecture recommendations - Code patterns and examples - Error handling strategies

5. *API Documentation* (API_DOCUMENTATION.md), when applicable: - Interface definitions - Usage examples - Configuration options

## Ticket Workflow Rules

### Creating Tickets

1. Create tickets in the appropriate BACKLOG directory 2. Use standard templates from .templates/ticket_template.md 3. Set status to *Status: BACKLOG* 4. Update the TICKET_INDEX.md file

### Updating Tickets

1. Move tickets to the appropriate status directory when status changes 2. Update the status indicator in the main ticket file 3. Update the "Last Updated" date when making significant changes 4. Document progress in IMPLEMENTATION_PROGRESS.md 5. Check off completed tasks in IMPLEMENTATION_PLAN.md

### Completing Tickets

1. Ensure all acceptance criteria are met 2. Move the ticket to the COMPLETED directory 3. Set status to *Status: COMPLETED* 4. Update the TICKET_INDEX.md file 5. Create a completion summary in the main ticket file

### Abandoning Tickets

1. Document reasons for abandonment 2. Move to COMPLETED/ABANDONED directory 3. Set status to *Status: ABANDONED* 4. Update the TICKET_INDEX.md file

## Ticket Linking

When referencing other tickets, use relative links with appropriate paths:

markdown @TICKET_NAME

Ensure all links are updated when tickets change status.

## Ticket Cleanup and Streamlining

### When to Streamline Tickets

Tickets should be streamlined and cleaned up at major transition points to maintain focus on remaining work:

1. *Major Phase Transitions* - When moving between phases (e.g., from implementation to testing) 2. *Milestone Achievements* - After completing significant portions of work (e.g., 80%+ complete) 3. *Infrastructure Readiness* - When moving from setup/building to operational phases 4. *Team Handoffs* - When different team members will be taking over the work

### What to Streamline

*Replace Historical Implementation Details With:* - Brief completed tasks checklist ( high-level achievements) - Current status summary - Forward-focused remaining work

*Remove or Simplify:* - Detailed session-by-session progress logs - Extensive implementation decision histories - Verbose research findings documentation - Historical status updates and coordination notes

### Why Streamline Tickets

1. *Git History Preservation* - All detailed progress, decisions, and implementation details are preserved in git commits 2. *Clarity for Future Work* - Makes it easier to quickly understand "what needs to be done next" 3. *Team Efficiency* - Anyone picking up the work can immediately see current state and next steps 4. *Maintainability* - Shorter, focused tickets are easier to read, understand, and keep updated

### How to Streamline

1. *Archive Detailed Progress* - Historical implementation details are preserved in git history 2. *Create Completion Summary* - Replace detailed progress with a brief "What's Complete" checklist 3. *Focus on Remaining Work* - Make current and future phases the primary content 4. *Update Status Sections* - Keep status concise and action-oriented 5. *Preserve Essential Context* - Keep architectural decisions, constraints, and key requirements

*Goal*: Transform tickets from "implementation logs" into "actionable work plans" while preserving essential context.

## Maintenance Requirements

1. Keep the TICKET_INDEX.md file up to date 2. Update "Last Updated" dates when making significant changes 3. Ensure all ticket files follow the standardized format 4. Include links between related tickets in both directions

## Complete Documentation

For detailed instructions on working with tickets, refer to:

- @Ticket Workflow Guide - @Ticket Index - @Tickets README

```

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#276
I've been programming in python for over 20 years. An LLM creates code that sometimes works, but it definitely doesn't meet my standards, and there's no way I'd use the code since I couldn't support it. People who have less experience in Python might take that working code and just support that with their LLM, still having no clue what it does or why it works. That's probably fine for MVP but it won't stand up in the real world where you have to support the code or refactor it for your environment.

I tried to use an LLM to write a simple curses app - something where there's a lot of code out there, but most of the code is bad, and of course it doesn't work and there's lots of quirks. I then asked it to see if there are libraries out there that are better than curses, it gave me 'textual' which at first seemed like an HTML library, but is actually a replacement for curses. It did work, and I had some working code at the end, but I had to work around platform inconsistencies and deal with the LLM including outdated info like inline styles that are unsupported in the current version of the library. That said, I don't quite understand the code that it produced, I know it works and it looks nice, but I need to write the code myself if I want a deeper understanding of the library, so that I can support it. You won't get that from asking an LLM to write your code for you, but from you using what you learn. It's like any language learning. You could use google translate to translate what you want, and it may seem correct at first glance, but ultimately won't convey what you want, with all the nuance you want, if you just learned the language yourself.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#277

Earlier quoted context omitted.

I couldn’t agree more and I’ve said the same thing many times. I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. It gets stuck in loops (removing and adding the same code/concept), it gets hung up on simple errors, etc. For greenfield it’s amazing, no doubt, but unless you are watching it very closely and approving/reviewing the code along the way it will go of…

>I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable. It will not. And I say this as someone whose been building internal LLM tools since 2021. The issue is their context window. If you increase the context window so they can see more code costs skyrocket as n^2 the size of the code base. If you don…

I can't help but wonder whether the solution here is something like building a multi-resolution understanding of the codebase. All the way from an architectural perspective including business context, down to code structure & layout, all the way down to what's happening in specific files and functions.

As a human, I don't need to remember the content of every file I work on to be effective, but I do need to understand how to navigate my way around, and enough of how the codebase hangs together to be able to make good decisions about where new code belongs, when and how to refactor etc.. I'm pretty sure I don't have the memory or reading comprehension to match a computer, but I do have the ability to form context maps at different scales and switch 'resolution' depending on what I'm hoping to achieve.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#278

Earlier quoted context omitted.

I completely agree with you that the coders who are "smitten" by LLMs are just inexperienced. I personally find that LLMs get subtle improvements in capability over time and it's usually worthwhile to check in on the progress from time to time. Even if it's just for fun.

I can see why you would chime in to say that in your experience you don't get any value out of it, but to chime in to say that the millions of people who do are "inexperienced" is pretty offensive. In the hands of skilled developers these tools are a complete gamechanger.

Where are these millions, and where is their output? You're in an echo chamber mate, there aren't millions of people using AI to do significant amounts of work.

Indie hackers just did an article on 4 vibe coded Startups and they all seem like a joke.

And they could only find 4!

I didn't look at them all, but the flight sim is spectacularly bad, the revenue numbers obviously unsustainable and it looks like something moderately motivated school children might have made for a school project in a week.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#279
post #78

Earlier quoted context omitted.

I find LLMs 100x more productive for greenfield work. If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results. But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. T…

Not really my case. Found that Codebase with good code benefit more with LLM, but it's not the prerequisite. I just rewrote 300ish advanced PostgreSQL queries to mysql queries. The process is not magical, but it will take me 1 week rather than 3 days. Now I'm on testing phase, seems promising. The point is if we can find a work to work along with the agent, can be very productive.

Why did you not use an SQL transpiler?

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#280
If you ever did a declarative programing, then you know what's LLM brought in it. If before you should say in a very formal way what you want to do, now you can do the same writing your prompt if a very relaxed form. You declarations are fed from thousand implementation sources now, than just few ones before. Otherwise I do not see much difference. I can also warn also that using a very formal language is much more concise, and you get a result in much shorter time. However, most people try to avoid any learning.
Post reply on HN