I have my suspicion: they estimate how long people would have taken to implement some feature, becasue they were trained on such data. I consistently see estimates of 2 week/3 weeks or 5 days, etc. But then implementation takes a day or 2 max using agents within Claude/GPT. Unless I am missing something? Anybody else notice this?
Why does it look like LLMs consistently overestimate implementation time?
1–10 of 18 posts
Re: Why does it look like LLMs consistently overestimate implementation time?
#2This is very common as they have no conception of time. They are just using ballpark figures based on what they see in the wild. I see estimates for modules in weeks/months when it can produce it in a single afternoon of prompting.
Re: Why does it look like LLMs consistently overestimate implementation time?
#3They tend to turn a small change into the whole cleanup plan. Sometimes that is useful, but it makes estimates too large.
Re: Why does it look like LLMs consistently overestimate implementation time?
#4You ask LLMs for estimates? Interesting.
Probably big model providers should do calibratuons for that and add an estimation skill.
Re: Why does it look like LLMs consistently overestimate implementation time?
#5You ask LLMs for estimates? Interesting. Probably big model providers should do calibratuons for that and add an estimation skill.
I've found that they declare estimates unprompted.
Re: Why does it look like LLMs consistently overestimate implementation time?
#6The problem is that LLMs do not have a conceptual grounding in actual time. They estimate based on statistical correlation found in their training data which is filled with standard corporate project management timelines legacy codebases and waterfall estimates.
Re: Why does it look like LLMs consistently overestimate implementation time?
#7>day or 2 max
I've frequently seen tasks that it thinks will take weeks being done in under an hour. And it will often recommend doing X instead of Y because X requires so much extra work. Basically I just remind it that it is an LLM.
If it worries something is error prone, I ask it to write tools to verify it.
Re: Why does it look like LLMs consistently overestimate implementation time?
#8they estimate based on how software is usually built in organizations, not how fast it can be built with modern AI tools and agents.
Re: Why does it look like LLMs consistently overestimate implementation time?
#9they estimate as if human will build it