21–27 of 27 posts
More tasks get "done" while rework is sky high and overall throughput to production drops. First, I'd like to thank all the people working on testing and doing the lord's work. Anyway, this isn't even a unique pattern to LLM use. We've all seen this exact same thing when more devs are added a project running late, teams are siloed, outsourcing to contractors, etc.
Earlier quoted context omitted.
Definitely not unique. Thats said, given how "easy" it is to scale LLM output, compared to human output, this pattern could be messier in the LLM era? The whole tokenmaxxing motion assumed more output equals more outcomes. What do you think?
Already covered in the post. > So this is inherent to the technology. No amount of tokenmaxxing is going to change it. LLM development even breaks common and well accepted quality norms in software development - like backwards compatibility. You literally can’t (and wouldn’t want!) an LLM to do the same thing in the same way twice. But this means, LLMs - on their own - are not a solid foundation to build a revolution…
The bit about backwards compatibility doesn't make sense either. LLMs, just like humans, can take backwards compatibility into consideration — or not. Sometimes they'll forget, and sometimes they'll fixate on it even in a system that hasn't been deployed yet. It has nothing to do with LLMs never doing "the same thing the same way twice".
LLMs may or may not become as reliable as other productivity-improving infrastructure, but it's not the nature of their technology that will decide this.
Earlier quoted context omitted.
Fair. I suppose, the question behind my question is -- what else do we need. Thanks.
Yeah I have a different take. We will end up scaling human output by hiring more devs as wages for entry to mid level continue to stagnate. I think LLMs make finding information and learning way more accessible. Even with realistic expectations it already is a revolution for literacy, education, and search. LLMs are a massive achievement that would be celebrated appropriately if only the public wasn't introduced to t…
Earlier quoted context omitted.
Yeah I have a different take. We will end up scaling human output by hiring more devs as wages for entry to mid level continue to stagnate. I think LLMs make finding information and learning way more accessible. Even with realistic expectations it already is a revolution for literacy, education, and search. LLMs are a massive achievement that would be celebrated appropriately if only the public wasn't introduced to t…
Looking forward to that hiring. So far, just layoffs. That said, there reports that consulting firms are hiring to fuel the demand around AI adoption itself. LLMs are a breakthrough like machines were. We will create new business models around them, and perhaps drives more humans. Wrt software, LLMs can make learning very easy. But with all the AI generated code, who is the "architect"?
New jobs are new jobs. The catch is these new jobs that lean heavily on AI will be a race to the bottom regarding pay. There are so many people with their own special interests that thus far never had a chance to contribute without coding skills. I don't mean that in a sinister way, but it makes total sense that we'd continue to see the rest of society find their way into the decision making. Right now we have a ton of lower priority stuff that never gets implemented because it's not part of the MVP. We also have codebases not as modular and robust as they should be behaving as platforms.
It also makes total sense this is a nightmare for those people who were most loudly proclaiming we'd replace white collar jobs. They showed their cards too early and will have the mob coming after them.
Code is meant to be shared and collaborated on. That was always where the value came from. There's a lot of untapped potential when truly anyone can write their plugins and build communities. This is really is like the early internet era repeating itself, but reaching way deeper this time.
Earlier quoted context omitted.
Already covered in the post. > So this is inherent to the technology. No amount of tokenmaxxing is going to change it. LLM development even breaks common and well accepted quality norms in software development - like backwards compatibility. You literally can’t (and wouldn’t want!) an LLM to do the same thing in the same way twice. But this means, LLMs - on their own - are not a solid foundation to build a revolution…
I thought this was the weakest argument in the article. Yes, non-determinism is inherent to LLMs. It's also inherent to human brains, yet somehow we still manage to call (some) people reliable. The bit about backwards compatibility doesn't make sense either. LLMs, just like humans, can take backwards compatibility into consideration — or not. Sometimes they'll forget, and sometimes they'll fixate on it even in a syst…
Earlier quoted context omitted.
This is discussed in the article, and I think the author makes pretty reasonable arguments for why by nature we will not see the reliability of LLM usage improve. They also discuss what I agree as the more effective method of using an LLM is, as a feedback and refinement tool, not a decision maker.
> This is not a limitation that can be overcome by LLMs. Their generative value is in their unreliability. If you turn temperature down to zero, you get a deterministic machine - but you also break every meaningful application I know of in production. This is not a reasonable argument. Setting the temperature down to zero does NOT give you a deterministic machine. And I have never seen that break any application in p…
More tasks get "done" while rework is sky high and overall throughput to production drops. First, I'd like to thank all the people working on testing and doing the lord's work. Anyway, this isn't even a unique pattern to LLM use. We've all seen this exact same thing when more devs are added a project running late, teams are siloed, outsourcing to contractors, etc.
I think the real bitter lesson is nine women will never make a baby in a month.
The idea that speed is the only thing that matters is a modern view that you can surrender any time you want.
A child is a joy. Nothing is wrong with the child for needing 9 months to grow.