221–230 of 237 posts
Earlier quoted context omitted.
shipped = paying real users ( or a non profit with users) doesnt matter what subjective opinions are. shipped is not pushing something on github.
I have used AI to ship software that users pay for.
Earlier quoted context omitted.
How was any of that incomprehensible? They gave a list of some actual things they shipped. The hand wavy dismissal where you pretend they didn’t say anything just exposes your bias. You don’t want to hear a success story so you don’t.
how do you ship a "wysiwyg"? and my point is he is doing the hand-waving. i'm quite happy to hear success stories - we get so few concrete ones here.
Earlier quoted context omitted.
what did you ship?
Thinking about it a better question [for me] would be what might have but didn't ship because of LLMs.
Earlier quoted context omitted.
Error percentages compound. Having an LLM do something with a 5% hallucination rate, then having an LLM (same or different) with a 5% error rate check it means there's a 10% chance of error, not .25%. Prompting can't save you here; it's the fundamental math.
How errors add depend upon how you treat them. Two independent task? You have a 95% chance of each task being correct, so total success is .95^2 = 90.25% correct. Having two AIs verify the same task with each being 95% correct? That's a 5% error per each, so 1-(.05^2) = 99.75% correct. That is assuming fully independent error rates, which is the real issue. If it is pure hallucinations, then that is an okay assumptio…
Earlier quoted context omitted.
I have to wonder what sort of evidence you would accept? Because it seems to me that the original question was answered clearly enough and now the goalposts have shifted to demanding evidence that the answer wasn't falsified.
Well you answered how it is useful by saying it writes more lines of code and that seems to be the only way it's useful. But we all know that lines of code aren't productivity.
The claim was that the same size team is now accomplishing more tasks than they would have without AI.
Earlier quoted context omitted.
Checking all the past "what are you working on?" threads just to realize that, somewhere in August 2024 stuff just exploded and the average thread goes reached ~1000 messages, from ~50 pre-August.
Tons of ppl got into CS in last few years worldwide.
The obvious notable event circa 2024 was LLMs becoming reasonably useful for code assistance (IMO and IIRC OFC).
TFA says "no hallucinations" but you can't fix hallucinations with rules or other models. I know I'm screaming into the void but whatever.
Earlier quoted context omitted.
You can reduce hallucinations with appropriate grounding checks. You can’t really get rid of them though, so feedback loops with verification are essential.
Doing a loop and having the model review its own earlier output tends to solve this stuff, and for code requiring good test coverage catches them before they even hit that phase. So while you can't remove "hallucinations" they're corrected before the process is complete, albeit at the cost of more tokens for verification.
Earlier quoted context omitted.
Error percentages compound. Having an LLM do something with a 5% hallucination rate, then having an LLM (same or different) with a 5% error rate check it means there's a 10% chance of error, not .25%. Prompting can't save you here; it's the fundamental math.
How errors add depend upon how you treat them. Two independent task? You have a 95% chance of each task being correct, so total success is .95^2 = 90.25% correct. Having two AIs verify the same task with each being 95% correct? That's a 5% error per each, so 1-(.05^2) = 99.75% correct. That is assuming fully independent error rates, which is the real issue. If it is pure hallucinations, then that is an okay assumptio…
I’ve written a bit about this and my approach is based on clean room and N-versions.