281–290 of 303 posts
Earlier quoted context omitted.
How much would it have cost a human to do the same work? The question isn’t how much tokens cost; the question is how much money is saved by using AI to do it.
Does the person prompting the AI work for free?
This isn't surprising. What is not mentioned is that Claude Code also found one thousand false positive bugs, which developers spent three months to rule out.
Every time I read these titles, I wonder if people are for some reason pushing the narrative that Claude is way smarter than it really is, or if I'm using it wrong. They want me to code AI-first, and the amount of hallucinations and weird bugs and inconsistencies that Claude produces is massive. Lots of code that it pushes would NOT have passed a human/human code review 6 months ago.
What models are you using, on what type of codebases, with what tools?
I understand how people use those tools if all they do is build CRUD endpoints and UIs for those endpoints (which is admittedly what most programmers probably do for their job). But for anything that requires any sort of problem solving skills, I don't understand how people use them. I feel like I live in a completely different world from some of the people who push agentic coding.
Not "hidden", but probably more like "no one bothered to look". declares a 1024-byte owner ID, which is an unusually long but legal value for the owner ID. When I'm designing protocols or writing code with variable-length elements, "what is the valid range of lengths?" is always at the front of my mind. it uses a memory buffer that’s only 112 bytes. The denial message includes the owner ID, which can be up to 1024 by…
"No one bothered to look" is how most vulnerabilities work. Systems development produces code artifacts with compounding complexity; it is extraordinarily difficult to keep up with it manually, as you know. A solution to that problem is big news. Static analyzers will find all possible copies of unbounded data into smaller buffers (especially when the size of the target buffer is easily deduced). It will then report…
Pasting a big batch of new code and asking Claude "what have I forgotten? Where are the bugs?" is a very persuasive on-ramp for developers new to AI. It spots threading & distributed system bugs that would have taken hours to uncover before, and where there isn't any other easy tooling. I bet there's loads of cryptocurrency implementations being pored over right now - actual money on the table.
ive had some remarkable successes with claude and quite a few "well that was a total waste of time" efforts with claude. for the most part i think trying to do uncharted/ambitious work with claude is a huge coinflip. he's great for guardrailed and well understood outcomes though, but im a little burnt out and unexcited at hearing about the gigantic-claude exercises.
Earlier quoted context omitted.
"No one bothered to look" is how most vulnerabilities work. Systems development produces code artifacts with compounding complexity; it is extraordinarily difficult to keep up with it manually, as you know. A solution to that problem is big news. Static analyzers will find all possible copies of unbounded data into smaller buffers (especially when the size of the target buffer is easily deduced). It will then report…
But what was the likelihood of this bug to be exploited by malicious actors?
Earlier quoted context omitted.
Definitely not my wheelhouse, but I would expect it to be considerably worse. Simply because the source code contains names that were intended to communicate meaning in a way that the LLM is specifically trained to understand (i.e., by choosing identifier names from human natural language, choosing those names to scan well when interspersed into the programming language grammar, including comments etc.). At least if…
I've cut-and-pasted some assembly code into the free version of ChatGPT to reverse engineer some old binaries and its ability to find meaning was just scary.
Earlier quoted context omitted.
> they are and they will be Calculate the approximate cost of raising a human from birth to having the knowledge and skills to do X, along with maintenance required to continue doing X. Multiply by a reasonable scaling factor in comparison to one of today's best LLMs (ie how many humans and how much time to do Xn, vs the LLM). Calculate the cost of hardware (from raw elements), training and maintenance for said LLM (…
But companies don't have to bear the cost of raising a human from birth, or training them. They only pay the cost of hiring them, and that includes cost of maintenence. Add to that the fact that we can't blindly trust LLM output just yet, so we need a mearbag to review it. LLM will always be more expensive than human +LLM, until we're at a stage where we can remove the human from the loop
The costs do exist somewhere though, and must be paid by someone. There's no free lunch, and the human lunch is very likely far more costly than the LLM lunch.
> Add to that the fact that we can't blindly trust LLM output just yet
Can't blindly trust human output either. That's why there are various tiers in roles, from junior-equivalent to senior-equivalent, and the actual user of the product is always the final arbiter. There's ultimately nothing different, except that the LLM iterates on issue resolution in seconds to minutes, whereas the human equivalent takes hours to days.
Supposedly humans have become “100x”™ more productive with these AI tools, but nowhere to be seen are the benefits for the wielders of said tools. Is your salary 100x higher? Are you able to spend more time with your family/friends instead of at the office? Why are we still putting up with these outdated work practices if LLMs have made everybody so much more productive?
Are you aware of how productivity has increased over the past century in general? That didn't lead to 100x wage increases or more free time. Labour is a market commodity and follows market rules. Increased productivity means more gets done in less time. It doesn't mean you spend less time working