bullshit and useless. Not even a proper comparison
When AI Costs More Than the Engineer
91–100 of 128 posts
Re: When AI Costs More Than the Engineer
#92Working regularly with AI is like managing a small team of unbelievably knowledgeable, very smart, and occasionally crashingly naïve junior developers. Because they're so knowledgeable and smart, they can get a lot done very quickly. Because they make a proportion of howling errors, you have to keep a close eye on them -- or carefully train another agent to do it for you, in which case you now have to keep a close ey…
I get the feeling that either I'm using LLMs wrong, or everyone else is. Outside of enthusiastic use of Tab and some one-off scripts, I don't really tell it to write code. Instead I ask vague questions about the codebase and its inner workings. Reading other people's code has always been my Achilles' heel - particularly if it's a huge project and has a lot of undocumented conventions. LLMs are brilliant at explaining…
Shit, I know people that don't code that are asking LLMs for advice about personal matters. As far as AI companies are concerned, as long as you're using their product and are paying them for it, who cares what you're actually asking it for.
Re: When AI Costs More Than the Engineer
#93Earlier quoted context omitted.
AI training uses wildly more compute than most companies, who are generally building domain specific CRUD apps. Compare AI costs per-engineer-salary-dollar, because more expensive engineers probably need more expensive AI.
> Compare AI costs per-engineer-salary-dollar, because more expensive engineers probably need more expensive AI. Let's see how this works out in the long run. For a historical analog, more expensive engineers don't use more expensive computers (by and large).
> more expensive engineers don't use more expensive computers
They don't? If you give your best engineers substandard hardware to work on, you're going to get worse output from them compared to if you give them more expensive computers to work with.
Re: When AI Costs More Than the Engineer
#94Re: When AI Costs More Than the Engineer
#95I've not seen anyone yet implement a true cost to productivity assessment or guardrails for AI usage yet. Sure this is hard to do with people, but performance management is a well understood field with a hundred years of practice for knowledge workers. We don't get unlimited hiring budget, so we also won't get unlimited token budgets, and we as the operators will be responsible for the productivity of our agents. Wha…
[flagged]
Re: When AI Costs More Than the Engineer
#96Earlier quoted context omitted.
I wonder if they ever will be. If the chinese open source models are only 3-6 months behind every major frontier model release, I can't see the business model. GLM-5.2 is supposedly on par to Opus depending on the case. And everybody and their mother can run that model in their datacenter and charge Dollars for tokens.
There is distillation going on where chinese providers give the model lots of outputs. We don't live in a world where chinese providers are not doing this so we can't compare the advantage of this distillation, but there is some advantage to it otherwise they wouldn't do it. If Anthropic can block distillations somehow (which are fair game imo given that Anthropic et al did the same with the written works of mankind)…
I think it's quite hard to block distillations, and everyone is distilling, even unintentionally.
If you feed the most recent Github repos into the training, most of that code will be written by frontier LLMs. Training on that is distillation.
Re: When AI Costs More Than the Engineer
#97By the way, isn't this normal outside the software industry? A carmaker spends more money on buying parts, a bakery on ovens and ingredients than staff etc.
It's just that for software cost is mostly in human labor. Just like a carmaker can argue whether it makes sense for a manufacturing step should be done by robots or humans, one day this question will come up in software as well.
Re: When AI Costs More Than the Engineer
#98Earlier quoted context omitted.
Exactly, it's like saying Shell is spending a fortune on fuel compared to what they spend on employees, if you count oil extraction costs as 'fuel'.
So where are these training costs getting paid from?
Compute (and IP theft) is the raw materials being used by the AI companies to build their products (the models).
It doesn't make sense to compare OpenAIs or Anthropics compute spend to that of our average software company, because different products require different raw materials. Dropbox also use way more storage than Snapchat, that's an equally silly comparison.
Re: When AI Costs More Than the Engineer
#99Re: When AI Costs More Than the Engineer
#100Garbage. You can't include training by the companies that develop an llm in the comparison against companies that merely use the same llm. Apples and potatoes.
Apples and oranges, or chalk and cheese. Why would you say apples and potatoes?
I came to appreciate typos, slightly ungrammatical sentences, and creative plays on common phrases. They are completely absent in LLM output without a specialized prompt (and they seem to struggle to make believable language mistakes even when prompted), so they can serve as a signal when judging whether the text I'm reading was written by a human.
As an aside, I observed my stance on proper English use changing in real time over the past 2-3 years. I used to be proud of writing clinically correct prose, and found mistakes in grammar and vocabulary grating. Now, I kind of welcome them and have stopped caring at all about committing such language crimes. I used to cringe when someone didn't capitalize the first words in their sentences - not anymore. I think we're years away from LLMs convincingly faking human-like mistakes (since all the work currently goes into avoiding them), so it's going to remain a useful signal for a while.