Live data from Hacker News

The Anthropic Economic Index

anthropic.com

131–140 of 238 posts

Re: The Anthropic Economic Index

#131

This more or less confirms what I imagine most of us thought, AI is mostly used by engineers, or for engineering tasks. Makes sense, I wonder how much traffic comes from automated tasks (co-pilot, etc). Every time I read a report like this I do wonder if we'll ever see an ROI on LLMs. HUNDREDS of billions of dollars of spend, and 3 years in its still primarily the same crowd using it, and has yet to create a "killer"…

Claude is mostly used by software engineers. That's an important distinction to make. I love Claude, but let's not ignore that in the LLM race, they're not exactly the leading player.

Can I ask a dumb question as an LLM newbie? What is it about Claude that makes it so good at basic software engineering tasks? Do you think it was finely tuned to be good at these tasks? No joke/trolling: A bunch of people have posted on HN in the last 6 months about creating MVPs (Minimum Viable Products) -- usually web apps -- using Claude. As a non-web-app programmer, I think this is amazing progress!

Re: The Anthropic Economic Index

#132

Earlier quoted context omitted.

> I fear the economic reality on the backside of this kind of spend. Minor nitpick. Use of the word 'spend' as a noun is not widespread and not well known.

As someone who works in finance, I would disagree. I asked ChatGPT: Is the noun spend rare? ChatGPT said: The noun "spend" is relatively rare compared to its more common form as a verb. While "spend" is widely used as a verb (meaning to give money or time for something), as a noun, it refers to an expenditure or the act of spending, and it’s not as commonly encountered. In most contexts, people would use alternatives…

Well, ChatGPT is making the same point. Not well known outside the financial industry.

The majority of audience and posters of ycombinator are not in that industry group, right?

Re: The Anthropic Economic Index

#133

This more or less confirms what I imagine most of us thought, AI is mostly used by engineers, or for engineering tasks. Makes sense, I wonder how much traffic comes from automated tasks (co-pilot, etc). Every time I read a report like this I do wonder if we'll ever see an ROI on LLMs. HUNDREDS of billions of dollars of spend, and 3 years in its still primarily the same crowd using it, and has yet to create a "killer"…

I don't think it's necessarily because of lack of generalizability. We (SWEs) built it, so we naturally have the most intimate knowledge of how to dogfood/use it. And so the cycle intensifies (use, provide feedback, improve). There's many positive examples of LLMs being useful in document based workflows in other domains as well!

Re: The Anthropic Economic Index

#134
post #43

Earlier quoted context omitted.

I think that strongly underestimates the impact LLMs, especially reasoning models, have on how code is written today.

Educate me. I find them useful but they are less so when you try to do something novel. To me, it seems like fancy regurgitation with some novel pattern matching but not quite intuition/reasoning per se. At the base of LLM reasoning and knowledge is a whole corpus of reasoning and knowledge. I am not quite convinced that LLMs will breach the confines of that corpus and the logical implications of the data there. No “…

Let's say I can't fully disclose the details because it is an area I am actively working on, but I had an algorithmical problem that was already solved in an ancient paper, but after a few hours of research I could find no open implementation of it anywhere. I thus spent quite some time re-implementing this algorithm from scratch, but it kept failing in quite a few edge cases that should have been covered by the original design.

Just to try it out, I uploaded the paper to DeepSeek-R1 and wrote a paragraph on the desired algorithm, that it should code it in Python and that the code should be as simple as possible while still working in exactly the way as described in the paper. About ten minutes later (quite a long reasoning time, but inspecting the chain of thought, it did almost no overthinking, but only reasoned about ideas I had or should have considered) it generated a perfect implementation that worked for every single test case. I uploaded my own attempt, and it correctly found two errors in my code that were actually attributable to naming inconsistencies in the original paper that the model was able to spot and fix on the fly. (The model did not output this, this I had to figure out myself.) I would have never expected AI to do that in my lifetime just two years ago.

I don't know whether that counts as "novel" to you, but before DeepSeek, I also thought that Copilot-like AI would not be able to really disrupt programming. But this one experience completely changed my view. It might be the case the model was trained on similar examples, but I find it unlikely just because the concrete algorithm cannot be found online except for the paper.

Re: The Anthropic Economic Index

#135
That tracks well with my subjective experience.

At day job - finance/office stuff - essentially zero traction despite everyone having enterprise AI subs & brainstorming sessions about use cases etc.

Then go home & do some hobby coding and suddenly it's next level useful.

It's not that the one is harder than the other, but rather that many jobs don't have an equivalent to a code base. The AI could I think grok parts of the job but typing up relevant content & what is required would take longer much than doing the task. There is nothing there to copy & paste for a quick win in the same way as code.

Re: The Anthropic Economic Index

#136

This more or less confirms what I imagine most of us thought, AI is mostly used by engineers, or for engineering tasks. Makes sense, I wonder how much traffic comes from automated tasks (co-pilot, etc). Every time I read a report like this I do wonder if we'll ever see an ROI on LLMs. HUNDREDS of billions of dollars of spend, and 3 years in its still primarily the same crowd using it, and has yet to create a "killer"…

I would agree that the products coming out so far lack imagination, but hard disagree on the impact. LLMs have completely transformed multiple industries already. In SWE, I would estimate that junior positions shrank by 70-80%, but even that is less extreme than what is going on in other industries.

In marketing, the entire low-end to mid-tier market is gone. Instead of having teams working on projects for small to mid-sized companies, there's now a single Senior managing projects with the help of LLMs. I know multiple agencies who cut staff by 80-90% without dropping revenue.

Translation (of books, articles, subtitles) was never well paid, even for very complex and demanding work. My partner did it a bit on the side, mostly justifying the low pay with some moral bla about spreading knowledge across cultures... With LLMs you can completely cut out the grunt part. You define the hard parts (terms that don't translate well), round out the edges and edge out the fluff, and every good translator becomes two to ten times more productive. Since work is usually paid by the page, people in the industry got a very decent (at least temporary) pay jump, I would imagine around 100%.

Support is probably the biggest one though. It is important to remember that outsourcing ot India only works for English speaking countries. And even that isn't super cheap. Here in Germany, if you don't have back-up wealth, it is your constitutional right to get some support from the state (~1400 euro), but you are obligated to find a job as soon as possible, and they will try to help you find a role. Support was always one of the biggest industries to funnel people towards. I talked to a friend working there, and according to them the complete industry basically stopped advertising new positions, the only ones that are left are financial services. The rest went all in on LLMs and just employ a fraction of the support stuff to deal with things escalating enough.

And that's not even touching on all the small things. How much energy is spent on creating pitch decks, communicating proposals, writing documentation etc? It probably goes up as far as 50% of work in large Orgs, and even if you can just save 5% of your time by using LLMs to phrase or organize, there is a decent ROI for companies to pay for them.

Re: The Anthropic Economic Index

#137

Earlier quoted context omitted.

Yeah. reminds me of the ancient okcupid data analysis blogs and not the creepy one by sleep8. The group I'm surprised not to see represented in their analysis is "personal", where people I know use ChatGPT as a therapist/life coach/sms analysis&editor. and of course they crucially but understandably left off the denominator. 35% of a million requests is different than 35% of a billion. and also how many of the conver…

> 35% of a million requests is different than 35% of a billion. Not statistically.

What do you mean by “statistically”? The end results would be like three orders of magnitude apart. Wouldn’t the desired sample size depend on the size of the population itself?

Re: The Anthropic Economic Index

#138

This more or less confirms what I imagine most of us thought, AI is mostly used by engineers, or for engineering tasks. Makes sense, I wonder how much traffic comes from automated tasks (co-pilot, etc). Every time I read a report like this I do wonder if we'll ever see an ROI on LLMs. HUNDREDS of billions of dollars of spend, and 3 years in its still primarily the same crowd using it, and has yet to create a "killer"…

I don't think it's necessarily because of lack of generalizability. We (SWEs) built it, so we naturally have the most intimate knowledge of how to dogfood/use it. And so the cycle intensifies (use, provide feedback, improve). There's many positive examples of LLMs being useful in document based workflows in other domains as well!

If you were on the early internet talking to someone about music or woodworking or whatever, you could reasonably assume they were a tech person because it was not simple to get online. It took a minute for it to spread.

Re: The Anthropic Economic Index

#139

Earlier quoted context omitted.

I think the problem is the data. Software engineering is a weird niche that is both a high paying job and something you can almost self-teach from widely available free online content. If not self-teach, you can rely on free online content for troubleshooting, examples, etc. A lot of other industries/jobs are more of an apprenticeship model, with little data and even less freely available on open internet.

> Software engineering is a weird niche that is both a high paying job and something you can almost self-teach If you meant programming, I agree it could be self-taught, but not SE. SE is the set of techniques, practices, tools that we have collected over decades for producing multi-versioned software that meets a certain reliability rating. Not all of these is freely available online.

There are plenty of self-taught people in the open source space making highly reliable software.

Re: The Anthropic Economic Index

#140

I'd love to see this analysis done for ChatGPT, which has a much bigger 'consumer' marketshare. I'm also very wary of their analysis method, given classifiers-gonna-classify. We already see it in their example of someone asking why their game is crashing and it buckets them into Computer & Mathematical occupation. I'm guessing the original question was not that of a game developer but rather a game player, so can you…

Yeah. reminds me of the ancient okcupid data analysis blogs and not the creepy one by sleep8. The group I'm surprised not to see represented in their analysis is "personal", where people I know use ChatGPT as a therapist/life coach/sms analysis&editor. and of course they crucially but understandably left off the denominator. 35% of a million requests is different than 35% of a billion. and also how many of the conver…

> not the creepy one by sleep8

What are you referring to?

Post reply on HN