Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Anthropic Economic Index

anthropic.com

Post reply on HN
Print view View original

111–120 of 238 posts

Re: The Anthropic Economic Index

#111

Post by fragmede »

azinman2 wrote: post #109

I'd love to see this analysis done for ChatGPT, which has a much bigger 'consumer' marketshare. I'm also very wary of their analysis method, given classifiers-gonna-classify. We already see it in their example of someone asking why their game is crashing and it buckets them into Computer & Mathematical occupation. I'm guessing the original question was not that of a game developer but rather a game player, so can you…

Yeah. reminds me of the ancient okcupid data analysis blogs and not the creepy one by sleep8. The group I'm surprised not to see represented in their analysis is "personal", where people I know use ChatGPT as a therapist/life coach/sms analysis&editor. and of course they crucially but understandably left off the denominator. 35% of a million requests is different than 35% of a billion. and also how many of the conversations had 1 message, indicating "just testing" vs 10 or 100 messages.

Re: The Anthropic Economic Index

#112

Post by hansonkd »

noah_buddy wrote: post #79
lblume wrote: post #43

Earlier quoted context omitted.

I think that strongly underestimates the impact LLMs, especially reasoning models, have on how code is written today.

Educate me. I find them useful but they are less so when you try to do something novel. To me, it seems like fancy regurgitation with some novel pattern matching but not quite intuition/reasoning per se. At the base of LLM reasoning and knowledge is a whole corpus of reasoning and knowledge. I am not quite convinced that LLMs will breach the confines of that corpus and the logical implications of the data there. No “…

I think the trap people fall in is that LLMs don't need to be novel or reason as well as a human to revolutionize society.

Plenty of value is already added just by converting unstructured data to structured data. If that is all LLMs did they would be still be a revolution in programming and human development. So much manual entry and development work has essentially evaporated overnight.

If there was never a chat based LLM "agent" LLMs just converting arbitrary text to structured JSON schema would be the biggest advancement in comp sci since the internet. There is nothing equivalent that existed before except for manual extraction or rule based hard coding.

Judging LLMs based on some criteria of creativity or intuition from a chat is missing the forest for the trees.

Re: The Anthropic Economic Index

#113

Post by jtrn »

It would be interesting to know how much of this effect is also explained by factors other than income and profession, and even types of work.

I work both as a software developer and a psychologist, and I love tinkering in the shop with welding and mechanics. It is extremely obvious that using AI is more available and appropriate when coding, as you're often in front of a very capable computer with a good interface to interact with. When I am a psychologist, it's not as fitting to bring out a computer and input prompts. And when I'm working in the shop, it's more of a hassle to grab the phone and ask a question.

Types of work and knowledge work, obviously, are ripe for integration with AI tools, but I think the pure ease of use/availability is a major factor. Sometimes two seconds of extra work to do something is the difference between not doing it and doing it.

I'm a heavy user of dictation and voice-assisted features on mobile phones, but it just doesn't cut it when you have to fight with the phone to select text and copy-paste. (The clicking of selected text to copy is so temperamental, and why the hell is the contextual menu so inconsistent after you've selected text still! I selected the text and waited for the tooltip to appear, but it only does so if it feels like it still.)

Anyways, "ease of use for a given profession" vs "Actual usage" is also important, is my point... [Edit for spelling]

Re: The Anthropic Economic Index

#114

Post by ramon156 »

> While Claude.ai data contains some non-work conversations, we used a language model to filter this data to only contain conversations relevant to an occupational task, which helps to mitigate this concern.

That's nice. My main prompt has a hint suggesting when I refer to work. I do this so Claude can assume my tech-stack. Of course I exclude or mask confidential data, but it's still nice this stuff gets filtered.

Re: The Anthropic Economic Index

#115

Post by Philpax »

jtrn wrote: post #113

It would be interesting to know how much of this effect is also explained by factors other than income and profession, and even types of work. I work both as a software developer and a psychologist, and I love tinkering in the shop with welding and mechanics. It is extremely obvious that using AI is more available and appropriate when coding, as you're often in front of a very capable computer with a good interface t…

Have you tried ChatGPT's advanced voice mode, and if so, what did you think of it?

Re: The Anthropic Economic Index

#116

Post by raylad »

azinman2 wrote: post #109

I'd love to see this analysis done for ChatGPT, which has a much bigger 'consumer' marketshare. I'm also very wary of their analysis method, given classifiers-gonna-classify. We already see it in their example of someone asking why their game is crashing and it buckets them into Computer & Mathematical occupation. I'm guessing the original question was not that of a game developer but rather a game player, so can you…

Very wary. Not weary.

Re: The Anthropic Economic Index

#117

Post by hn_acc1 »

(not an experienced AI person, just a software dev whose used it a bit. Don't know all the players, every term for every model, etc)

We have github copilot and augment available for making suggestions inside vscode. I don't think either are anthropic - but I'm sure they offer a similar feature. I wonder if they count EVERY suggestion offered as a "use"? Sometimes it really helps, but it makes plenty of suggestions I ignore. Does it essentially treat every keystroke as a use then, since it updates / re-suggests sometimes with every keystroke?

Re: The Anthropic Economic Index

#118

Post by Ancalagon »

frankfrank13 wrote: post #92

This more or less confirms what I imagine most of us thought, AI is mostly used by engineers, or for engineering tasks. Makes sense, I wonder how much traffic comes from automated tasks (co-pilot, etc). Every time I read a report like this I do wonder if we'll ever see an ROI on LLMs. HUNDREDS of billions of dollars of spend, and 3 years in its still primarily the same crowd using it, and has yet to create a "killer"…

Maybe spend and better models will help this (I’ve not used the deep research models so maybe we are there already). But even day to day coding, the LLMs are great helpers but giving them anything more than a slightly complicated prompts and it seems like these models become completely helpless. You just constantly need a human in the loop because these models are too dumb or lack the context to understand the big picture.

Maybe these models will get better as they’re given more context and can understand the full stack but for now they cannot.

And this is just with code where it already has billions of examples. Nevermind any less data-rich fields. The models still need to get smarter.

Re: The Anthropic Economic Index

#119

Post by steveBK123 »

frankfrank13 wrote: post #92

This more or less confirms what I imagine most of us thought, AI is mostly used by engineers, or for engineering tasks. Makes sense, I wonder how much traffic comes from automated tasks (co-pilot, etc). Every time I read a report like this I do wonder if we'll ever see an ROI on LLMs. HUNDREDS of billions of dollars of spend, and 3 years in its still primarily the same crowd using it, and has yet to create a "killer"…

I think the problem is the data.

Software engineering is a weird niche that is both a high paying job and something you can almost self-teach from widely available free online content. If not self-teach, you can rely on free online content for troubleshooting, examples, etc.

A lot of other industries/jobs are more of an apprenticeship model, with little data and even less freely available on open internet.

Re: The Anthropic Economic Index

#120

Post by ActionHank »

acomms wrote: post #24
ActionHank wrote: post #15

Earlier quoted context omitted.

As PR

Is there a way to do it as not PR?

Yes, publish a paper or the stats.

Putting it on your company blog is marketing, always.

The only exception is PornHub insights, because they don't need to advertise and the people reading are there for the insights.

Post reply on HN