Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We ran Anthropic’s interviews through structured LLM analysis

playbookatlas.com

Post reply on HN
Print view View original

11–20 of 91 posts

Re: We ran Anthropic’s interviews through structured LLM analysis

#11

Post by Terretta »

jp8585 wrote: post #2

Anthropic released 1,250 interviews about AI at work. Their headline: "predominantly positive sentiments." We ran the same interviews through structured LLM analysis, and the true story is a bit different. Key findings: • 85.7% have unresolved tensions (efficiency vs quality, convenience vs skill) • Creatives struggle MOST yet adopt FASTEST • Scientists have lowest anxiety despite lowest trust (see ai as a tool, plai…

“Not X. Not Y. Z.” – Are you not willing to edit the tropes out?

Or maybe teach your LLM to fix itself. Starting rule set:

https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing

Re: We ran Anthropic’s interviews through structured LLM analysis

#12

Post by huevosabio »

``` Creatives have the highest struggle scores and the highest adoption rates. ```

Here is my guess for the puzzle: creative work is subjective and full of scaffolding. AI can easily generate this subjective scaffolding to a "good enough" level so it can get used without much scrutiny. This is very attractive for a creative to use on a day to day basis.

But, given the amount of content that wasn't created by the creative, the creative feels both a rejection of the work as foreign and a feeling of being replaced.

The path is less stark in more objective fields because the quality is objective, so harder to just accept a merely plausible solution, and the scaffolding is just scaffolding so who cares if it does the job.

Re: We ran Anthropic’s interviews through structured LLM analysis

#13

Post by Terretta »

jp8585 wrote: post #6
cmiles8 wrote: post #3

The story that’s solidifying is the tech is cool, it’s useful for certain things (eg, meeting note taking), but business have run a ton of “innovation lab” pilots that have returned little to no measurable value with leaders getting frustrated at the invested red ink. In short the substance isn't living up to the hype. Everywhere I look the adoption metrics and impact metrics are a tiny fraction of what was projected…

I actually think things improved substantially when compared to last year. The latest batch of sota models is incredible (just ask any software engineer about what’s happening to their profession). It’s only a matter of time until other knowledge workers start getting the asphyxiating “vibe” coding treatment and that drama is what really fascinates me. People are absolutely torn. It seems that ai usage starts as a cl…

There’s a sense of dread that comes from realizing that it’s not useful to “do work” anymore. That in order to thrive now, we need to outsource as much of your thinking to GPT as possible. If your sense of identity comes from “pure” intellectual pursuits, you are gonna have a bad time.

This is 180 degrees from how to think about it.

The more thinking you do as ratio to less toil, the better. The more time to apply your intellect with the better machine execution to back that up, the more profit.

The Renaissance grand masters used ateliers of apprentices and journeymen while the grand masters conceived, directed, critiqued, and integrated their work into commissioned art; at the end signing their name: https://smarthistory.org/workshop-italian-renaissance-art/

This is how to leverage the machine. It's your own atelier in a box. Go be Leonardo.

Re: We ran Anthropic’s interviews through structured LLM analysis

#14

Post by Terretta »

WhyOhWhyQ wrote: post #9
dionian wrote: post #5

Earlier quoted context omitted.

I multi task much more now that i can farm off small coding assignments to agents. i pay hndreds per month in tokens. for my role personally its been a massive paradigm shift.

To each his own, but multi-tasking feels bad to me. I want to spend my life pursuing mastery of a craft, not lazily delegating. Not that everyone should have the same goals, but the mastery route feels like it's dying off. It makes me sad. I get it that some people just want to see the thing on the screen. Or your priority is to be a high status person with a loving family etc.. etc... All noble goals. I just don't f…

> I want to spend my life pursuing mastery of a craft, not lazily delegating.

And yet, the Renaissance "grand masters" became known as masters through systematizing delegation:

https://smarthistory.org/workshop-italian-renaissance-art/

Re: We ran Anthropic’s interviews through structured LLM analysis

#15

Post by WhyOhWhyQ »

Terretta wrote: post #14
WhyOhWhyQ wrote: post #9

Earlier quoted context omitted.

To each his own, but multi-tasking feels bad to me. I want to spend my life pursuing mastery of a craft, not lazily delegating. Not that everyone should have the same goals, but the mastery route feels like it's dying off. It makes me sad. I get it that some people just want to see the thing on the screen. Or your priority is to be a high status person with a loving family etc.. etc... All noble goals. I just don't f…

> I want to spend my life pursuing mastery of a craft, not lazily delegating. And yet, the Renaissance "grand masters" became known as masters through systematizing delegation: https://smarthistory.org/workshop-italian-renaissance-art/

I have wondered about that actually. Thanks, I'll read that, looks interesting.

Surely Donald Knuth and John Carmack are genuine masters though? There's the Elon Musk theory of mastery where everyone says you're great, but you hire a guy to do it, and there's the theory where you make average income but live a life fulfilled. On my deathbed I want to be the second. (Sorry this is getting off topic.)

Re: We ran Anthropic’s interviews through structured LLM analysis

#16

Post by ctoth »

huevosabio wrote: post #12

``` Creatives have the highest struggle scores and the highest adoption rates. ``` Here is my guess for the puzzle: creative work is subjective and full of scaffolding. AI can easily generate this subjective scaffolding to a "good enough" level so it can get used without much scrutiny. This is very attractive for a creative to use on a day to day basis. But, given the amount of content that wasn't created by the crea…

Possible confound (seems important):

"creatives" tend to have a certain political tribe, that political tribe is well-represented in places that have this precise type of authenticity/etc. language around AI use...

Basically a good chunk of this could be measuring whether or not somebody is on Bluesky/is discourse-pilled... and there's no way to know from the study.

Re: We ran Anthropic’s interviews through structured LLM analysis

#17

Post by jp8585 »

Terretta wrote: post #13
jp8585 wrote: post #6

Earlier quoted context omitted.

I actually think things improved substantially when compared to last year. The latest batch of sota models is incredible (just ask any software engineer about what’s happening to their profession). It’s only a matter of time until other knowledge workers start getting the asphyxiating “vibe” coding treatment and that drama is what really fascinates me. People are absolutely torn. It seems that ai usage starts as a cl…

There’s a sense of dread that comes from realizing that it’s not useful to “do work” anymore. That in order to thrive now, we need to outsource as much of your thinking to GPT as possible. If your sense of identity comes from “pure” intellectual pursuits, you are gonna have a bad time. This is 180 degrees from how to think about it. The more thinking you do as ratio to less toil, the better. The more time to apply yo…

I definitely understand that this is the rational way of viewing it. Leveraging these tools is an incredible feeling, but the sense of dread is always there in the corner. You can just feel a deep sense of angst in a lot of these interviews. In any case, I would rather have them and use them to their full extent than to become obsolete. Becoming Leonardo it is.

Re: We ran Anthropic’s interviews through structured LLM analysis

#18

Post by layer8 »

huevosabio wrote: post #12

``` Creatives have the highest struggle scores and the highest adoption rates. ``` Here is my guess for the puzzle: creative work is subjective and full of scaffolding. AI can easily generate this subjective scaffolding to a "good enough" level so it can get used without much scrutiny. This is very attractive for a creative to use on a day to day basis. But, given the amount of content that wasn't created by the crea…

One issue with AI for creatives is that it’s virtually impossible to get AI to create a specific vision you have in mind. It creates something, but you just have to accept whatever that is, you can only steer it very roughly. It can be useful for getting inspiration, but not for getting exact results. If AI was better suited for realizing one’s own creative vision and working in a detail-oriented fashion, creators would likely embrace it more.

Re: We ran Anthropic’s interviews through structured LLM analysis

#19

Post by fragmede »

blindhippo wrote: post #10
dionian wrote: post #5

Earlier quoted context omitted.

I multi task much more now that i can farm off small coding assignments to agents. i pay hndreds per month in tokens. for my role personally its been a massive paradigm shift.

Might work for you, but if I multi task too much, the quality of my output drops significantly. Where I work, that does not fly. I cannot trust any agent to handle anything without babysitting them to avoid going off the rails - but perhaps the tools I have access to just aren't good (underlying model is claude 4.5, so it the model isn't the cause). I've said this in the past and I'll continue to say it - until the t…

If you can predict that hitting “summarize conversation” equals rework, what can you change upstream so you avoid triggering it? Are you relying on the agent to carry state instead of dumping it into .MD files? What happens if your computer crashes?

> so it the model isn't the cause

Thing is, the prompts, those stupid little bits of English that can't possiu matter all that much? It turns out they affect the models performance a ton.

Re: We ran Anthropic’s interviews through structured LLM analysis

#20

Post by fragmede »

blindhippo wrote: post #4
cmiles8 wrote: post #3

The story that’s solidifying is the tech is cool, it’s useful for certain things (eg, meeting note taking), but business have run a ton of “innovation lab” pilots that have returned little to no measurable value with leaders getting frustrated at the invested red ink. In short the substance isn't living up to the hype. Everywhere I look the adoption metrics and impact metrics are a tiny fraction of what was projected…

If anything, the AI bubble is reinforcing to me (and hopefully many more people) that the "markets" are anything but rational. None of the investments going on have followed any semblance of fundamentals - it's all pure instinct and chasing hype. I just hope it doesn't tear down the world for the 99% of us unable to actually reap any benefits from it. AI is basically a toy for 99% of us. It's a long long ways away fr…

> AI is basically a toy for 99% of us.

So you're at the "first they laugh at us" stage then.

Post reply on HN