Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We ran Anthropic’s interviews through structured LLM analysis

playbookatlas.com

Post reply on HN
Print view View original

81–90 of 91 posts

Re: We ran Anthropic’s interviews through structured LLM analysis

#81

Post by polo »

Terretta wrote: post #13
jp8585 wrote: post #6

Earlier quoted context omitted.

I actually think things improved substantially when compared to last year. The latest batch of sota models is incredible (just ask any software engineer about what’s happening to their profession). It’s only a matter of time until other knowledge workers start getting the asphyxiating “vibe” coding treatment and that drama is what really fascinates me. People are absolutely torn. It seems that ai usage starts as a cl…

There’s a sense of dread that comes from realizing that it’s not useful to “do work” anymore. That in order to thrive now, we need to outsource as much of your thinking to GPT as possible. If your sense of identity comes from “pure” intellectual pursuits, you are gonna have a bad time. This is 180 degrees from how to think about it. The more thinking you do as ratio to less toil, the better. The more time to apply yo…

“It's your own atelier in a box. Go be Leonardo.”

So well put. 100% agree. Paraphrasing Steve Jobs I think of it as a mech suit for the mind.

Re: We ran Anthropic’s interviews through structured LLM analysis

#82

Post by nl »

delusional wrote: post #70
nl wrote: post #63

Earlier quoted context omitted.

I'm a decent dev and I'm possibly 100 times as productive using AI. It lets me concentrate on the parts I'm good at and ignore things I don't care about. Claude is a lot better at React than I am and I don't care.

100 times? You do in a day what used to take you 3 months? Those are just not realistic numbers.

Yes.

In the last 2 months I’ve built 3 end-to-end micro-SAAS apps.

It would easily take me a year to do that in the past because I hated some parts so much I'd just avoid working on them.

Now it's literally "bite down for and evening and it's done" whereas it could be 6 months or more before.

If anything 100 times is underestimating how much more I'm doing.

Re: We ran Anthropic’s interviews through structured LLM analysis

#83

Post by nl »

godelski wrote: post #65
wongarsu wrote: post #44

Earlier quoted context omitted.

The catch is that many professional environments have evolved values that above a certain quality floor reward quantity over quality. Even more so in the US where pointless torment is "work ethic" and pausing to think something through is "lazy" (see Bill Gate's famous quote about hiring lazy people, or "work smarter, not harder" almost being a rebel motto). Granted, that's not everywhere. There are absolutely places…

> see Bill Gate's famous quote about hiring lazy people I think this is part of why all this is so contentious. There's been a huge culture shift over the last decade and AI is really just a catalyst to it. We went from managers needing to stop engineers from using too much abstraction and optimizing what doesn't need to be optimized to the engineers themselves attacking abstraction. Just look how people turn Knuth's…

I've been programming professionally since the 1990s and our software has never been less buggy.

When was the last time you rebooted your OS, or even restarted your browser?

Software has never been as high quality as it is now.

Re: We ran Anthropic’s interviews through structured LLM analysis

#84

Post by jp8585 »

travisgriggs wrote: post #64

How do I know this fine article wasn’t the result of “Create a web page infographic report that is convincing and boils down the essential truths of how people are feeling about AI in different professions and domains.. Include statistics and numbers and some rolling/animated sound bite quotes.”

We actually have a full page on the methodology we used. We did over 20 experiments on the best way to digest this dataset and ended up settling on just using llms to review each interview in-depth. This helped us unlock some insights that would only be available if we paid someone to actually sit down and read every single interview. All other nlp approaches we used, like topic modeling, tfidf, and the roaring 2010s wordcloud,s were just too boring to even warrant a write-up. We tried bringing up as many raw excerpts to light as they make our point that much clearer. People are not simply "happy" about ai as Anthropic would have us believe, they are torn by it. It gives them a deep sense of angst and make them question the authenticity of their very existence.

Re: We ran Anthropic’s interviews through structured LLM analysis

#85

Post by jp8585 »

userbinator wrote: post #61
jp8585 wrote: post #28

Earlier quoted context omitted.

We have a full page on the methodology we used! Let me know if you’d like access to the dataset we created for this. The aim was not to be scientific but to flush out some deeper meanings from these interviews that typical nlp techniques struggle with. Ps: Of course we used llm tools as a writing aid, I’d be willing to bet those “signals” probably come from my own writing though and my appreciation of Tom Wolfe. I’ve…

We have a full page on the methodology we used! Let me know if you’d like access to the dataset we created for this. I'm not sure if you realise that those two sentences sound like 100% verbatim LLM output, or am I actually replying to a bot and not a human.

Now you are just being paranoid, lol. You have me wondering now why am I writing like an llm. (strawberry has 3 rs). Are you a bot? Shoot me an email (that we listed on the page) and let's have a zoom call

Re: We ran Anthropic’s interviews through structured LLM analysis

#86

Post by userbinator »

jp8585 wrote: post #85
userbinator wrote: post #61

Earlier quoted context omitted.

We have a full page on the methodology we used! Let me know if you’d like access to the dataset we created for this. I'm not sure if you realise that those two sentences sound like 100% verbatim LLM output, or am I actually replying to a bot and not a human.

Now you are just being paranoid, lol. You have me wondering now why am I writing like an llm. (strawberry has 3 rs). Are you a bot? Shoot me an email (that we listed on the page) and let's have a zoom call

is a stereotypical ChatGPT response.

Re: We ran Anthropic’s interviews through structured LLM analysis

#87

Post by godelski »

nl wrote: post #83
godelski wrote: post #65

Earlier quoted context omitted.

> see Bill Gate's famous quote about hiring lazy people I think this is part of why all this is so contentious. There's been a huge culture shift over the last decade and AI is really just a catalyst to it. We went from managers needing to stop engineers from using too much abstraction and optimizing what doesn't need to be optimized to the engineers themselves attacking abstraction. Just look how people turn Knuth's…

I've been programming professionally since the 1990s and our software has never been less buggy. When was the last time you rebooted your OS, or even restarted your browser? Software has never been as high quality as it is now.

I think you're looking at different timeframes and different types of bugs.

The last 5 years have been a drastic change for me. For over a decade I've used a Linux desktop and a laptop with various OSes. But in the last 5 years we went from my Linux desktop (Arch of all things!) from having "typical Linux issues" to my macbook having 10x more (and my arch machine being incredibly stable).

Yes, big picture stuff everything is stable and general purpose computers have blurred the line with servers.

BUT there's also a lot of frustrating day to day bugs that did not exist even a year ago. This Apple keyboard bug [0]? Infuriating! I personally hit a similar issue multiple times a day where, despite auto correct being disabled, it will change the word previous to the one I'm typing, even though it was already correct. Worse, pressing backspace delete two words. I'm with this user from years ago... "Am I getting older or is it becoming unbearable?"[1]

Many of the big picture things? Fine. But that doesn't mean I'm not being killed by a million little cuts. That's the problem. They're everywhere and when you complain about any single instance it is easy to brush off. But it isn't a single instance. It is eating hours of my day in 5 sec intervals. That's "buggy as shit"

[0] https://news.ycombinator.com/item?id=46232528

[1] https://news.ycombinator.com/item?id=33256168

Re: We ran Anthropic’s interviews through structured LLM analysis

#88

Post by qotgalaxy »

latentsea wrote: post #45
delusional wrote: post #25

Earlier quoted context omitted.

> just ask any software engineer about what’s happening to their profession I'm a professional developer, and nothing interesting is happening to the field. The people doing AI coding were already the weakest participants, and have not gained anything from it, except maybe optics. The thing that's suffocating is the economics. The entire economy has turned its back on actual value in pursuit of silicon valley smoke.

Nothing interesting happening in the field? If you've been paying attention the trend over the last two years has been that the problem space that requires humans to solve has been shrinking. It's continuing to shrink. That's interesting. Significantly interesting. As an engineer that's lead multiple teams including one at a world leading SaaS company, I don't consider myself one of the weakest participants in the fi…

[dead]

Re: We ran Anthropic’s interviews through structured LLM analysis

#89

Post by Terretta »

zdragnar wrote: post #66
pertymcpert wrote: post #60

Earlier quoted context omitted.

I don't necessarily agree with you completely, but I think that's a really great analogy. At the very least full of optimism.

It's a fundamentally flawed analogy. Leo's apprentices learned and improved. They studied under a master and faced serious repercussions if they bullshitted about their ability or what they had accompolished. LLM capabilities are tied to their model, and won't improve on their own. You learn the quirks of prompting them, but they have fixed levels of skill. They don't lie, because they don't understand concepts such…

All my LLM assistants are in night school learning and improving. At least I assume, since they're on such an astonishing pace of improvement.

Re: We ran Anthropic’s interviews through structured LLM analysis

#90

Post by jonplackett »

jp8585 wrote: post #79
jonplackett wrote: post #76

Earlier quoted context omitted.

If you are capable of being a leonardo, then this approach will work. Not everyone is capable of being Leonardo

I know, right? That’s part of the angst these professionals suffer. Failure, despite having the infinite leverage provided by these tools.

People who have ideas seem to forget that a lot of people do not have ideas!

And even if everyone all of a sudden starts having ideas, only a handful will be useful and successful, and all the other ideas, and people who now have their future riding on them aren’t going to do well.

If everyone was Leonardo, what does the world look like? They can’t all be the greatest polymath of the age

Post reply on HN