Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We ran Anthropic’s interviews through structured LLM analysis

playbookatlas.com

Post reply on HN
Print view View original

41–50 of 91 posts

Re: We ran Anthropic’s interviews through structured LLM analysis

#41

Post by re-thc »

blindhippo wrote: post #4
cmiles8 wrote: post #3

The story that’s solidifying is the tech is cool, it’s useful for certain things (eg, meeting note taking), but business have run a ton of “innovation lab” pilots that have returned little to no measurable value with leaders getting frustrated at the invested red ink. In short the substance isn't living up to the hype. Everywhere I look the adoption metrics and impact metrics are a tiny fraction of what was projected…

If anything, the AI bubble is reinforcing to me (and hopefully many more people) that the "markets" are anything but rational. None of the investments going on have followed any semblance of fundamentals - it's all pure instinct and chasing hype. I just hope it doesn't tear down the world for the 99% of us unable to actually reap any benefits from it. AI is basically a toy for 99% of us. It's a long long ways away fr…

> the "markets" are anything but rational

No, they are rational. At least those with a lot of money.

> None of the investments going on have followed any semblance of fundamentals - it's all pure instinct and chasing hype

That's not what investments are about. Their fundamentals are if they can get a good return on their money. As long as the odds of the next sucker to buy them up exists it is a good investment.

> AI is basically a toy for 99% of us.

You do pay for toys right? Toy shops aren't irrational?

Re: We ran Anthropic’s interviews through structured LLM analysis

#42

Post by nphardon »

gopher_space wrote: post #33
nphardon wrote: post #21

I'm a scientist and I mostly agree with the scientist part, but I am definitely collaborating with my bot, I don't view it as "just a tool". I know this because this morning I had to do a forced reboot and my VsCode wasn't connecting to our remote servers, it took like over 5 minutes after reboot to reload my bot chat, and from like minutes 3-5 I had the distinct feeling of losing a valuable colleague.

Personification can build empathy up to a point, but the machine has no desires.

I don't have illusions about whats going on on the other end, but we've done some deep collaborating and I 90% anthropomorphize it; much like how people on Star Trek TNG interact with Data.

Re: We ran Anthropic’s interviews through structured LLM analysis

#43

Post by re-thc »

fragmede wrote: post #27
WhyOhWhyQ wrote: post #15

Earlier quoted context omitted.

I have wondered about that actually. Thanks, I'll read that, looks interesting. Surely Donald Knuth and John Carmack are genuine masters though? There's the Elon Musk theory of mastery where everyone says you're great, but you hire a guy to do it, and there's the theory where you make average income but live a life fulfilled. On my deathbed I want to be the second. (Sorry this is getting off topic.)

Masters of what though? Steve Jobs wrote code early on, but he was never a great programmer. That didn’t diminish his impact at all. Same with plenty of people we label as "masters" in hindsight. The mastery isn’t always in the craft itself. What actually seems risky is anchoring your identity to being the best at a specific thing in a specific era. If you're the town’s horse whisperer, life is great right up until c…

> Steve Jobs wrote code early on, but he was never a great programmer. That didn’t diminish his impact at all.

I doubt Jobs would classify himself as a great programmer, so point being?

> So I’m not convinced mastery is about skill depth alone. It's about what survives the tool shift.

That's like saying karate masters should drop the training and just focus on the gun? It does lose meaning.

Re: We ran Anthropic’s interviews through structured LLM analysis

#44

Post by wongarsu »

Terretta wrote: post #13
jp8585 wrote: post #6

Earlier quoted context omitted.

I actually think things improved substantially when compared to last year. The latest batch of sota models is incredible (just ask any software engineer about what’s happening to their profession). It’s only a matter of time until other knowledge workers start getting the asphyxiating “vibe” coding treatment and that drama is what really fascinates me. People are absolutely torn. It seems that ai usage starts as a cl…

There’s a sense of dread that comes from realizing that it’s not useful to “do work” anymore. That in order to thrive now, we need to outsource as much of your thinking to GPT as possible. If your sense of identity comes from “pure” intellectual pursuits, you are gonna have a bad time. This is 180 degrees from how to think about it. The more thinking you do as ratio to less toil, the better. The more time to apply yo…

The catch is that many professional environments have evolved values that above a certain quality floor reward quantity over quality. Even more so in the US where pointless torment is "work ethic" and pausing to think something through is "lazy" (see Bill Gate's famous quote about hiring lazy people, or "work smarter, not harder" almost being a rebel motto).

Granted, that's not everywhere. There are absolutely places where you will be recognized for doing amazing work. But I think many feel pressured to use AI to produce high volumes of sub-par work instead of small volumes of great work

Re: We ran Anthropic’s interviews through structured LLM analysis

#45

Post by latentsea »

delusional wrote: post #25
jp8585 wrote: post #6

Earlier quoted context omitted.

I actually think things improved substantially when compared to last year. The latest batch of sota models is incredible (just ask any software engineer about what’s happening to their profession). It’s only a matter of time until other knowledge workers start getting the asphyxiating “vibe” coding treatment and that drama is what really fascinates me. People are absolutely torn. It seems that ai usage starts as a cl…

> just ask any software engineer about what’s happening to their profession I'm a professional developer, and nothing interesting is happening to the field. The people doing AI coding were already the weakest participants, and have not gained anything from it, except maybe optics. The thing that's suffocating is the economics. The entire economy has turned its back on actual value in pursuit of silicon valley smoke.

Nothing interesting happening in the field? If you've been paying attention the trend over the last two years has been that the problem space that requires humans to solve has been shrinking. It's continuing to shrink. That's interesting. Significantly interesting.

As an engineer that's lead multiple teams including one at a world leading SaaS company, I don't consider myself one of the weakest participants in the field and neither do my peers generally. I'm long on agents for coding, and have started investing heavily in making our working environment productive not only for humans, but now for agents too.

Re: We ran Anthropic’s interviews through structured LLM analysis

#46

Post by bronco21016 »

I use AI coding almost daily. I’m able to move my repositories into context easily through the multitude of AI coding tools and I see a massive boost in productivity. I say this as a junior dev. Often the outputs are “almost” and I make the necessary fixes to get it the rest of the way there.

To contrast with this, my org tried using a simple QA bot for internal docs and has been struggled to move anything beyond proof of concept. The proof of concepts have been awful. It answers maybe 60-70% of questions correctly. The major issue seems to be related to taking PDFs laced with images and poorly written explanations. To get decent performance from these RAG bots, a large FAQ has to be written for every question it gets wrong. Of course this is just my org so it can’t necessarily be extrapolated across industry. However, how often have people come across a new team and find there is little to no documentation, poorly written documentation, or outdated documentation?

Where am I going with these two thoughts? Maybe the blocker to pushing more adoption within orgs is twofold, getting the correct context into the model and having decent context to start with.

Extracting value from these things is going to require a heavy lift in data curation and developing the harnesses. So far most of that effort has gone into coding. It will take time for the nontechnical and technical to work together to move the rest of an org into these tools in my opinion.

The big bet of course then is ROI and time to adoption vs current burn rates of the model providers.

Re: We ran Anthropic’s interviews through structured LLM analysis

#47

Post by doug_durham »

WhyOhWhyQ wrote: post #40
doug_durham wrote: post #37

Is this any different than the adoption of any technology. I think of the transition from practical effects to CGI in Hollywood. Anxiety levels of the creative model builders was sky high at the time. It worked itself out and now there are different jobs.

Are they happy in their new jobs?

I presume many are. It's a different medium, but it's still creative. We got the "Mythbusters" show out of some of the model builders who didn't want to move to CGI.

Re: We ran Anthropic’s interviews through structured LLM analysis

#48

Post by latentsea »

bronco21016 wrote: post #46

I use AI coding almost daily. I’m able to move my repositories into context easily through the multitude of AI coding tools and I see a massive boost in productivity. I say this as a junior dev. Often the outputs are “almost” and I make the necessary fixes to get it the rest of the way there. To contrast with this, my org tried using a simple QA bot for internal docs and has been struggled to move anything beyond pro…

Yep. There are a lot of things that apply equally to human engineering teams in terms of productivity. Poor documentation and information architecture is a thing I have seen time and time again, and is always something I put time into course correcting for because it makes performing cognitive work much easier. Same goes for poorly factored codebases. They make doing any work feel like wading through mud. Throughout my career I have done a lot of work on what I would call platform engineering and product re-engineering and it's always to course correct for how difficult an environment has become to work in.

Agents are going to struggle with those same difficulties the way humans do too. You need to put work into making an environment productive to work in, and after having purposely switched my development workflow for the stuff I do outside of work to being "AI first on mobile", that's such a bandwidth constrained setup that it's really helping me to find all the things to optimise for to increase the batting average and minimise the back and forth.

Re: We ran Anthropic’s interviews through structured LLM analysis

#49

Post by nphardon »

fragmede wrote: post #34
nphardon wrote: post #21

I'm a scientist and I mostly agree with the scientist part, but I am definitely collaborating with my bot, I don't view it as "just a tool". I know this because this morning I had to do a forced reboot and my VsCode wasn't connecting to our remote servers, it took like over 5 minutes after reboot to reload my bot chat, and from like minutes 3-5 I had the distinct feeling of losing a valuable colleague.

Mine gets ashamed and embarrassed and goes and deletes the evidence (and my project folder! Good thing I've got backups.) when it fails. It also gets lazy and tells me to go stuff when it could do it, and I have to tell it to go do it instead of me having to go do it.

Thats wild! I have had nothing but consistent, stable experiences. It's possible they just take on the personalities of whoever theyre working with. So for me, it's become this like idealized version of a scientific collaborator. Also, I assume different models and versions have different personalities. As far as I can tell gpt-mini has no personality, whereas my claude sonnet 4.5 has a big one.
Post reply on HN