Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Telling Me to Ask an LLM

blog.yaelwrites.com

Post reply on HN
Print view View original

101–110 of 154 posts

Re: Stop Telling Me to Ask an LLM

#101

Post by ohyes »

Okay. In my experience asking an LLM requires you to know the right set of questions and the right process for working through the problem that you’re having. You have to know the correct terminology to use to describe your problem.

These are not trivial things even though they are things that senior developers tend to trivialize.

If you’re describing the wrong problem, you’ll get a right answer that doesn’t fix it.

If you’re describing the right problem wrongly, you’ll get another wrong answer that talks past being right.

If you’re describing a difficult problem that maybe isn’t even solved in existing stuff, you can get help figuring out the necessary steps if you have an idea where to start.

If you don’t know where to start you can start by asking where to start.

Anyway, point is, I can’t help you if you don’t tell me what you know (or think you know) and what you don’t know. I don’t have a magic wand, I have years of experience grinding down problems until they submit.

Re: Stop Telling Me to Ask an LLM

#102

Post by t-writescode »

paulhebert wrote: post #86
t-writescode wrote: post #68

Earlier quoted context omitted.

Beyond mirroring the engineering practices that you yourself want to see other people perform, have you found any techniques to get people to … in short, do their job again? Understand context, understand what they did, why they did it, what they’re doing, etc. The +/-2000 line MR was bad when humans wrote it. It’s way worse when the human didn’t even write or read it. And just vomiting automated CodeRabbit talking p…

I tried to get our team to enforce a max PR size. It worked for a while. There is a GitHub action that you can configure to fail if the PR is too large. But then we started a new project and I didn’t add it right away since it’s a small team and I figured we could use the honor system. Since then there have been lots of massive PRs but there’s not much willingness to go back to enforcing the rule because it might slo…

Out of curiosity, with their current paradigm, what's the regression rate *and* how long do PRs stay in pending?

Re: Stop Telling Me to Ask an LLM

#104

Post by yaelwrites »

Krssst wrote: post #103
triyambakam wrote: post #48

Ok but he clearly used a model to write this so...

Oh, it did not feel AI but Pangram does say AI, high confidence. Good catch. (I'd trust an ML algorithm that only has to classify in two boxes, thus easy to evaluate, over my own gut feeling).

I wrote this by hand but I did run it through an AI for editing. The changes were pretty minor; just a couple of lines.

Re: Stop Telling Me to Ask an LLM

#105

Post by fallingfrog »

Yeah I mean LLM's don't generally offer useful answers to questions. They just kinda make the same guesses that any reasonably educated and articulate but not very smart person would make. They will answer, "maybe its this, maybe its that", but they can't go and do actual investigation and digging and find the real answer to a question. They just wing it. I can already pop possibilities off the top of my head.

I don't know, maybe if you were generally not very informed the AI might seem knowledgeable by comparison?

Re: Stop Telling Me to Ask an LLM

#106

Post by rubyfan »

Lots of people I work with seem to use Claude for things they should be learning or doing themselves. Stuff that really isn’t hard and if they took the time to learn themselves then they would be better at their job and more valuable.

Instead they are telling me that literally any other human could do their job and their only value is prompting a chat bot.

The thing is they aren’t even accelerating outcomes. Since they can’t be bothered to learn they pump out stuff that looks good but the detail that matters is trash and requires someone else to actually finish to an acceptable standard.

Re: Stop Telling Me to Ask an LLM

#107

Post by peteforde »

em-bee wrote: post #100
peteforde wrote: post #64

Earlier quoted context omitted.

You're projecting several layers of bias onto this scenario. The more busy you are, the more valuable your time... the more expensive context switching is. When you are known as the person with the answers, your day is at least partially structured around getting people to leave you tf alone so that you can actually concentrate on getting your own work done. There's a really toxic expectation that people who are seni…

if this is the experience in a team, then the team already has a problem. i expect these things to be made clear up front. when someone joins a team they should be assigned a mentor whom they can ask any question, no matter how dumb, and the mentor then guides the new team member in how to approach such a question, which at that point can include asking the LLM. it is the mentors responsibility to point out when it i…

Alternative take: the assumption that senior engineers get pressured into agreeing to mentor without it formally being an acknowledged part of their job - that is, that a significant percentage of their time is supposed to be allocated to making themselves available to juniors - is fundamentally a problem.

There is so much stigma associated with being an senior engineer that simply wants to spend 95% of their day working on the problems that they were hired to solve. The worst part is that the vast majority of people in this situation are not compensated for this time, and they are expected to keep up with their actual assigned responsibilities.

This state of affairs is a relatively new thing. The idea that you would join a company with the expectation that the most important people to the success of a project should drop what they are doing to context shift to someone else's problem several dozen times a day is not something that would have been remotely normal twenty years ago.

I am not saying that mentoring is bad or that asking for help is bad, just that there's been a change and the unspoken vibe is that if you're not happy to work at ~30% capacity because you need to mentor people, you're some kind of antisocial jerk.

If a company wants seniors to mentor, pay them to mentor. It's very simple.

Re: Stop Telling Me to Ask an LLM

#108

Post by jeffnash »

Aurornis wrote: post #11

The subtitle is: > I already did. They repeat multiple times in the article that asking Claude was something they already did. So this isn’t an anti-LLM article. This seems to be a communication problem. The other party either doesn’t know that they’ve put a lot of effort into researching this already, or their trying to give a gentle let-down instead of saying they don’t have time for this. For the first case, the s…

I agree 100%. I've been the resident 'tech guy' among my friends and family for as long as I can remember, and I thought that LLMs would be the definite end to all 'blurry picture of computer screen captioned with "what should I do"' texts, yet that didn't seem to be the case. In fact, LLMs made it worse, as it actually made me seem more knowledgeable; I would be the one asking AI on their behalf and responding with beautifully formatted, easy to understand answers, encouraging them to ask me more.

I chuckled because, as a joke to express my frustration about this, I made a website called djfa.ai (Dude, Just Fucking Ask AI), which is essentially LMGTFY for ChatGPT. As one can imagine, the gimmick wore off rather quickly with my loved ones (turns out moms of nerds sometimes ask their children tech questions because they miss their voice). I ended up haphazardly turning it into my personal blog, which I sometimes choose to abbreviate and sometimes not on my resume.

I would never have made it in the first place had most of the people asking me questions been like Yael: seeking my informed opinion on thought-provoking questions they'd already researched. Going back to your larger point about the social aspect of it all, even among my wife and my social group, "just ask Claude" is almost the new "I don't really know" or "that's a tough one" when it comes to any sort of question. Almost colloquial in the sense that it isn't to be taken literally, but more as an indication of uncertainty.

Re: Stop Telling Me to Ask an LLM

#109

Post by dwaltrip »

yaelwrites wrote: post #104
Krssst wrote: post #103

Earlier quoted context omitted.

Oh, it did not feel AI but Pangram does say AI, high confidence. Good catch. (I'd trust an ML algorithm that only has to classify in two boxes, thus easy to evaluate, over my own gut feeling).

I wrote this by hand but I did run it through an AI for editing. The changes were pretty minor; just a couple of lines.

The first paragraph screamed AI to me. I'll try to dissect each bit, see below. It's hard to put it into words...

---

Send an e-mail + pick up the phone. Mismatched analog. First one is outgoing, second is incoming. Yes, I know you can pick up "to make a call". In which case, then we have the classic LLM pattern of eliding details that matter to make it sound "smooth" (like blended cardboard smooth, glug glug).

someone senior enough - Way too vague. Most people would add details here that are unique to the actual human being that they spoke with. But Claude can't do that, so we get this.

a hard question - Same thing as the previous one. Which question? Why is there literally no detail or actual flavor here?

- Real - Claude looooves punchy, empty emphasis words like this.

... , the kind you ... - Tacked on extra phrase that should be a separate sentence. Claude does this all the time. We aren't writing a shitting noir novel, Claude.

-a decision go sideways in a boardroom - vague shadow of an anecdote, of just the right length. The "a" in front of "boardroom" is a blinking red light for me, can't fully explain why.

Not what the textbook says. - Too short, too vague, with the rhythmic punchy feel. (Damn... I've been infected with the triplet pattern myself).

textbook, 5 studies, 30 years of experience - Pattern of 3...

- that a search engine couldn't - Oh man... this type of phrasing is so triggering for me, heh. I've been busting my ass trying to sanitize Claude's output so it haven't doesn't these overly cute, inverted constructions.

Post reply on HN