Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs reward expertise

seangoedecke.com

Post reply on HN
Print view View original

21–30 of 607 posts

Re: LLMs reward expertise

#21

Post by thewebguyd »

natsucks wrote: post #13
champagnepapi wrote: post #5

Earlier quoted context omitted.

Unfortunately the software industry is saying things like "don't look at the code", "LLMs have made developers 10-100x faster", etc. The only way they can make such claims is by saying what you said above: "Claude, make me Microsoft Flight Simulator, make no mistakes". Additionally engineers are facing pressures via deadlines to work in the paradigm of "Claude, make me Microsoft Flight Simulator, make no mistakes"...

The question i wonder about is, when will an event come along that persuades everyone that human understanding is still required? Or will it never come?

Such an event would have to be pretty catastrophic at this point to slow down the inertia. Perhaps the tech debt will just pile up until someone's product implodes, or there's a massive safety issue that causes loss of life, or some big hedge fund goes bust.

Re: LLMs reward expertise

#22

Post by budsniffer952 »

postalcoder wrote: post #4

Not sure I agree with this. The math guy at anthropic's prompts are essentially: "suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!" https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7 https://xcancel.com/__alpoge__/stat…

[flagged]

Re: LLMs reward expertise

#23

Post by theredleft »

boron1006 wrote: post #17

This is true but also false. In my experience (scientific programming) AI is a giant multiplier for people with specialized knowledge. But it’s also a giant devaluer for that same knowledge as people with no idea what they’re doing can clog the field with plausible bullshit. It’s now the case that if someone tells me they’ve done something, and I look into it and find out it’s completely AI slop, then I will have spe…

we are slowly punishing reading comprehension

this will have educational consequences (that I'm trying to solve). I don't think that we can adjust without rapid education and making extreme specialists of us all.

This requires coordination, certification, licensing, and other tiers of authenticity. False experts can ruin sample gathering, can ruin training. False expertise is exemplified by the current American Administration. Look at Robert F. Kennedy Jr.; he's a false expert. He is responsible for the measles outbreak. He is responsible for ivermectin abuse by humans. False expertise is overtaking real expertise. And the results are continuously disastrous and large-scale.

Re: LLMs reward expertise

#24

Post by ModernMech »

natsucks wrote: post #8

I am feeling this a lot lately. Getting the most out of agents seems to require being able to ask the right question. And how can you ask the right questions without deep domain expertise?

Yes sometimes it’s a matter of just using the right word. You can talk to an agent about a general concept for hours and hours and it may never mention $Concept_X, but you mention $Keyword_Y and all of a sudden the AI is going on about how $Concept_X is foundational to understanding the whole thing.

Re: LLMs reward expertise

#25

Post by colechristensen »

atleastoptimal wrote: post #7
postalcoder wrote: post #4

Not sure I agree with this. The math guy at anthropic's prompts are essentially: "suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!" https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7 https://xcancel.com/__alpoge__/stat…

This works better for math because math is self-verifiable. Once you have a proof it needs no outside evidence. Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction. However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.

Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about".

This includes things like "before you start fixing this bug, write two tests that fail proving it exists".

Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.

Re: LLMs reward expertise

#26

Post by bt1a »

I often do my best to represent a genuine interest in the subject at hand and learning in general to models. Imagine the model's response prose and mannerisms being on the other polar end of answering questions simply to get the correct answers as they're often scoped for on quantitative benchmarks. Not sure I explained this well, sorry. An LLM could help

Re: LLMs reward expertise

#28

Post by bonoboTP »

postalcoder wrote: post #4

Not sure I agree with this. The math guy at anthropic's prompts are essentially: "suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!" https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7 https://xcancel.com/__alpoge__/stat…

It depends on the levels. People with differing fitness levels and ages run at very different paces. Now, do cars make them more equal or less? On the bottom end, the tide lifts all boats. Most healthy people can learn to drive and will drive "fine", they get from A to B. Out there in the city streets the car flattens the differences, everyone roughly takes the same time to get from A to B in a car.

But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.

Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.

Re: LLMs reward expertise

#29

Post by walrus01 »

bonoboTP wrote: post #10
walrus01 wrote: post #2

LLMs reward architecture knowledge of how to structure things and how to not just say "Claude, make me Microsoft Flight Simulator, make no mistakes".

Many, including myself, report having a lot of success with braindumping and not structuring anything. Just talking into speech recognition for 2-10 minutes as a stream of consciousness about what my context is, what I want, what I know already, what I have a vague hunch about, how it fits into a bigger picture, what aspects are most important to me, any footguns I already know about, really like having a chat with a…

I don't completely disagree with the concept of giving a free association thought process ramble into context. But I also bet that when you start getting it to actually generate code and link modules of things together, subroutines, functions, code structure and filenames, you still pay attention to what it does and you guide it into the architecture that makes logical sense to you.

Re: LLMs reward expertise

#30

Post by colechristensen »

fragmede wrote: post #18
natsucks wrote: post #12

Earlier quoted context omitted.

And what about problems that cannot be one-shotted but helped along?

There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it. Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.

Yes, we're in the low hanging fruit stage where folks can just say "here's the problem" and "keep going" until a result is found and it will work sometimes.

The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.

Post reply on HN