Live data from Hacker News

Domain expertise has always been the real moat

brethorsting.com

411–420 of 592 posts

Re: Domain expertise has always been the real moat

#411

I find this take off. In general, LLMs collapse a bit of all knowledge work, including many domain experts. I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions I need to design the system and have an agentic system help me build parts. And it works, quite well. The places it falls flat is domain expertise that isn’t well documented, like specif…

Shallow generic domain knowledge is not really a helpful domain expertise, is it.

Lets take banks - knowing how banks work in general means little for that one specific bank who is paying you, which has different portfolio than most, different processes because everything is homebrew over decades of doing business and they differentiate from rest of the market in many ways and thats their key proposition, different data infra, set of internal apps, protocols and so on. There is no general llm knowledge you can distill in few minutes that would help you much.

You need to know your specific company, no way around it. In fact, to be proficient in those deliveries, you need to know those internal politics, processes, their quirks and so on, 100x more than some code fast delivery since this is majority of any bigger project. You need some reputation to get things done effectively.

This, for some time, won't be something llm can massively improve, unless those companies change themselves.

Re: Domain expertise has always been the real moat

#412

While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with. Personal example: I had a software engineering colleague who was the best coder of financial management systems I've ever encountered. H…

> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years.

This is the part I disagree with.

In a non-agentic world, the expert and the programmer are two different people. If the expert finds a bug in the software, they have to describe that bug, send it over to a programmer to fix, wait for a new release and until that scenario occurs again, realize that their description was actually wrong, send a corrected description, rinse and repeat for a few iterations until the bug is finally fixed for good.

In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly, and that's it. Bug fixed; with 10 minutes of work instead of a few weeks.

Re: Domain expertise has always been the real moat

#413

Earlier quoted context omitted.

Not really. The primary stopper was never time or effort. It was need (and wisdom). If a project was important enough, you’d do it. If it’s not, it falls on the wayside. Now with LLM tools, what you got is a slew of projects their creators aren’t even interested in. It’s theater.

There's just no way this can be true. Every project I've committed to has been a bet made with incomplete information. Sometimes it pans out, sometimes it doesn't. Every time I've made one of those bets, I've had to shoulder the opportunity cost of 2-3 other 1/8th-finished but promising projects I could have driven to completion instead. Not having that opportunity cost anymore wildly changes the dynamics of what I b…

> This weekend I'm playing with a SwiftUI MusicKit player[…]"Keep a continuous queue of music playing while I'm working in the wood shop"

> I have a real problem, one that's been annoying me since ~2000, which is that I "own" a lot of music but find myself stuck in an epicycle of the same 200 songs.

What’s the algorithm there? How is it different from queueing up your whole library and shuffling it?

> Go build an operating system. I'm serious! Claude will practically one-shot it.

Why would I want this? If I wanted to learn how an operating systems is built, there’s xv6 and the OSTEP book (and I’ve already gone through both) And I’ve been going through OpenBSD code, for a few questions that I had (usb audio and related code, usb mass storage and related code,…). Why would I ever wanted a generated project when there’s plenty out there to learn from.

Re: Domain expertise has always been the real moat

#414

While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with. Personal example: I had a software engineering colleague who was the best coder of financial management systems I've ever encountered. H…

> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years. This is the part I disagree with. In a non-agentic world, the expert and the programmer are two different people. If the expert finds a bug in the software,…

> In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly

That would require clearly (a) knowing what you want, and (b) expressing it unambiguously and in detail, including all edge cases. Essentially producing a spec.

Most people are not able to do either. Talking to an LLM does not change that.

Re: Domain expertise has always been the real moat

#415
post #410

While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with. Personal example: I had a software engineering colleague who was the best coder of financial management systems I've ever encountered. H…

Are we thinking about the wrong side here? Wouldn't the LLM be a super-intelligent financial expert as well as a super-intelligent coder?

It is neither.

Re: Domain expertise has always been the real moat

#416

Earlier quoted context omitted.

Can you give an example (even if contrived) of how that would have looked ? I’m very curious !

I had the experts write markdown files that contained the rules looked somewhat like: ## 1A Rule name Some prose explaining the rules liking to official documentation. ``` if municipality and inhabitants > 10000 then functionA else functionB ``` Then a trivial parser would extract the rules, the DSL was then handled by Lark[1]. So pretty simple, but it made collaborating with experts easier as simulated results would…

Sounds like "Literate Programming", where the code and comments are reversed: instead of everything being code except what's marked as a comment, everything is a comment except what's marked as code.

Re: Domain expertise has always been the real moat

#417

I find this take off. In general, LLMs collapse a bit of all knowledge work, including many domain experts. I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions I need to design the system and have an agentic system help me build parts. And it works, quite well. The places it falls flat is domain expertise that isn’t well documented, like specif…

Shallow generic domain knowledge is not really a helpful domain expertise, is it. Lets take banks - knowing how banks work in general means little for that one specific bank who is paying you, which has different portfolio than most, different processes because everything is homebrew over decades of doing business and they differentiate from rest of the market in many ways and thats their key proposition, different d…

>> You need to know your specific company, no way around it

Of course there are ways around it. For example you can build the foundation yourself (which will cover the 80% that is shared across every company in that industry) and give users the ability to build the rest themselves inside the software. This type of abstraction-based architecture is how platforms like Salesforce took off and became ubiquitous.

Re: Domain expertise has always been the real moat

#418

I find this take off. In general, LLMs collapse a bit of all knowledge work, including many domain experts. I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions I need to design the system and have an agentic system help me build parts. And it works, quite well. The places it falls flat is domain expertise that isn’t well documented, like specif…

> I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions

And you will not notice the obvious errors in its answers to even basic questions, because you are no expert and should have really consulted with one.

When talking about their own area of expertise, almost everybody immediately notices the glaring faults. But then they are very impressed and take LLMs as gospel when talking about anything else. I do not get it.

Re: Domain expertise has always been the real moat

#420

While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with. Personal example: I had a software engineering colleague who was the best coder of financial management systems I've ever encountered. H…

> So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec.

Right, so the spec is derived from examples, an interactive process that doesn’t require a programmer.

Post reply on HN