Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anyone else disillusioned with "AI experts" in their team?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 63 posts

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#41

Post by zurtri »

rglover wrote: post #40

This is the nature of tech now (perhaps the whole time due to being a relatively "new" field). Most people don't have the slightest clue what they're doing beyond their ability to parrot buzzwords. Mean? Sure. Reality? You betcha. It's incredibly rare these days to encounter truly competent professionals. Most are just hoping the guy below them doesn't know enough to spot their shortfalls and speak up. This aligns sh…

This median of 5yrs experience is also backed up by Stack Overflow surveys (and Python surveys).

So where do they all go (I doubt the number of grads is doubling)?

I think a lot realise that programming is not their bag and move into account management, IT support, Business Support, or even other career's entirely.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#42

Post by randomgermanguy »

AJRF wrote: post #35
sunrunner wrote: post #31

Earlier quoted context omitted.

Ignoring floating point errors, assuming a perfectly spherical cow, and taking air resistance as zero.

Imagine you are predicting the next token, you have two tokens very close in probability in the distribution, kernel execution is not deterministic because of floating point non-associativity - the token that gets predicted impacts the tokens later in the prediction stream - so it's very consequential which one gets picked. This isn't some hypothetical - it happens all the time with LLM's - it isn't some freak accide…

Okay yes, but would you really say that the main part of non-determinism in LLM-usage stems from this ? No its obviously the topk sampling.

I don't think my tech-lead was trying to suggest the floating-point error/non-associativity was the real source.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#44

Post by linkjuice4all »

Hey I resemble that remark!

There's definitely a rush of people trying to upskill/reskill into this technology space despite having no formal training or background beyond basic dev skills. There's other people (such as myself) that came from the big data/NLP space (ads & search) that are trying to add AI to our extensive skillsets but aren't necessarily deep-math experts.

Unfortunately there's not a lot of room at the top and the vast majority of AI implementations at smaller companies are just OpenAI API wrappers. Essentially there's very little lived experience since it's expensive to experiment at home and smaller companies just aren't going to invest in self-hosted models that are expensive to run and quickly fall behind state of the art.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#45

Post by thefourthchime »

mathattack wrote: post #22
randomgermanguy wrote: post #19

Earlier quoted context omitted.

Less then 100 people all-in-all.

Shouldn’t be a political play then. In a good market I’d say you should look around. In this market, keep your head down and get some experience.

I've been at companies as small as 10 and as large as 30,000. and there is no lack of politics in smaller companies from what I've seen.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#46

Post by thefourthchime »

rvz wrote: post #8

> One senior-dev (team-lead also) tried to explain to me that AI is a subfield of machine-learning, and always stochastic in nature (since ChatGPT responds differently to the same prompt). This "senior dev" has it all mixed up and is incorrect. "AI" is all encompassing umbrella term that includes other fields of "AI" such as the very old GOFAI (good old fashioned AI) which is rule-based, machine learning (statistical…

I wonder if the senior dev actually said LLM, or at least meant LLM. If he said that, most of this checks out. The only thing is that they don't have to be stochastic, but in practice they almost always are.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#47

Post by AJRF »

randomgermanguy wrote: post #42
AJRF wrote: post #35

Earlier quoted context omitted.

Imagine you are predicting the next token, you have two tokens very close in probability in the distribution, kernel execution is not deterministic because of floating point non-associativity - the token that gets predicted impacts the tokens later in the prediction stream - so it's very consequential which one gets picked. This isn't some hypothetical - it happens all the time with LLM's - it isn't some freak accide…

Okay yes, but would you really say that the main part of non-determinism in LLM-usage stems from this ? No its obviously the topk sampling. I don't think my tech-lead was trying to suggest the floating-point error/non-associativity was the real source.

> Would you really say that the main part of non-determinism in LLM-usage stems from this

Yes I would because it causes exponential divergence (P(correct) = (1-e)^n) and doesn't have a widely adopted solution. The major labs have very expensive researchers focused on this specific problem.

There is a paper from Thinking Machines from September around Batch Invariant kernels you should read, it's a good primer on this issue of non-determinism in LLM's, you might learn something from it!

Unfortunately the method has quite a lot of overhead, but promising research all the same.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#48

Post by dboreham »

LLMs (and ML before that) have attracted a class of hand-waving bullshitter. Hardly surprising --- anyone who knows what they're doing in whatever field is going to be busy doing their thing. Meanwhile some new hot tech comes along, who has the time to poke into it? Mr Useless who never had anything to do. Meanwhile we're digging into the math of transformers and finding it fascinating while they're goofing around with "prompt engineering".

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#49

Post by randomgermanguy »

AJRF wrote: post #47
randomgermanguy wrote: post #42

Earlier quoted context omitted.

Okay yes, but would you really say that the main part of non-determinism in LLM-usage stems from this ? No its obviously the topk sampling. I don't think my tech-lead was trying to suggest the floating-point error/non-associativity was the real source.

> Would you really say that the main part of non-determinism in LLM-usage stems from this Yes I would because it causes exponential divergence (P(correct) = (1-e)^n) and doesn't have a widely adopted solution. The major labs have very expensive researchers focused on this specific problem. There is a paper from Thinking Machines from September around Batch Invariant kernels you should read, it's a good primer on this…

Alright fair enough.

I dont think this is relevant to the main-point, but it's definitely something I wasn't aware of. I would've thought it might have an impact on like O(100)th token in some negligible way, but glad to learn.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#50

Post by randomgermanguy »

bawolff wrote: post #33

The entire AI ecosystem is a giant hype bubble. I dont really think it matters much if your team understands AI, the bubble is going to pop either way.

I guess the question is if it's like the crypto-bubble, where theres no real value left in the end (haven't heard of a good use for those ASICs). Or more like the dot-com bubble where fiber-cable installed is still valuable without pets.com around.

But since I wasn't really around for either of those ... ¯\_ (ツ)_/¯

Post reply on HN