Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anyone else disillusioned with "AI experts" in their team?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 63 posts

Ask HN: Anyone else disillusioned with "AI experts" in their team?

#1

Post by randomgermanguy »

We had an internal-workshop led by our internal AI-team (mostly just LLMs), and had the horrible realisation that no one in that team actually knows what the term "AI" even means, or how a language model works.

One senior-dev (team-lead also) tried to explain to me that AI is a subfield of machine-learning, and always stochastic in nature (since ChatGPT responds differently to the same prompt).

We/they are selling tailor-made "AI-products" to other businesses, but apparently we don't know how sampling works...? Also, no one could tell me where exactly our "self-hosted" models even ran (turns out 50% of the time its just OpenAI/Anthropic), or what OCR-model our product was using.

Am I just too junior/naive to get this or am I cooked?

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#2

Post by illwrks »

I work in house and had a similar AI agency day over the last few months.

I came to the same observations; lots of experts not much expertise.

I think my wider team are on par with their ability and understanding so we now can sift through the BS a bit easier.

Nod, smile, accept that no one has a clear understanding.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#3

Post by illwrks »

illwrks wrote: post #2

I work in house and had a similar AI agency day over the last few months. I came to the same observations; lots of experts not much expertise. I think my wider team are on par with their ability and understanding so we now can sift through the BS a bit easier. Nod, smile, accept that no one has a clear understanding.

The real talent is building this stuff, everyone else is just part of the marketing effort.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#4

Post by PaulHoule »

illwrks wrote: post #3
illwrks wrote: post #2

I work in house and had a similar AI agency day over the last few months. I came to the same observations; lots of experts not much expertise. I think my wider team are on par with their ability and understanding so we now can sift through the BS a bit easier. Nod, smile, accept that no one has a clear understanding.

The real talent is building this stuff, everyone else is just part of the marketing effort.

No, it is being able to evaluate models. 5 builders without eval produce zero value. 1 eval person can pick and choose the best model out of a bunch that are open source and commercial and maybe one of them is good enough. Put an eval person together with N builders and you have a chance of making a good enough model.

If you want to know why Hacker News is full of people disappointed or skeptical with AI ask yourself why they put 99.9% of their effort into “zero-shot” when it is clear as day that if you get a few thousand examples and train in that you wipe the floor with “zero-shot”

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#5

Post by randomgermanguy »

illwrks wrote: post #2

I work in house and had a similar AI agency day over the last few months. I came to the same observations; lots of experts not much expertise. I think my wider team are on par with their ability and understanding so we now can sift through the BS a bit easier. Nod, smile, accept that no one has a clear understanding.

But why do you think this is? Like is it just the money/status that comes with calling yourself an "AI"-expert ?

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#6

Post by randomgermanguy »

illwrks wrote: post #3
illwrks wrote: post #2

I work in house and had a similar AI agency day over the last few months. I came to the same observations; lots of experts not much expertise. I think my wider team are on par with their ability and understanding so we now can sift through the BS a bit easier. Nod, smile, accept that no one has a clear understanding.

The real talent is building this stuff, everyone else is just part of the marketing effort.

What do you mean with "building this stuff"? As in building LLMs, or building applications on-top of them.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#7

Post by incomingpain »

>We had an internal-workshop led by our internal AI-team (mostly just LLMs), and had the horrible realisation that no one in that team actually knows what the term "AI" even means, or how a language model works.

I'm the AI expert for my org. Everyone else is more or less opposed to AI.

>One senior-dev (team-lead also) tried to explain to me that AI is a subfield of machine-learning, and always stochastic in nature (since ChatGPT responds differently to the same prompt).

machine learning is the sub field of AI.

Not really stochastic as far as I know. The whole random seed and temperature thing is a bit of a grey area for my full understanding. Let alone the topk, top p, etc. I often just accept what's recommended from the model folks.

>We/they are selling tailor-made "AI-products" to other businesses, but apparently we don't know how sampling works...?

Sales people dont tend to know jack. That doesnt mean they dont have an introvert in the back who does know what's going on.

>Am I just too junior/naive to get this or am I cooked?

AI for the most part has been out a couple years. With rapid improvement and changes that make 2023 knowledge obsolete. 100% of us are juniors in AI.

You're disillusioned because the "ai experts" basically dont exist.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#8

Post by rvz »

> One senior-dev (team-lead also) tried to explain to me that AI is a subfield of machine-learning, and always stochastic in nature (since ChatGPT responds differently to the same prompt).

This "senior dev" has it all mixed up and is incorrect.

"AI" is all encompassing umbrella term that includes other fields of "AI" such as the very old GOFAI (good old fashioned AI) which is rule-based, machine learning (statistical, bayesian) methods, and neural networks which deep learning and more recently generative AI (which ChatGPT) uses.

More accurately, it is neural networks which are more "stochastic" with their predictions and decisions, not just transformer models which ChatGPT is based on.

> Am I just too junior/naive to get this or am I cooked?

Quite frankly, the entire team (except you) is cooked, as you have realized what you don't know.

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#9

Post by randomgermanguy »

incomingpain wrote: post #7

>We had an internal-workshop led by our internal AI-team (mostly just LLMs), and had the horrible realisation that no one in that team actually knows what the term "AI" even means, or how a language model works. I'm the AI expert for my org. Everyone else is more or less opposed to AI. >One senior-dev (team-lead also) tried to explain to me that AI is a subfield of machine-learning, and always stochastic in nature (s…

> machine learning is the sub field of AI.

That's what I tried to explain then as well, and i brought up stuff like path-finding algorithms for route-finding (A*/heuristic-search) as an more old-school AI part, which didn't really land I think.

> Not really stochastic as far as I know. The whole random seed and temperature thing is a bit of a grey area for my full understanding. Let alone the topk, top p, etc. I often just accept what's recommended from the model folks.

I mean LLMs are often treated in stochastic nature, but like ML models aren't usually? Like maybe you have some dropout, but that's usually left out during inference AFAIK. I dont think a Resnet or YOLO is very stochastic, but maybe someone can correct me.

> AI for the most part has been out a couple years.

With this you just mean LLMs right? Because I understand AI to be way more then just LLMs & ML

Re: Ask HN: Anyone else disillusioned with "AI experts" in their team?

#10

Post by randomgermanguy »

rvz wrote: post #8

> One senior-dev (team-lead also) tried to explain to me that AI is a subfield of machine-learning, and always stochastic in nature (since ChatGPT responds differently to the same prompt). This "senior dev" has it all mixed up and is incorrect. "AI" is all encompassing umbrella term that includes other fields of "AI" such as the very old GOFAI (good old fashioned AI) which is rule-based, machine learning (statistical…

Okay thanks for saving my sanity somewhat.

And also just to nitpick/joke:

> More accurately, it is neural networks which are more "stochastic" with their predictions and decisions

I would defend NNs to not even be necessarily stochastic. I had to handwrite weights for NNs in atleast two exams, to fit XOR for example ;)

Post reply on HN