Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anyone else witnessing a panic inside NLP orgs of big tech companies?

old.reddit.com

Post reply on HN
Print view View original

451–460 of 527 posts

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#451

Post by stevenhuang »

oska wrote: post #416
xp84 wrote: post #179

Earlier quoted context omitted.

^ This. I think the more we internalize the fact that we're also basically LLMs, the more we'll realize that there likely isn't some hard barrier beyond which no AI can climb. If you watch the things kids who are learning language say, you'll see the same kinds of slip-ups that belie the fact that they don't yet understand all the words themselves, but nobody thinks that 2-year-olds aren't people or thinks they will…

Internalising the idea ( not 'fact') that 'we're basically LLMs' will only take you to a place of deep sterility, delusion and nihilism.

This has the same sort of energy as the fears of the devotedly religious that a world without religion leads people to moral depravity.

Just more pearl clutching and reverence for the mystical is what it looks like to me.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#452

Post by tremon »

msm_ wrote: post #208
moffkalast wrote: post #191

Earlier quoted context omitted.

Can't wait for the economy that is 97% twitch streamers because that's all what humans are left qualified for. /s

You joke, but an economy that is 97% artists (aka content creators) sounds... good? Isn't this the utopic end goal after we automate the scarcity out of our lifes?

And who is going to pay those artists? Entertainment professions usually serve those who have money to spend.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#453

Post by aimbivalent »

jll29 wrote: post #336
bippingchip wrote: post #4

As one of the comments on reddit posts - it's not just big tech companies, but also entire university teams which feel the goalposts moving miles ahead all of a sudden. Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already...

We are just building a chatbot to assist students that study (that study AI itself, to be precise). We could not "just use ChatGPT" because the answers have to be correct, or students could take legal action against the university in situations where they'd give an answer in an exam that the chatbot gave them and that might be wrong. It's exciting for sure to compare e.g. ChaptGPT with other approaches, for a set of…

This is variation on the common "it has to be perfect" argument. I don't really buy it.

Right now, GPT 4 would earn a top 10% SAT score, implying that it can give very high quality answers on a range of scholarly topic. This output is basically free.

Nobody that's beholden to even mild economic pressures is going to pay for an expensively constructed 100% solution if they can have the 99% solution for free.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#454

Post by Technotroll »

levidos wrote: post #189
echelon wrote: post #68

Earlier quoted context omitted.

(My posting rate limit went away) The demand for AI/ML will fast outstrip available talent. We'll be pulling students right out of undergrad if they can pass an interview. I'm hiring folks off Reddit and 4chan that show an ability to futz with PyTorch and read papers. Also, from your sibling comment: > Maybe it is also a matter of location. I am in Germany. Huge factor. US cares about getting work done and little els…

I'm DevOps engineer and I became super interested in AI recently. Any tips on how can I shift to an AI/ML career?

I recommend taking all the introductory courses you can find on both AI and ML. If you like the introductory courses, and you feel compelled to move on, then chances are you'll do well in a job regarding AI or ML. There are also several ways into it, either through pure mathematics, statistical modelling, data science, particularly through learning about various algorithms and reading papers, or even through practical application within data warehousing or day-to-day programming. I'd say it helps to have an academic background in either IT, statistics or mathematics, though, but depending on what you're aiming for it doesn't need be a firm prerequisite. Btw. linguists or anyone interested in natural language ought also apply!

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#455

Post by vkazanov »

oska wrote: post #445
vkazanov wrote: post #442

Earlier quoted context omitted.

Well, I didn't say a brain is just an LLM. It's more complicated than just this. But what LLM showed is that it works using the same simple building blocks + specialised parts of the brain: memory, constant relearning and world input, etc. So in this particular case of LLMs we've managed to optimise our way around having specialised brain structures using a powerful enough math function. Next steps require improvemen…

I don't share any of the excitement that ppl like you appear to be feeling, and I recoil from the grandiose claims being made by people who, in my opinion, are being fooled by the 21st century equivalent of a ventriloquist's doll. (I did feel excitement while following the development of AlphaZero and its played Go matches, but that was because it was revealing greater depths and beauty in the human created game of G…

I am not excited. I am terrified.

Right at the beginning of the current wave (2010-2012) of ML approaches I did some work on ML systems and NLP, and back then I clearly saw how nothing truly outstanding is happening, we were only starting to figure our what GPUs were capable of.

So all of this was fun: NLP, ML, vintage AI. But nothing felt like it did was groundbreaking, or would solve fundamental true GAI problems, or was even close.

Yet, 10 years later, here we are. Language is solved. In most areas I know /something/ about (programming, ML, NLP, compilers) this is huge and makes mountains of knowledge obsolete.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#456

Post by oska »

stevenhuang wrote: post #451
oska wrote: post #416

Earlier quoted context omitted.

Internalising the idea ( not 'fact') that 'we're basically LLMs' will only take you to a place of deep sterility, delusion and nihilism.

This has the same sort of energy as the fears of the devotedly religious that a world without religion leads people to moral depravity. Just more pearl clutching and reverence for the mystical is what it looks like to me.

> a world without religion leads people to moral depravity

This is exactly the case. Religion is a constant in human society across time and space. And one of its main functions is exactly that - to keep people away from moral depravity. Speaking against this function of religion (again, proved across all cultures and all times) only shows profound shallowness.

Personally I'm not much of a fan of Christianity or the other Abrahamic religions. And I think generally its time is coming to an end (with a long tapering off). But I think you can see quite clearly that the moving away from Christianity (probably inevitable) over the last century or so has led to moral decline and depravity in the West. If Christianity's time is coming to an end, we will neeed (and I believe we will generate) a new religion to replace it. And I don't think it will be in the Abrahamic tradition.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#457

Post by vkazanov »

oska wrote: post #445
vkazanov wrote: post #442

Earlier quoted context omitted.

Well, I didn't say a brain is just an LLM. It's more complicated than just this. But what LLM showed is that it works using the same simple building blocks + specialised parts of the brain: memory, constant relearning and world input, etc. So in this particular case of LLMs we've managed to optimise our way around having specialised brain structures using a powerful enough math function. Next steps require improvemen…

I don't share any of the excitement that ppl like you appear to be feeling, and I recoil from the grandiose claims being made by people who, in my opinion, are being fooled by the 21st century equivalent of a ventriloquist's doll. (I did feel excitement while following the development of AlphaZero and its played Go matches, but that was because it was revealing greater depths and beauty in the human created game of G…

For me AlphaZero was boring. :-) Solution space is vast but rules are simple. It was a question of time when somebody could put things together here. There was nothing unknowable about it, unlike how natural languages were always a mystery to me. Even with all the syntax, grammars, linguistic knowledge, NLP... Something was lacking.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#458

Post by vkazanov »

oska wrote: post #456
stevenhuang wrote: post #451

Earlier quoted context omitted.

This has the same sort of energy as the fears of the devotedly religious that a world without religion leads people to moral depravity. Just more pearl clutching and reverence for the mystical is what it looks like to me.

> a world without religion leads people to moral depravity This is exactly the case. Religion is a constant in human society across time and space. And one of its main functions is exactly that - to keep people away from moral depravity. Speaking against this function of religion (again, proved across all cultures and all times) only shows profound shallowness. Personally I'm not much of a fan of Christianity or the…

And astrology, and magical thinking, and many other things were always there.

So what you say is that humanity is not capable of learning moral behaviour without gods? And this is why it needs deities?

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#459

Post by oska »

vkazanov wrote: post #457
oska wrote: post #445

Earlier quoted context omitted.

I don't share any of the excitement that ppl like you appear to be feeling, and I recoil from the grandiose claims being made by people who, in my opinion, are being fooled by the 21st century equivalent of a ventriloquist's doll. (I did feel excitement while following the development of AlphaZero and its played Go matches, but that was because it was revealing greater depths and beauty in the human created game of G…

For me AlphaZero was boring. :-) Solution space is vast but rules are simple. It was a question of time when somebody could put things together here. There was nothing unknowable about it, unlike how natural languages were always a mystery to me. Even with all the syntax, grammars, linguistic knowledge, NLP... Something was lacking.

Interesting to have this contrast in perspectives. For me, the language generated by ChatGPT is flat and boring. No spark of human creativity or originality or flair. And this cheap trick of getting it to write in rhyme or 'in the style of' such and such I find awfully tacky.

I'm not saying AlphaZero was creative either. But because it was operating inside a system that was already beautiful and which had such a vast 'solution space' as you put it, its exploration into greater depths of that space I found intriguing.

I think that's the contrast for me. Machine learning can be useful and even intriguing inside constrained spaces That's why I liked AlphaZero, working inside a very constrained (but deep) space. And why I also find Tesla's progress with self-driving interesting. It's a constrained task, even though it has a huge range of variables. And again why I find ChatGPT potentially useful in drawing from a constrained corpus but still don't find the language it generates appealing. It comes across as exactly what it is - machine generated text.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#460

Post by oska »

vkazanov wrote: post #458
oska wrote: post #456

Earlier quoted context omitted.

> a world without religion leads people to moral depravity This is exactly the case. Religion is a constant in human society across time and space. And one of its main functions is exactly that - to keep people away from moral depravity. Speaking against this function of religion (again, proved across all cultures and all times) only shows profound shallowness. Personally I'm not much of a fan of Christianity or the…

And astrology, and magical thinking, and many other things were always there. So what you say is that humanity is not capable of learning moral behaviour without gods? And this is why it needs deities?

No, humans continually develop moral behaviour and codify it in religion (for transmission and social maintenance). Gods and deities aren't strictly necessary. Buddhism started without such but it is interesting that the conceptualisations of Buddhas & Boddhisattvas have become more and more like the gods & deities of other religions (at least in the popular religious traditions of Buddhism).
Post reply on HN