Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anyone else witnessing a panic inside NLP orgs of big tech companies?

old.reddit.com

Post reply on HN
Print view View original

251–260 of 527 posts

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#251

Post by xwdv »

yoyohello13 wrote: post #231
xwdv wrote: post #205

Earlier quoted context omitted.

Even in a world of perfect AI, there will be plenty of jobs. Anything involving movement and manipulation of matter will still require humans for the time being. We’re not at a point yet where an intelligent an AI could simply build you a house without human labor involved. Many of these jobs are cheap and easy to understand and quick to train in. These aren’t the kind of jobs people probably wanted, but they’ll be t…

Now 90% of humanity can toil for 12 hours a day in the fields to support the 10% who own all the machines. Super awesome!

When 90% of people are toiling in fields, there won’t be as many complaints I’d think because there would be a lot more equality among the 90%, vs the 50/50 split that we have today.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#252

Post by mashygpig »

org3 wrote: post #126
mashygpig wrote: post #109

Earlier quoted context omitted.

> Here's a new trend happening these days. Upon releasing new non-fiction books to the general public, authors are simultaneously offering an LLM-based chatbot box where you can ask the book any question. Can you link to an example?

https://portal.konjer.xyz/

Fascinating, thanks

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#253

Post by nottathrowaway3 »

toss1 wrote: post #214
jurassic wrote: post #180

Maybe this is alarmist, but I don't see how LLMs don't collapse our entire economic system over the next decade or so. This is coming for all of us, not just the NLP experts in big company research groups. Being able to cheaply/instantly perform virtually any task is great until you realize there is now nobody left to buy your product or service because the entire middle class has been put out of work by LLMs. And th…

The owner class gets enlightened and makes sure that the govt taxes them and implements a solid Universal Basic Income This is part of what the original UBI concept was about. If this doesn't happen, yes, there will likely be violence until it is fixed. The other view is that many technologies that were supposed to reduce work actually net added work, because now more sophisticated tasks could be done by the humans,…

The answer will either be some socialist rationing policy, or genocide.

Current "work for a living" systems only sustained the population because a human could be the most cost-effective way to get something done. Unless there are still tasks where human labor is the best option (research jobs maybe), this entire economic system will collapse.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#254

Post by GraphLover9000 »

org3 wrote: post #126
mashygpig wrote: post #109

Earlier quoted context omitted.

> Here's a new trend happening these days. Upon releasing new non-fiction books to the general public, authors are simultaneously offering an LLM-based chatbot box where you can ask the book any question. Can you link to an example?

https://portal.konjer.xyz/

I couldn't get "designing data intensive applications" to explain to me how to design a graph database (from scratch, without using existing graph frameworks or technologies), but it only suggested reasons why graph databases are useful and the properties I have to keep in mind while designing it. I want to know how I can build one in practice.

Using a prompt like "Tell me how to build a graph database from scratch. Specifically, how to design the data model, implement the data storage layer, and design the query language." only gives a very vague answer. Sometimes it suggests using existing technologies.

Anyone know what I'm missing?

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#255

Post by nl »

theGnuMe wrote: post #93
goethes_kind wrote: post #41

Earlier quoted context omitted.

That is not how it works at all. You won't get hired if you don't have the academic pedigree in the first place. That means a completed Ph.D and good publications in good journals.

Just as an fyi some of the top AI folks at OpenAI don't have PhDs. I remember reading that on Twitter (I think).

I think Greg Brockman falls into this category.

Chris Olah was at OpenAI but is now one of the founders of Anthropic but doesn't have any degree (he joined Google Brain after dropping out of his undergrad degree).

I think they are pretty exceptional though - most people on the ML team do AFAIK. Perhaps on the infrastructure side the backgrounds are different.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#256

Post by jnwatson »

jll29 wrote: post #154
hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> Imagine if you spent your entire career working on NLP, and now find GPT-4 will run rings around whatever you've done. What do you do now? I have been doing NLP since 1993. Before ca. 1996, there were mostly rule-based systems that were just toys. They lacked robustness. Then statistical systems came up and things like spell-checking (considering context when doing it), part of speech tagging and eventually even pa…

GPT-4 can already produce excellent sentence diagrams: https://twitter.com/davidad/status/1636150606384582656?s=46&...

If that’s a statistical parrot, well so are the rest of us.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#257

Post by ashout33 »

GraphLover9000 wrote: post #254
org3 wrote: post #126

Earlier quoted context omitted.

https://portal.konjer.xyz/

I couldn't get "designing data intensive applications" to explain to me how to design a graph database (from scratch, without using existing graph frameworks or technologies), but it only suggested reasons why graph databases are useful and the properties I have to keep in mind while designing it. I want to know how I can build one in practice. Using a prompt like "Tell me how to build a graph database from scratch.…

I don't really think that book is about building a graph database from scratch

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#258

Post by nl »

gattilorenz wrote: post #23
mach1ne wrote: post #16

Earlier quoted context omitted.

Depending on definition, it is solved.

You're using the wrong definition, then. /s Where is some evidence that NLP is 'solved'? What does it even mean? OpenAI itself acknowledges the fundamental limitations of ChatGPT and the method of training it, but apparently everybody is happily sweeping them under the rug: "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training,…

Natural language processing/understanding/generation is solved (at least in English).

LLMs produce perfectly fluent output and can understand natural language input as well as any human.

However knowledge representation is not solved. We still don't know how to interface a perfect LLM to other systems in the same way a human does things like looking up facts we aren't confident of or using a calculator to do math we cant' do in our head.

These are very significant problems and super important. But they are more adjacent to NLP in the same way tasks like something like Text-to-SQL [1] isn't a pure NLP task.

[1] for example https://github.com/salesforce/WikiSQL

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#259

Post by jnwatson »

mr_toad wrote: post #213
dongobread wrote: post #107

I worked in a research capacity in the voice assistant org of a big tech company until very recently. There was a lot of panic when ChatGPT came out, as it became clear that the vast bulk of the org's modeling work and research essentially had no future. I feel bad for some of my colleagues who were really specialized in specific NLP technology niches (e.g. building NLU ontologies) which have been made totally obsole…

“ Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. “ http://www.incompleteideas.net/IncIdeas/BitterLesson.html

See also the GPT-4 technical report, page 37.

https://images.app.goo.gl/vRP8368Z17zW2hvC9

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#260

Post by nl »

gattilorenz wrote: post #77
throwaway4aday wrote: post #46

Earlier quoted context omitted.

Is the goal of NLP for the model to actually understand the language it is processing? By understand I mean having the ability to relate the language to the real world and reason about it the same way a human would. To me, that goes far beyond NLP into true AI territory where the "model" is at the least conscious of its environment and possesses a true memory of past experiences. Maybe it would not be consciously awa…

LLMs have (maybe/probably) solved the language modeling problem, sure. That’s hardly NLP, right? NLG is more than “producing text with no semantics” and both NLG and NLU are only part of NLP. ChatGPT cannot even reason reliably on what it knows and doesn’t know… it’s the library of Babel, but every book is written in excellent English.

Fluency = NLP

Knowledge representation is a separate problem. NLP gives us some insights into what works here, but the multi-modal aspects of things like GPT4 show there is a lot more to knowledge presentation than just NLP.

Post reply on HN