Anyone else witnessing a panic inside NLP orgs of big tech companies? • Hacker News Board

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#171

Post by ChrisMarshallNY » Thu, Mar 16, 2023, 9:45 PM UTC

csa wrote: post #135

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> I think a lot of our current sociological problems, problems associated with wealth inequality, etc., I see where you’re coming from, but is this really the main source of the inequality? Based on numbers relating to workers’ diminishing share of profits, it seems to be that the capital class has been able to take a bigger piece of the profit pie without sharing. In the past, companies have shared profits more wide…

I have family that has been on the front lines of fighting global poverty and corruption, for their entire life (more than 50 years -at the very highest levels).

I submit that it is not hyperbole to say that probably 95% of all global human problems can have their root cause traced to poverty. That is not a scientific number, so don't ask for a citation (it ain't happening).

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#172

Post by zwkrt » Thu, Mar 16, 2023, 9:50 PM UTC

csa wrote: post #135

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> I think a lot of our current sociological problems, problems associated with wealth inequality, etc., I see where you’re coming from, but is this really the main source of the inequality? Based on numbers relating to workers’ diminishing share of profits, it seems to be that the capital class has been able to take a bigger piece of the profit pie without sharing. In the past, companies have shared profits more wide…

IMO the "main source of inequality" is that tech allows a small number of people to use technological and fiscal leverage to make an outsized impact on society as a whole. Anyone who has a job that produces value in a 1:1 way is positioned to be 'disrupted'. NLP, etc, just provides more tools for companies to increase their leverage in the market. My bet is that GPT-4 is probably better at being a paralegal than at least some small number of paralegals. GPT-5 will be better at that job than a larger percentage.

Anyone who only has the skills to affect the lives and/or environments of the people in their immediate surrounding are going to find themselves on the 'have nots' end of the spectrum in the coming decades.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#173

Post by bloppe » Thu, Mar 16, 2023, 9:50 PM UTC

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

Technology never affects the economy in isolation. It acts in concert with policy. Broadly speaking, inequality rises when capital is significantly more valuable than labor. The value of either depends on taxes, the education system, technology, and many other factors. We're never going to stop technology. We just have to adjust the other knobs and levers to make its impact positive.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#174

Post by mbgerring » Thu, Mar 16, 2023, 9:50 PM UTC

xp84 wrote: post #149

csa wrote: post #135

Earlier quoted context omitted.

> I think a lot of our current sociological problems, problems associated with wealth inequality, etc., I see where you’re coming from, but is this really the main source of the inequality? Based on numbers relating to workers’ diminishing share of profits, it seems to be that the capital class has been able to take a bigger piece of the profit pie without sharing. In the past, companies have shared profits more wide…

I think you and the one you're replying to are both very right. Yes, more of this money is going, instead of middle-class workers, straight to the capital class who own the "machines" that do the work people used to do. Except instead of it being a factory that makes industrial machines owned by some wealthy industrialist, the machines are things like Google and AWS and the owners are the small number of people with…

You can still do that with plumbing and welding

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#175

Post by malermeister » Thu, Mar 16, 2023, 9:51 PM UTC

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> given our current economic systems, how are these people supposed to eat?

I've said it before and I'll say it again. This right here is the crux of the issue. The only way people get to eat is if we change the economic systems.

Capitalism supercharged by AI will lead to misery for almost everyone, with a few Musks, Bezoses and Thiels being our neofeudal overlords.

The only hope is a complete break in economic systems, towards a techno-utopian socialism. AI could free us from having to do work to survive and usher in a Star Trek-like vision of the future where people are free to pursue their passions for their own sake.

We're at a fork in the road. We need to make sure we take the right path.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#176

Post by dogcomplex » Thu, Mar 16, 2023, 9:52 PM UTC

hnbad wrote: post #26

When I was studying Computational Linguistics I kept running into the unspoken question: given that Google Translate already exists, what is even the point of all of this? We were learning all these ideas about how to model natural language and tag parts of speech using linguistic theory so we could eventually discover that utopian solution that would let us feed two language models into a machine to make it perfectl…

Ask a toddler how the world works and you'll get a very similar response. It is entirely likely the 80%-of-human-intelligence barrier is not a "dead end" but merely a temporary limitation until these models are made to hone their understanding and update over time (i.e. get feedback) instead of going for zero-shot perfection. The GPT models incorporating video should start developing this "memory" naturally as they incorporate temporal coherence (time) into the model.

The fact we got this far through brute force is just insanely telling. This is a natural phenomena we're stumbling upon, not something crafted by humans.

Also - fun fact, the Facebook Llama model that fits on a Raspberry Pi and is almost as good as GPT3? Also basically brute force. They just trained it a lot longer and it shrunk the model. Food for thought.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#177

Post by jhrmnn » Thu, Mar 16, 2023, 9:52 PM UTC

gniv wrote: post #134

I remember thinking about this when AlphaFold was announced. Did it happen back then? Were there large shifts in companies/universities that were doing folding research?

I’ve been thinking about this. My current theory is that molecular simulation is a much more heterogeneous activity than language modeling. Language is always the same kind of data. Molecular simulations span orders of magnitude in space and time and depending on that, data and even objectives have very different form. AlphaFold is just one small piece in this puzzle and it’s very easy for a research project to incorporate AlphaFold into an existing pipeline and shift its goal.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#178

Post by mr_toad » Thu, Mar 16, 2023, 9:53 PM UTC

csa wrote: post #135

hn_throwaway_99 wrote: post #119

Wow - this is just wild. I've seen lots of arguments around "AI won't take everyone's job, it will just open up new areas for new jobs." Even if you take that with the benefit of the doubt (which I don't really think is warranted): 1. You don't need to take everyone's job. You just need to take a shitload of people's jobs. I think a lot of our current sociological problems, problems associated with wealth inequality,…

> I think a lot of our current sociological problems, problems associated with wealth inequality, etc., I see where you’re coming from, but is this really the main source of the inequality? Based on numbers relating to workers’ diminishing share of profits, it seems to be that the capital class has been able to take a bigger piece of the profit pie without sharing. In the past, companies have shared profits more wide…

There is no sharing and there never was. Companies don’t share profits with workers and they never have. Workers get paid on the marginal value of their productivity, not some portion of the total or average.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#179

Post by xp84 » Thu, Mar 16, 2023, 9:55 PM UTC

vkazanov wrote: post #97

hnfong wrote: post #53

Earlier quoted context omitted.

I learnt some very basics of computational linguistics since it was related to a side project. I kept wondering why people were spending huge amounts of resources into tagging and labelling corpora of thousands of words, while to me it seems that in theory it should be possible to feed wikipedia (of a certain language) into a program and have it spit out some statistically correct rules about words and grammar. I gue…

The secret is that there are no grammars in our brains. Rules are statistical, not precise. Rules, idioms are fluid and... statistical. We're a bit more specialised than these new models. But that's it, really.

^ This. I think the more we internalize the fact that we're also basically LLMs, the more we'll realize that there likely isn't some hard barrier beyond which no AI can climb. If you watch the things kids who are learning language say, you'll see the same kinds of slip-ups that belie the fact that they don't yet understand all the words themselves, but nobody thinks that 2-year-olds aren't people or thinks they will never learn to understand these concepts.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#180

Post by jurassic » Thu, Mar 16, 2023, 9:59 PM UTC

Maybe this is alarmist, but I don't see how LLMs don't collapse our entire economic system over the next decade or so. This is coming for all of us, not just the NLP experts in big company research groups. Being able to cheaply/instantly perform virtually any task is great until you realize there is now nobody left to buy your product or service because the entire middle class has been put out of work by LLMs. And the service industries that depend on those middle class knowledge workers will be out of work because nobody can afford to purchase their services. I don't see how this doesn't end with guillotines coming out for the owner class and/or terrorism against the companies powering this revolution. I hope I'm wrong.