Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anyone else witnessing a panic inside NLP orgs of big tech companies?

old.reddit.com

Post reply on HN
Print view View original

241–250 of 527 posts

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#241

Post by drewda »

I wonder if this will be a repeat of what happened with speech recognition. It used to be a specialized field dominated by smaller companies like Nuance.

More recently Google, Microsoft, Apple, etc. decided they wanted to have speech recognition as an internal piece of their platforms.

Google poached lots of Nuance's talent. And then Microsoft bought what remained of the company.

Now speech recognition is a service integrated into the larger tech company's platforms, and also uses their more statistical/ML approaches, rather than being a component created by specialist companies/groups.

(I'm sure I'm grossly simplifying this — just seeing a potential parallel.)

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#242

Post by levidos »

CuriouslyC wrote: post #49
api wrote: post #2

It does seem like the (misnamed because it’s not open) OpenAI is very far ahead of most other efforts, especially at the edges in areas like instruction training and output filtering. Playing with Llama 65G gave me a sense for what the median raw effort is probably like. It seems to take a lot of work to fine tune and harness these systems and get them reliably producing useful output.

I don't think it's possible to build a moat around models at all. The model architectures are public, and there are already distributed group training projects so the compute isn't a barrier. The only moat is data.

OpenAI stopped publishing the architecture of GPT-4, so I'm worried that architecture availability will not be as available in the futire

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#243

Post by HPsquared »

amrocha wrote: post #198
prottog wrote: post #182

Earlier quoted context omitted.

> but you could take lots of retail or clerical jobs, advance through hard work and support a family on those wages. Those are the days that are super gone Only in certain places, and only mostly due to crazy policies that made housing ridiculously unaffordable. I'm in an area where my barber lives on 10 acres of land he didn't inherit and together with his wife raises two children. This type of relaxed life is possi…

That kind of life is not achievable on minimum wage, even if you choose to live in a small city

Only about 1% of workers are on minimum wage, you wouldn't expect an average lifestyle from that.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#244

Post by jgust »

mostlysimilar wrote: post #131
jgust wrote: post #128

Earlier quoted context omitted.

Presumably this problem is solved with technology improvements or the need is recognized to hire experts capable of generating high quality training material. In either situation, there's going to be extreme discomfort.

GPT is good because of collective knowledge, lots of data. What do you have in mind by "hire experts"? Isn't that what we have now? Many experts in many fields, hired to do their work. Cut this number down and you reduce training data.

Let's assume that GPT eliminates an entire field of experts, runs out of training data, and whoever is at the helm of that GPT program decides that it's lucrative enough to obtain more/better data. One alternative is subsidizing these experts to do this type of work and plug it directly into the model. I don't expect the nature of the work to change, more likely it's the signature on the check and the availability of the datasets.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#245

Post by salad-tycoon »

mostlysimilar wrote: post #129
MonkeyMalarky wrote: post #127

Earlier quoted context omitted.

Also what happens to the intuition and unwritten skills that humans learned and passed on over time? Sure, the model has probably internalized them implicitly from the training data. But what happens in a case where you need to have a human perform the task again (say after a devastating war)? The ones with the arcane knowledge are gone, and now humans are starting from scratch.

Incredible that we've been writing speculative fiction about this for decades and still we sleepwalk right into it. I'd love to be wrong, but I think we're all still too divided and self-interested for this kind of technology to be successfully integrated. A lot of people are going to suffer.

It’s not just sci fi. It’s has already happened in past with construction. Things like pyramids and certain cathedrals and what not are no longer possible even with machines. At least this is what I’ve read and heard, I’m not actually an engineer or architect.

Tangent, I’m looking for some sci fi about this topic. Any suggestions?

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#246

Post by layer8 »

adelie wrote: post #233
layer8 wrote: post #207

Earlier quoted context omitted.

> was too expensive. This is probably the real problem. Translators are payed shit nowadays for what is a really high-skill job. I have translators in the extended family who had to give up on that line of work because the pay wouldn’t sustain them anymore.

yep, exactly. the issue isn't that there will no longer be a need for human translators - machine translation makes subtle mistakes that legal/technical fields will need a human to double-check. the issue is that many translation jobs will, and already are, being replaced with 'proofread machine translation output' jobs that simply don't pay enough. translation checking is careful, detailed work that often takes almo…

I don’t think it’s really because “the machine is doing most of the work”, but because there’s no good way for clients to assess the quality of the supplemental human work, and therefore the market gets flooded with subpar translators who do the task sloppily on the cheap, in a way that still passes as acceptable.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#247

Post by tkgally »

hnfong wrote: post #53
hnbad wrote: post #26

When I was studying Computational Linguistics I kept running into the unspoken question: given that Google Translate already exists, what is even the point of all of this? We were learning all these ideas about how to model natural language and tag parts of speech using linguistic theory so we could eventually discover that utopian solution that would let us feed two language models into a machine to make it perfectl…

I learnt some very basics of computational linguistics since it was related to a side project. I kept wondering why people were spending huge amounts of resources into tagging and labelling corpora of thousands of words, while to me it seems that in theory it should be possible to feed wikipedia (of a certain language) into a program and have it spit out some statistically correct rules about words and grammar. I gue…

I've just started testing GPT-4 on translation from Japanese to English, but it seems to blow Google Translate out of the water. It was particularly good with a novel excerpt. I encourage people to try it with a variety of languages and texts to see if those results hold up.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#248

Post by nl »

visarga wrote: post #90
goethes_kind wrote: post #63

Earlier quoted context omitted.

I guess we are living in two different universes. Any job ad for an ML role or ML adjacent role says Ph.d required or Ph.d preferable. Maybe it is also a matter of location. I am in Germany. For a plain SWE role a Ph.d might be a disadvantage here too, but for anything ML related it is mandatory from what I can see.

In my hiring experience as an interviewer, 90% of candidates with PhD or not will actually have mediocre grasp on ML. It is a rare happy day when I get a good candidate. We interview for months for one hire. I got to interview candidates worldwide so I've seen people from many countries.

Was this hiring for ML positions?

As someone who hired for this in general we'd use PhD (or maybe a Masters degree) as a filter by HR before I even saw them.

It's true that a PhD doesn't guarantee anything though. I once interviewed a candidate with 2 PhDs who couldn't explain the difference between regression and classification (which was sort of our "ok lets calm your nerves" question).

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#249

Post by jnwatson »

moffkalast wrote: post #191
WalterBright wrote: post #161

Earlier quoted context omitted.

Imagine the devastation wrought by automatic looms, that put all the weavers out of a job! 97% of jobs used to be working on the farm. Now it's something like 2%.

Can't wait for the economy that is 97% twitch streamers because that's all what humans are left qualified for. /s

I saw a stream the other day that was just the output of an AI trained on a popular streamer’s past streams. It would select a random clip for video, respond to viewers’ comments in the voice of the streamer. It even superimposed roughly corresponding lip movements on the video.

Re: Anyone else witnessing a panic inside NLP orgs of big tech companies?

#250

Post by atemerev »

prottog wrote: post #182
xp84 wrote: post #149

Earlier quoted context omitted.

I think you and the one you're replying to are both very right. Yes, more of this money is going, instead of middle-class workers, straight to the capital class who own the "machines" that do the work people used to do. Except instead of it being a factory that makes industrial machines owned by some wealthy industrialist, the machines are things like Google and AWS and the owners are the small number of people with…

> but you could take lots of retail or clerical jobs, advance through hard work and support a family on those wages. Those are the days that are super gone Only in certain places, and only mostly due to crazy policies that made housing ridiculously unaffordable. I'm in an area where my barber lives on 10 acres of land he didn't inherit and together with his wife raises two children. This type of relaxed life is possi…

The idea that an average working person could buy a house in their twenties was only possible: 1) in America, and 2) for the baby boomers generation, maybe for some of Gen X. Nowhere else, never after. This is the exception of exceptions, not a norm.
Post reply on HN