What it feels like to work in AI right now • Hacker News Board

Re: What it feels like to work in AI right now

#141

Post by hackerlight » Fri, Apr 07, 2023, 2:39 AM UTC

UncleOxidant wrote: post #25

fdgsdfogijq wrote: post #12

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?

It probably is the secret sauce which remains undisclosed. Differences that seem small can lead to large differences in model quality.

Re: What it feels like to work in AI right now

#142

Post by Robotbeat » Fri, Apr 07, 2023, 2:39 AM UTC

VirusNewbie wrote: post #131

fdgsdfogijq wrote: post #12

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

I work at a FAANG and our unreleased models are fantastic. Now, there might be panic about how to productize it all, but tech wise i'm pretty surprised how good they are.

Not releasing the models may be the same as the models never existing in the end.

Re: What it feels like to work in AI right now

#143

Post by carabiner » Fri, Apr 07, 2023, 2:39 AM UTC

fdgsdfogijq wrote: post #12

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

Not just NLP, even 3D art: https://www.youtube.com/watch?v=SzGEfYh9ITQ

Top comment: I love seeing my job get transformed from 3D artist into prompt writer into jobless in a year or less, yay!

Re: What it feels like to work in AI right now

#144

Post by letitgo12345 » Fri, Apr 07, 2023, 2:40 AM UTC

VirusNewbie wrote: post #131

fdgsdfogijq wrote: post #12

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

I work at a FAANG and our unreleased models are fantastic. Now, there might be panic about how to productize it all, but tech wise i'm pretty surprised how good they are.

Hope so. Don't want a single corporate entity (OAI/MS) dominating the entire economy. This sector desperately needs competition

Re: What it feels like to work in AI right now

#145

Post by mmaunder » Fri, Apr 07, 2023, 2:40 AM UTC

LLM sizes have been increasing 10X every year for the last few years. Source is NVidia. That pace is staggering. GPT4’s lead is temporary and it’s a shiny ball distraction from doing dev on your own projects rather than being a user on someone else’s. Get to it and get over it. There are plenty of breakthroughs to be had.

Re: What it feels like to work in AI right now

#146

Post by letitgo12345 » Fri, Apr 07, 2023, 2:42 AM UTC

sberens wrote: post #111

JumpCrisscross wrote: post #54

Earlier quoted context omitted.

> Why do you think OpenAPI is so far out in front? There is a network effect forming around its models. The strengths of its kit speak for themselves. (It also cannot be understated how making ChatGPT public, something its competitors were too feeble, incompetent and behind the curve to do, dealt OpenAI a massive first-mover advantage.) But as others note, other models are in the ballpark. Where OpenAI is different i…

> other models are in the ballpark How true is this? From playing around with Bard and Claude, GPT-4 seems to be significantly better, especially around code generation / understanding.

People really don't seem to understand just how far ahead OAI is in its reasoning abilities (https://crfm.stanford.edu/helm/v0.2.2/?group=reasoning)

Maybe PaLM is near there (it's not evaluated on that page) but nothing else even comes close at all

Re: What it feels like to work in AI right now

#147

Post by maxilevi » Fri, Apr 07, 2023, 2:42 AM UTC

PeterisP wrote: post #139

maxilevi wrote: post #130

Earlier quoted context omitted.

But if people are paying for the product then they value it more than the money right? Tech might bring more inequality, but what is the problem with it if it substantially increases everyones standard of living?

If something makes everything take half as much effort but removes your ability to earn an income (or slashes it more than twice) then it decreases your standard of living despite substantially increasing it for many others.

You are assuming everything else stays static. Maybe if this happens, the ability to do everything in as half as much time will increase competition in a lot of areas bringing prices down and increasing your purchasing power.

Re: What it feels like to work in AI right now

#148

Post by majormajor » Fri, Apr 07, 2023, 2:45 AM UTC

UncleOxidant wrote: post #25

fdgsdfogijq wrote: post #12

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?

A lot of FAANG data folks aren't on the teams that were doing research into this stuff and weren't using the latest fruits of that research.

OpenAI has released a ton more easy-to-use-for-everyone stuff that has really leapfrogged what a lot of "applied" folks everywhere else were trying to build themselves, despite being on-the-face-of-it more "general."

Re: What it feels like to work in AI right now

#149

Post by letitgo12345 » Fri, Apr 07, 2023, 2:45 AM UTC

sangnoir wrote: post #118

UncleOxidant wrote: post #25

Earlier quoted context omitted.

Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?

> Do you think OpenAI has discovered something they haven't been open about? They have not, which makes me curious about which company gp works for because the "F" and "G" in FAANG are publicly known to already have LLMs. Not sure about Amazon, but I'm guessing they do too. As an outsider, the amazing thing about ML/AI research is that you get a revolutionary discovery of a technique or refinement that changes everyt…

Issue is the data moat OAI is building. They'll have hundreds of millions of high quality user interactions with ChatGPT they can use to finetune their models. What will anyone else including Google have?

Re: What it feels like to work in AI right now

#150

Post by chillfox » Fri, Apr 07, 2023, 2:50 AM UTC

UncleOxidant wrote: post #61

mewpmewp2 wrote: post #51

Earlier quoted context omitted.

Maybe it is just time to train and also good training data for prompts which openai has gathered for so long already? E.g. there is a bottleneck on how fast you can train and also gather good data.

Possibly, but wouldn't Google and Meta have access to way more compute resources and data than OpenAI? Google has been touting their TPUs for several years now.

Google has the compute, from the comparisons I have seen Bard smokes GPT-3.5-Turbo on response times. So my guess is that internal politics prevents them from putting out something better. There would have to be immense pressure from the search division to not make them obsolete.