Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What it feels like to work in AI right now

robotic.substack.com

Post reply on HN
Print view View original

161–170 of 768 posts

Re: What it feels like to work in AI right now

#161

Post by simonw »

SamvitJ wrote: post #106

"Every single person I know working in AI these days (in both the academy and industry) has been sparked by the ChatGPT moment." This does not ring true to me at all. Anecdotally, it feels the rush to get into AI (in both industry and academia, for both individuals and organizations) peaked around 2016-2020, post-AlexNet/ResNet, around the time Transformers became very popular. Hiring for ML research roles in particu…

"Hiring for ML research roles in particular definitely slowed down in 2021, and 2022..."

ChatGPT was released November 30th 2022. How's ML research hiring looking over the past four months?

Re: What it feels like to work in AI right now

#162

Post by simonw »

chrisgd wrote: post #134

Go to CHATGpT and ask for a link to some examples and half of them are 404. There is certainly a drawback to having scraped the web to build the model

Asking ChatGPT 3.5 for links to things rarely works, it usually causes it to hallucinate wildly. GPT4 is a lot better in that regard.

Re: What it feels like to work in AI right now

#163

Post by nr2x »

lexandstuff wrote: post #108
redox99 wrote: post #47

Earlier quoted context omitted.

I don't know about GPT4, but GPT3.5 I'd bet is pretty traditional and boring. It's power comes from a really good, properly curated dataset (including the RLHF). GPT3.5 turbo is much more interesting probably, because they seem to have found out how to make it much more efficient (some kind of distillation?). GPT4 if I had to make a very rough guess, probably flash attention, 100% of the (useful) internet/books for i…

In this interview [1] with Ilya Sutskever, he indicates that they aren't even close to tapping out of data. [1] https://www.youtube.com/watch?v=Yf1o0TQzry8&t=656s

That interview is mind blowing stuff.

Re: What it feels like to work in AI right now

#164

Post by ModernMech »

iamleppert wrote: post #107
xyzzy123 wrote: post #92

Earlier quoted context omitted.

I think it's not just run cost - there's also a risk of ad revenue destruction, that has to be worrying.

100% this -- a Google search requires multiple input queries, refinement, and scrolling a list of possible answers that are really just links to other web sites. The ChatGPT experience is far superior to this, for the average consumer and getting close for the power user. It's a better way to ask the Internet what it knows with a more natural interface that everyone already knows how to use -- real natural language.…

> The ChatGPT experience is far superior

Now it is. Google used to be good too, until ads started looking like search results, and then the first page became entirely ads.

In the future, when you ask ChatGPT to help you write your resume, it will try to upsell you a premium account in linked in. It will withhold its best resume advice only for LinkedIn premium users after all.

You think Clips was bad? You’ve seen nothing yet.

Re: What it feels like to work in AI right now

#165

Post by nr2x »

majormajor wrote: post #148
UncleOxidant wrote: post #25

Earlier quoted context omitted.

Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?

A lot of FAANG data folks aren't on the teams that were doing research into this stuff and weren't using the latest fruits of that research. OpenAI has released a ton more easy-to-use-for-everyone stuff that has really leapfrogged what a lot of "applied" folks everywhere else were trying to build themselves, despite being on-the-face-of-it more "general."

Exactly!

Re: What it feels like to work in AI right now

#167

Post by chopete3 »

At least, people working on ML models that handle these tasks must be feeling terrible. They know their models will be abandoned sooner or later and composed on top an LLM.

1. Classification

2. Named Entity Recognition (NER)

3. Dialog Engine

4. Sentiment Analysis

5. Tone Analysis

6. Language Translation

7. Summarization

8. Tokenization

9. Simple NLP Tasks (part-of-speech tagging, dependency parsing, lemmatization, morphological analysis)

10. Sentence Segmentation

11. Content Parsing

12. Question Answering (Structured & Unstructured)

13. Similarity

14. Grammar Correction

15. Speech to Text (ASR)

16. Text to Speech (TTS)

17. OCR

18. Image Recognition

19. Text Test Data Generation

Re: What it feels like to work in AI right now

#168

Post by jakeinspace »

lexandstuff wrote: post #108
redox99 wrote: post #47

Earlier quoted context omitted.

I don't know about GPT4, but GPT3.5 I'd bet is pretty traditional and boring. It's power comes from a really good, properly curated dataset (including the RLHF). GPT3.5 turbo is much more interesting probably, because they seem to have found out how to make it much more efficient (some kind of distillation?). GPT4 if I had to make a very rough guess, probably flash attention, 100% of the (useful) internet/books for i…

In this interview [1] with Ilya Sutskever, he indicates that they aren't even close to tapping out of data. [1] https://www.youtube.com/watch?v=Yf1o0TQzry8&t=656s

To be fair, if the opposite were true, it might not be wise to admit. Saturating available high quality training data is one of the few ways anyone can see OpenAI slowing down.

Re: What it feels like to work in AI right now

#169

Post by what-no-tests »

cgearhart wrote: post #152

From my perspective it’s just _confusing_ to work in AI right now. We have some massive models that are doing some really neat stuff, and apparently hundreds of millions of people are using them—but I keep wondering: to do _what_, exactly? I’m not asking what the models can do, I’m asking what people want the models to do every day, all the time. I’ve been shown some neat pictures people made that they thought were c…

I have read a lot of books (sociology, psychology, philosophy, history, religion, etc), and often I'll have an insight that stands in apparent contradiction with some concept I learned in one book but in alignment with a concept I've learned from another.

I use ChatGPT to understand these concepts, their background, and to bridge the gap between them.

Getting to the bottom of what Jean Baudrillard really was saying in "Simulacra and Simulation" and applying that to what Larry Wall meant when he said that Perl is the first postmodern language is really something!

Re: What it feels like to work in AI right now

#170

Post by xpe »

cgearhart wrote: post #152

From my perspective it’s just _confusing_ to work in AI right now. We have some massive models that are doing some really neat stuff, and apparently hundreds of millions of people are using them—but I keep wondering: to do _what_, exactly? I’m not asking what the models can do, I’m asking what people want the models to do every day, all the time. I’ve been shown some neat pictures people made that they thought were c…

Stop looking for the “killer app”; that metaphor is not useful here. Language models don’t need to have one, two, or a hundred “killer apps”. They are likely going to be highly distributed. They can revolutionize every textual interaction point with any person, group, or organization. For worse and better.

Instead, pay attention to how industries and people redefine themselves. There are going to be winners and losers.

Post reply on HN