Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What it feels like to work in AI right now

robotic.substack.com

Post reply on HN
Print view View original

91–100 of 768 posts

Re: What it feels like to work in AI right now

#91

Post by letitgo12345 »

fdgsdfogijq wrote: post #39
UncleOxidant wrote: post #25

Earlier quoted context omitted.

Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?

I dont work at Google, I think other FAANGS underinvested in this area as they didnt think it was promising. But I will admit, I am suspicious that Google is incompetent. Probably they can come back given how much money they will be forced to throw at it. But Bard is clearly behind and I dont believe their "abundance of caution" arguments for why Bard cant even code.

From what I heard, they don't have scaling capabilities to deploy the bigger models right now which they're working on and that's the biggest issue.

Re: What it feels like to work in AI right now

#92

Post by xyzzy123 »

pixl97 wrote: post #75
UncleOxidant wrote: post #48

Earlier quoted context omitted.

The impression I get is that they purposely limited Bard in certain ways. It does not seem to be willing to code and makes a lot of excuses when you ask it to (at least as of a couple of weeks ago when I last tried it). Did they put this limitation on it because it's not so good at coding or because they don't want it to be abused in some way? In my experience I had to trick it into coding by telling it to pretend th…

I am still wondering if Google is experiencing it Kodak moment. They have the dominant product that makes them billions and billions of dollars at 'relatively' low cost. The new dominant product is on its way, but it costs far more to operate and will net them far less money, so... um no one wants to kill the goose that is still laying golden eggs, even though its days are numbered already.

I think it's not just run cost - there's also a risk of ad revenue destruction, that has to be worrying.

Re: What it feels like to work in AI right now

#93

Post by [deleted] »

paulryanrogers wrote: post #89
JumpCrisscross wrote: post #54

Earlier quoted context omitted.

> Why do you think OpenAPI is so far out in front? There is a network effect forming around its models. The strengths of its kit speak for themselves. (It also cannot be understated how making ChatGPT public, something its competitors were too feeble, incompetent and behind the curve to do, dealt OpenAI a massive first-mover advantage.) But as others note, other models are in the ballpark. Where OpenAI is different i…

Doubtful "feeble and incompetent" are fitting considering much of the research behind OpenAI was pioneered and published by those predecessors. More like "unwilling", and perhaps for good reason. Time will tell.

[deleted]

Re: What it feels like to work in AI right now

#94

Post by TechnicolorByte »

anon7725 wrote: post #21
fdgsdfogijq wrote: post #12

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

Can confirm. People are scrambling to remain relevant.

How does that manifest specifically?

Re: What it feels like to work in AI right now

#95

Post by NikolaNovak »

__loam wrote: post #38
hvs wrote: post #26

Earlier quoted context omitted.

It has helped me learn things more interactively because I can ask questions and get answers without having to read 10 blog posts or a book before understanding things. For my personality, it would be hard to recreate without having a living mentor available 24/7 that had knowledge of everything I wanted to ask of it. Does it get things wrong? Sure. But generally the subtle ways it might get stuff wrong would be the…

It's shocking to me how many people are treating chatGPT as a reliable source given how much we know it hallucinates.

But... How much does it hallucinate? Are there safer areas or ways of asking questions, than others? Genuine question. I found that when I ask simple factual questions in a... "non pushy non presumptive" manner, it gives accurate basic answers in areas I'm familiar with. I ask it simple things about basic French or basic Python or basic music theory and so far it's been... brilliant!

I found most examples of hallucination on the web to be pushy or presumptive or edgy I. E. "tell me the ways in which vaccines cause male pattern baldness" and then manipulate prompts.

So I assume there are areas and approaches that are safer and more useful, and areas and approaches that are riskier?

Re: What it feels like to work in AI right now

#96

Post by WalterSear »

__loam wrote: post #34
pgwhalen wrote: post #4

Earlier quoted context omitted.

I'm curious why you draw the parallel - did crypto ever have a ChatGPT/iPhone moment? As an observant outsider, my impression is that every "moment" of note in crypto has been about some price going way up or way down - there has never been a "this is the use case" moment.

Theoretically ethereum is that for crypto, if you believe the crypto bros.

If Ethereum was that, we wouldn't still be waiting for real world use cases. GPT is already making me more productive.

Re: What it feels like to work in AI right now

#97

Post by 2-718-281-828 »

fdgsdfogijq wrote: post #12

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

come on, it's not that bad, at least you're doing linear regression

Re: What it feels like to work in AI right now

#98

Post by brendamn »

redox99 wrote: post #47
UncleOxidant wrote: post #25

Earlier quoted context omitted.

Why do you think OpenAPI is so far out in front? It's not like there's a lot of secret sauce here - most of this stuff (transformers,etc.) is all out there in papers. And places like Google & Meta must have a lot more computing resources to train on that OpenAI does thus they should be able to train faster. Do you think OpenAI has discovered something they haven't been open about?

I don't know about GPT4, but GPT3.5 I'd bet is pretty traditional and boring. It's power comes from a really good, properly curated dataset (including the RLHF). GPT3.5 turbo is much more interesting probably, because they seem to have found out how to make it much more efficient (some kind of distillation?). GPT4 if I had to make a very rough guess, probably flash attention, 100% of the (useful) internet/books for i…

> I'd say with GPT4 they probably reached the limit of how big the dataset can be

I’m curious about this too; not just on the dataset size, but also the model size. My hunch is that the rapid improvements of the underlying model by making it bigger/giving it more data will slow, and there’ll be more focus on shrinking the models/other optimisations.

Re: What it feels like to work in AI right now

#99

Post by twelve40 »

letitgo12345 wrote: post #64
xnx wrote: post #33

Earlier quoted context omitted.

Not sure why LLM would make Facebook (ads), Apple (hardware), Amazon (hosting, retail), Netflix (tv) obsolete. It's definitely something Google needs to think about, but there's no reason to think they won't again be the leader soon.

Imagine Walmart launching a ChatGPT interfaced bot for shopping that customers take a liking to. Walmart starts acquiring both new customers as well as high quality data they can use for RLHF for shopping. Eventually Walmart's data moat becomes so big, that Amazon retail cannot catch up and customers start leaving Amazon. For AWS, if MS starts giving discounts for OAI model usage to regular Azure customers, that's go…

> Imagine Walmart launching a ChatGPT interfaced bot for shopping that customers take a liking to.

I can't imagine that, because it doesn't seem to fit the use case. Especially not to the point of bankruptcy of Amazon, maybe as a small novelty? Can you list some killer features that the chat would bring that would make the existing shopping experience irrelevant? Maybe not everything is a nail to the hammer?

Re: What it feels like to work in AI right now

#100

Post by bagels »

tayo42 wrote: post #70
karmakurtisaani wrote: post #55

Earlier quoted context omitted.

A PhD student I recently talked to was complaining how in academia they will never have the resources it takes to train models like GPT. And how now some academics are only researching the input/outputs to these LLMs. Seems like it's pretty dire or at least uninspiring to all but very few at the moment.

why is that? don't like those academic particle researchers get tons of money to build wild experiments to measure waves or collide particles? unless i get the funding part wrong? are those not academic projects?

They should join Facebook or Google, get paid a ton, and do research there with the resources they need.
Post reply on HN