I know it, it was a scary period for programmers. The tide is turning. Meatsuits are back in the game.
Deep Learning, Deep Scandal
11–20 of 23 posts
Re: Deep Learning, Deep Scandal
#12One of the comments in the article says: "I don't see how it's not a net negative tech," to which Marcus replies: "That’s my current tentative conclusion, yes."
What is the negative effect I'm not seeing? Bad code? Economic waste in datacenter investment? Wasted effort of researchers who could be solving other problems?
I've been writing software for over a decade, and I’ve never been as productive as I am now. Jumping into a new codebase - even in unfamiliar areas like a React frontend - is so much easier. I’m routinely contributing to frontend projects, which I never did before.
There is some discipline required to avoid the temptation to just push AI-generated code, but otherwise, it works like magic.
Re: Deep Learning, Deep Scandal
#13GPT-1 to GPT-2: June 2018 to February 2019 = 8 months.
GPT-2 to GPT-3: February 2019 to June 2020 = 16 months.
GPT-3 to GPT-4: June 2020 to March 2023 = 33 months.
Looks like time to get to the next level is doubling. So we can expect GPT-5 sometime June 2028.
Feels like people are being premature about claiming AI winter or that it is somehow a scandal that we don't already have GPT-5.
It's going to take time. We need some more patience in this industry.
Re: Deep Learning, Deep Scandal
#14One of the comments in the article says: "I don't see how it's not a net negative tech," to which Marcus replies: "That’s my current tentative conclusion, yes." What is the negative effect I'm not seeing? Bad code? Economic waste in datacenter investment? Wasted effort of researchers who could be solving other problems? I've been writing software for over a decade, and I’ve never been as productive as I am now. Jumpi…
I assumed it was a reference to the nonsensical memes of the AI compute resources causing damage to the environment, using up water and electricity etc.
Re: Deep Learning, Deep Scandal
#15One of the comments in the article says: "I don't see how it's not a net negative tech," to which Marcus replies: "That’s my current tentative conclusion, yes." What is the negative effect I'm not seeing? Bad code? Economic waste in datacenter investment? Wasted effort of researchers who could be solving other problems? I've been writing software for over a decade, and I’ve never been as productive as I am now. Jumpi…
I’ve been playing a lot with Claude Code recently and also making my first significant foray into front end development, and I think that LLMs are the tech that have finally made front end development broadly accessible.
Re: Deep Learning, Deep Scandal
#16The technology is just a couple years old, and this article is derived from a couple months of evidence. We can't yet say what the future holds. The nay Sayers who were so confident that LLMs were stochastic parrots are now embarrassingly wrong. This article sounds like that. Whether we are actually at a dead end or not is unknown. Why are people talking with such utter conviction when nobody truly understands what's…
According to AI, the technology dates back to 1763.
Re: Deep Learning, Deep Scandal
#17As I said many times, I have been in the game for 30 years. I started doing AI with rules as early as the beginning of the 90 and I never...never expected to see anything like LLMs in my lifetime.
When I read Marcus once again saying that: yes this time LLM have reached their limit, which he has been saying for 2 years in a row, I'm really feeling tired of his tune. The idea that LLM are a dead end, a failing technology is pretty weird.
Compared to what???
I use LLM everyday in my work, to write summaries, to make translations, to generate some code or to get explanations about a given code... And I even use them as research sparring partners to see how I could improve my work...
Gary Marcus has been involved in the domain for 30 years as well... Where is his technology that would match or surpass the LLM???
Re: Deep Learning, Deep Scandal
#18GPT-1 to GPT-2: June 2018 to February 2019 = 8 months. GPT-2 to GPT-3: February 2019 to June 2020 = 16 months. GPT-3 to GPT-4: June 2020 to March 2023 = 33 months. Looks like time to get to the next level is doubling. So we can expect GPT-5 sometime June 2028. Feels like people are being premature about claiming AI winter or that it is somehow a scandal that we don't already have GPT-5. It's going to take time. We ne…
Considering the capital burn rate here, I suspect that waiting years between iterations will be a hard pill to swallow for investors.
Re: Deep Learning, Deep Scandal
#19GPT-1 to GPT-2: June 2018 to February 2019 = 8 months. GPT-2 to GPT-3: February 2019 to June 2020 = 16 months. GPT-3 to GPT-4: June 2020 to March 2023 = 33 months. Looks like time to get to the next level is doubling. So we can expect GPT-5 sometime June 2028. Feels like people are being premature about claiming AI winter or that it is somehow a scandal that we don't already have GPT-5. It's going to take time. We ne…
This is overlooking the massive capital that's been invested in the part of the 33 months...
Which alters the doubling timeline significantly.
Re: Deep Learning, Deep Scandal
#20The technology is just a couple years old, and this article is derived from a couple months of evidence. We can't yet say what the future holds. The nay Sayers who were so confident that LLMs were stochastic parrots are now embarrassingly wrong. This article sounds like that. Whether we are actually at a dead end or not is unknown. Why are people talking with such utter conviction when nobody truly understands what's…
According to AI, the technology dates back to 1763.
The technology that allows AI to lie and hallucinate is only a couple years old at best.