Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is different

antirez.com

Post reply on HN
Print view View original

771–780 of 920 posts

Re: AI is different

#771

Post by dmead »

app134 wrote: post #761
dmead wrote: post #699

Earlier quoted context omitted.

I had such a professor as well, but those people used to use the more accurate term "machine learning". There was also wide understanding that those architectures were trying to imitate small bits of what we understood was happening in the brain (see marvin minsky's perceptron etc). The hope was, as I understood it that there would be some breakthrough in neuroscience that would let the computer scientists pick up th…

You asked earlier if you were being overly cynical, and I think the answer to that is "yes" We are indeed simulating what we find in nature when we create neural networks and transformers, and AI companies are indeed investing heavily in BCI research. ChatGPT can write an original essay better than most of my students. Its also artificial. Is that not artificial intelligence?

It is not intelligent.

Hiding the training data behind gradient descent and then making attributions to the program that responds using this model is certainly artificial though.

This analogy just isn't holding water.

Re: AI is different

#772

Post by analog31 »

gdubs wrote: post #622

AI has been improving at a very rapid pace, which means that a lot of people have really outdated priors. I see this all the time online where people are dismissive about AI in a way that suggests it's been a while since they last checked-in on the capabilities of models. They wrote off the coding ability of ChatGPT on version 3.5, for instance, and have missed all the advancements that have happened since. Or they t…

There's a middle ground which is to watch and see what happens around us. Is it unholy to not have an opinion?

Re: AI is different

#773

Post by iLoveOncall »

andai wrote: post #583
iLoveOncall wrote: post #177

> However, if AI avoids plateauing long enough I'm not sure how someone can seriously write this after the release of GPT-5. Models have started to plateau since ChatGPT came out (3 years ago) and GPT-5 has been the final nail in this coffin.

o3 was actually GPT-5. They just gave it a stupid name, and made it impractical for general usage. But in terms of wow factor, it was a step change on the order of GPT-3 -> GPT-4. So now they're stuck slapping the GPT-5 label on marginal improvements because it's too awkward to wait for the next breakthrough now. On that note, o4-mini was much better for general usage (speed and cost). It was my go-to for web search…

I have acid reflux every time I see the term "step change" used to talk about a model change. There hasn't been any model that has been a "step change" over its predecessor.

It's much more like each new model climbs another step of the ladder that goes up the step, and so far we can't even see the top of the ladder.

My suspicion is also that the ladder actually ends way before it reaches the next step, and LLMs are a dead end. Everything indicates it so far.

Let's not even talk about "reasoning models", aka spend twice the tokens and twice the time on the same answer.

Re: AI is different

#774

Post by cobbzilla »

IX-103 wrote: post #764
cobbzilla wrote: post #564

Earlier quoted context omitted.

no matter what you ask AI to do, it’s going to give you an “average“ answer. Even if you tell it to use a very distinct specific voice and write in a very specific tone, it’s going to give you the “average“ specific voice and tone you’ve asked for. AI is the antithesis of creativity and originality. This gives me hope.

That's mostly true of humans though. They almost always give average answers. That works out because 1) most of the work that needs to be done is repetitive, not new so average answers are okay 2) the solution space that has been explored by humans is not convex, so average answers will still hit unexplored territory most of the time

Absolutely! You can communicate with without (or with minimal) creativity. It’s not required in most cases. So AI is definitely very useful, and it can ape creativity better and better, but it will always be “faking it”.

Re: AI is different

#775

Post by loandbehold »

libraryofbabel wrote: post #763
bigstrat2003 wrote: post #734

Earlier quoted context omitted.

I don't understand people who say this. My knee jerk reaction (which I rein in because it's incredibly rude) is always "wow, that person must really suck at programming then". And I try to hold to the conviction that there's another explanation. For me, the vast, vast majority of the time I try to use it, AI slows my work down, it doesn't speed it up. As a result it's incredibly difficult to understand where these su…

Usually the "10x" improvements come from greenfield projects or at least smaller codebases. Productivity improvements on mature complex codebases are much more modest, more like 1.2x. If you really in good faith want to understand where people are coming from when they talk about huge productivity gains, then I would recommend installing Claude Code (specifically that tool) and asking it to build some kind of small p…

I was able to realize huge productivity gains working on a 20 years old codebase with 2+ million loc, as I mentioned in the sister post. So I disagree that big productivity gains are only on greenfield projects. Realizing productivity gains on mature code based requires more skill and upfront setup. You need to put some work in your claude.md and give Claude tools for accessing necessary data, logs, build process. It should be able to test your code autonomously as much as possible. In my experience, people who say they are not able to realize productivity gains don't put enough effort to understand these new tools and setup them properly for their project.

Re: AI is different

#776

Post by nuancebydefault »

IX-103 wrote: post #750
libraryofbabel wrote: post #697

Earlier quoted context omitted.

I do see this a lot. It's hard to have a reasonable conversation about AI amidst, on the one hand, hype-mongers and boosters talking about how we'll have AGI in 2027 and all jobs are just about to be automated away, and on the other hand, a chorus of people who hate AI so much they have invested their identify in it failing and haven't really updated their priors since ChatGPT came out. Both groups repeat the same se…

> But they're not just "stochastic parrots." They can model the world and reason about it, albeit imperfectly and not like humans do. I've not seen anything from a model to persuade me they're not just stochastic parrots. Maybe I just have higher expectations of stochastic parrots than you do. I agree with you that AI will have a big impact. We're talking about somewhere between "invention of the internet" and "inven…

Stochastic parrot here (or not?). Can you tell the difference?

Re: AI is different

#777

Post by kelseyfrog »

semi-extrinsic wrote: post #762
kelseyfrog wrote: post #754

Earlier quoted context omitted.

There's three important beliefs at play in the A(G)I story: 1. When(if) AGI will arrive. It's likely going to be smeared out over a couple months to years, but relative to everything else, it's a historical blip. This really is the most contention belief with the most variability. It is currently predicted to be 8 years[1]. 2. What percentage of jobs will be replaceable with AGI? Current estimates between 80-95% of p…

Your only reference [1] is to a page where anybody in the world can join and vote. It literally means absolutely nothing. For [2] you have no reference whatsoever. How does AI replace a nurse, a vet, a teacher, a construction worker?

What are you talking about? This is common knowledge.

Median forecasts indicated a 50% probability of AI systems being capable of automating 90% of current human tasks in 25 years and 99% of current human tasks in 50 years[1]

The scope of work replaceable by embodied AGI and the speed of AGI saturation of vastly under estimated. The bottle necks are production of a replacement workforce, not retraining human laborers.

1. https://arxiv.org/pdf/1901.08579

Re: AI is different

#778

Post by pm »

libraryofbabel wrote: post #755
IX-103 wrote: post #750

Earlier quoted context omitted.

> But they're not just "stochastic parrots." They can model the world and reason about it, albeit imperfectly and not like humans do. I've not seen anything from a model to persuade me they're not just stochastic parrots. Maybe I just have higher expectations of stochastic parrots than you do. I agree with you that AI will have a big impact. We're talking about somewhere between "invention of the internet" and "inven…

What is your definition of "stochastic parrot"? Mine is something along the lines of "produces probabilistic completions of language/tokens without having any meaningful internal representation of the concepts underlying the language/tokens." Early LLMs were like that. That's not what they are now. An LLM got Gold on the Mathematical Olympiad - very difficult math problems that it hadn't seen in advance. You don't do…

Ignoring its negative connotation, it's more likely to be a highly advanced "stochastic parrot".

> "You don't do that without some kind of working internal model of mathematics."

This is speculation at best. Models are black boxes, even to those who make them. We can't discern a "meaningful internal representation" in a model, anymore than a human brain.

> "There is just no way you can get to the right answer by spouting out plausible-sounding sentence completions without understanding what they mean."

You've just anthropomorphised a stochastic machine, and this behaviour is far more concerning, because it implies we're special, and we're not. We're just highly advanced "stochastic parrots" with a game loop.

Re: AI is different

#779

Post by suddenlybananas »

pixl97 wrote: post #724
layer8 wrote: post #495

Earlier quoted context omitted.

Believing a thing is possible doesn’t by itself make it so, however.

This is kind of weird. It's like saying "Driving in snow is impossible", well we know it is possible because humans do it. And this even ignores all the things modern computer controlled vehicles do above and beyond humans as it is. Take most people used to driving modern cars and chunk them an old armstrong steering car and they'll put themselves into a ditch on a rainy day. Really the last things in self driving ca…

Ah we only need general intelligence, something so ineffable and hard to understand that we don't even have a clear definition of it.

Re: AI is different

#780

Post by saltcured »

barrell wrote: post #759
libraryofbabel wrote: post #740

Earlier quoted context omitted.

There’s really three points mixed up in here. 1) LLMs are controlled by BigCorps who don’t have user’s best interests at heart. 2) I don’t like LLMs and don’t use them because they spoil my feeling of craftsmanship. 3) LLMs can’t be useful to anyone because I “kick the tires” every so often and am underwhelmed. (But what did you actually try? Do tell.) #1 is obviously true and is a problem, but it’s just capitalism.…

How exactly am I betting my career on LLMs failing? The inverse is definitely true — going all in on LLMs feels like betting on the future success of LLMs. However not using LLMs to program today is not betting on anything, except maybe myself, but even that’s a stretch. After all, I can always pick up LLMs in the future. If a few weeks is long enough for all my priors to become stale, why should I have to start now?…

I wonder, what drives this intense FOMO ideation about AI tools as expressed further upthread?

How does someone reconcile a faith that AI tooling is rapdily improving with that contradictory belief that there is some permanent early-adopter benefit?

Post reply on HN