Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is not a coworker, it's an exoskeleton

kasava.dev

Post reply on HN
Print view View original

161–170 of 608 posts

Re: AI is not a coworker, it's an exoskeleton

#161

Post by moron4hire »

jacquesm wrote: post #39
oxag3n wrote: post #32

> We're thinking about AI wrong. And this write up is not an exception. Why even bother thinking about AI, when Anthropic and OpenAI CEOs openly tell us what they want (quote from recent Dwarkesh interview) - "Then further down the spectrum, there’s 90% less demand for SWEs, which I think will happen but this is a spectrum." So save thinking and listen to intent - replace 90% of SWEs in near future (6-12 months accor…

Not without some major breakthrough. What's hilarious is that all these developers building the tools are going to be the first to be without jobs. Their kids will be ecstatic: "Tell me again, dad, so, you had this awesome and well paying easy job and you wrecked it? Shut up kid, and tuck in that flap, there is too much wind in our cardboard box."

"Well son, we made a lot of shareholder value."

Re: AI is not a coworker, it's an exoskeleton

#162

Post by GeoAtreides »

hintymad wrote: post #65

In the latest interview with Claude Code's author: https://podcasts.apple.com/us/podcast/lennys-podcast-product... , Boris said that writing code is a solved problem. This brings me to a hypothetical question: what if engineers stop contributing to open source, in which case would AI still be powerful enough to learn the knowledge of software development in the future? Or is the field of computer science plateaued to…

>writing code is a solved problem

sure is news for the models tripping on my thousands of LOC jquery legacy app...

Re: AI is not a coworker, it's an exoskeleton

#164

Post by dwohnitmok »

runarberg wrote: post #49
dwohnitmok wrote: post #22

Earlier quoted context omitted.

Not anymore. This benchmark is for LLM chess ability: https://github.com/lightnesscaster/Chess-LLM-Benchmark?tab=r... . LLMs are graded according to FIDE rules so e.g. two illegal moves in a game leads to an immediate loss. This benchmark doesn't have the latest models from the last two months, but Gemini 3 (with no tools) is already at 1750 - 1800 FIDE, which is approximately probably around 1900 - 2000 USCF (about…

Wait, I may be missing something here. These benchmarks are gathered by having models play each other, and the second illegal move forfeits the game. This seems like a flawed method as the models who are more prone to illegal moves are going to bump the ratings of the models who are less likely. Additionally, how do we know the model isn’t benchmaxxed to eliminate illegal moves. For example, here is the list of games…

The LLMs do play rated engines (maia and eubos). They provide the baselines. Gemini e.g. consistently beats the different maia versions.

The rest is taken care of by elo. That is they then play each other as well, but it is not really possible for Gemini to have a higher elo than maia with such a small sample size (and such weak other LLMs).

Elo doesn't let you inflate your score by playing low ranked opponents if there are known baselines (rated engines) because the rated engines will promptly crush your elo.

You could add humans into the mix, the benchmark just gets expensive.

Re: AI is not a coworker, it's an exoskeleton

#165

Post by dwohnitmok »

emp17344 wrote: post #75
runarberg wrote: post #49

Earlier quoted context omitted.

Wait, I may be missing something here. These benchmarks are gathered by having models play each other, and the second illegal move forfeits the game. This seems like a flawed method as the models who are more prone to illegal moves are going to bump the ratings of the models who are less likely. Additionally, how do we know the model isn’t benchmaxxed to eliminate illegal moves. For example, here is the list of games…

That’s a devastating benchmark design flaw. Sick of these bullshit benchmarks designed solely to hype AI. AI boosters turn around and use them as ammo, despite not understanding them.

> That’s a devastating benchmark design flaw

I think parent simply missed until their later reply that the benchmark includes rated engines.

Re: AI is not a coworker, it's an exoskeleton

#166

Post by paulryanrogers »

alphazard wrote: post #158

There's an undertone of self-soothing "AI will leverage me, not replace me", which I don't agree with especially in the long run, at least in software. In the end it will be the users sculpting formal systems like playdoh. In the medium run, "AI is not a co-worker" is exactly right. The idea of a co-worker will go away. Human collaboration on software is fundamentally inefficient. We pay huge communication/synchroniz…

This assumes every individual is capable of succinctly communicating to the AI what they want. And the AI is capable of maintaining it as underlying platforms and libraries shift.

And that there is little value in reusing software initiated by others.

Re: AI is not a coworker, it's an exoskeleton

#167

Post by dwohnitmok »

cesarvarela wrote: post #48
dwohnitmok wrote: post #22

Earlier quoted context omitted.

Not anymore. This benchmark is for LLM chess ability: https://github.com/lightnesscaster/Chess-LLM-Benchmark?tab=r... . LLMs are graded according to FIDE rules so e.g. two illegal moves in a game leads to an immediate loss. This benchmark doesn't have the latest models from the last two months, but Gemini 3 (with no tools) is already at 1750 - 1800 FIDE, which is approximately probably around 1900 - 2000 USCF (about…

Yeah, but 1800 FIDE players don't make illegal moves, and Gemini does.

1800 FIDE players do make illegal moves. I believe they make about one to two orders of magnitude less illegal moves than Gemini 3 does here. IIRC the usual statistic for expert chess play is about 0.02% of expert chess games have an illegal move (I can look that up later if there's interest to be sure), but that is only the ones that made it into the final game notation (and weren't e.g. corrected at the board by an opponent or arbiter). So that should be a lower bound (hence why it could be up to one order lower, although I suspect two orders is still probably closer to the truth).

Whether or not we'll see LLMs continue to get a lower error rate to make up for those orders of magnitude remains to be seen (I could see it go either way in the next two years based on the current rate of progress).

Re: AI is not a coworker, it's an exoskeleton

#168

Post by [deleted] »

hintymad wrote: post #65

In the latest interview with Claude Code's author: https://podcasts.apple.com/us/podcast/lennys-podcast-product... , Boris said that writing code is a solved problem. This brings me to a hypothetical question: what if engineers stop contributing to open source, in which case would AI still be powerful enough to learn the knowledge of software development in the future? Or is the field of computer science plateaued to…

[deleted]

Re: AI is not a coworker, it's an exoskeleton

#169

Post by paulryanrogers »

cheema33 wrote: post #146
hintymad wrote: post #65

In the latest interview with Claude Code's author: https://podcasts.apple.com/us/podcast/lennys-podcast-product... , Boris said that writing code is a solved problem. This brings me to a hypothetical question: what if engineers stop contributing to open source, in which case would AI still be powerful enough to learn the knowledge of software development in the future? Or is the field of computer science plateaued to…

> is the field of computer science plateaued to the point that most of what we do is linear combination of well established patterns? Computer science is different from writing business software to solve business problems. I think Boris was talking about the second and not the first. And I personally think he is mostly correct. At least for my organization. It is very rare for us to write any code by hand anymore. On…

> Not everybody's got these bits figured out. They stumble around and them blame the tools for their failures.

Possible. Yet that's a pretty broad brush. It could also be that some businesses are more heavily represented in the training set. Or some combo of all the above.

Re: AI is not a coworker, it's an exoskeleton

#170

Post by dwohnitmok »

recursive wrote: post #128
famouswaffles wrote: post #100

Earlier quoted context omitted.

That benchmark methodology isn't great, but regardless, LLMs can be trained to play Chess with a 99.8% legal move rate.

That doesn't exactly sound like strong chess play.

It's enough to reliably beat amateur (e.g. maia-1900) chess engines.
Post reply on HN