Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI writes your code, why use Python?

medium.com

Post reply on HN
Print view View original

511–520 of 1001 posts

Re: If AI writes your code, why use Python?

#511

Post by andai »

riedel wrote: post #420
gertlabs wrote: post #174

Earlier quoted context omitted.

Surprisingly, LLMs are actually much worse at reasoning in Python than other common programming languages for agentic coding tasks. Data here: https://gertlabs.com/rankings?mode=agentic_coding

My feeling is that for agentic tasks this is not only language design but also LSPs, error messages and static analysis capabilities that dominate the benchmarks. It would IMHO be interesting to look into better subsets of python and style/rewrite techniques as well as alternative linter and their effects on performance.

But then why does JS score 50% better? (Almost identical to TypeScript.)

Actually, JS can get a surprising amount of "intellisense" as well. Not sure if that was used here though.

Re: If AI writes your code, why use Python?

#512

Post by librasteve »

isityettime wrote: post #237
gertlabs wrote: post #174

Earlier quoted context omitted.

Surprisingly, LLMs are actually much worse at reasoning in Python than other common programming languages for agentic coding tasks. Data here: https://gertlabs.com/rankings?mode=agentic_coding

I would love to see how they do with functional languages and especially Lisps here. I've noticed pretty good performance with Emacs Lisp relative to overall model strength, but I haven't used LLMs to application code in any such languages. It would also be interesting to see how Python compares to other languages in its niche (Ruby, Perl, Raku). Thanks for putting this together! It's interesting.

I just did a side-by-side with Claude Code Python vs. Raku for DSL use ... https://slangify.org if you are interested.

Re: If AI writes your code, why use Python?

#513

Post by andai »

kevinautumn wrote: post #438
riedel wrote: post #420

Earlier quoted context omitted.

My feeling is that for agentic tasks this is not only language design but also LSPs, error messages and static analysis capabilities that dominate the benchmarks. It would IMHO be interesting to look into better subsets of python and style/rewrite techniques as well as alternative linter and their effects on performance.

A strict compiler is basically a free feedback loop for the LLM.

Also the human. (I like being told about my bugs when I write them, instead of at some generally much more unpleasant moment in the future.)

Re: If AI writes your code, why use Python?

#514

Post by morningsam »

bensyverson wrote: post #144
_boffin_ wrote: post #80

Read the first few comments and surprised I didn’t see it, but training data. The voluminous amount of Python in the training data. I could write in brainfuck with ai, but I presume, wouldn’t get the same results than if going with python. My follow up question: with AI now, why care about a lang until you need to?

Just use Go. LLMs have seen a ton of it, they write it well, it compiles practically instantly, and it has all the advantages of a typed compiled language. I created a big Python codebase using AI, and the LLM constantly guesses arguments or dictionary formats wrong. Unit tests and stuff like pydantic help, but it's better to avoid that whole class of runtime errors altogether.

>the LLM constantly guesses arguments or dictionary formats wrong [...] it's better to avoid that whole class of runtime errors altogether.

Use Mypy in strict mode and run it in the post-turn hook of your LLM harness so the LLM has no choice but to obey it. And don't use overly general dictionary types when the keys are known at development time; use TypedDicts for annotations if you must use dicts at runtime.

Re: If AI writes your code, why use Python?

#515

Post by librasteve »

BariumBlue wrote: post #210
gertlabs wrote: post #174

Earlier quoted context omitted.

Surprisingly, LLMs are actually much worse at reasoning in Python than other common programming languages for agentic coding tasks. Data here: https://gertlabs.com/rankings?mode=agentic_coding

Hah, I was just thinking that Python likely has a vast ocean of training data, but it's likely of lower quality, being much of it is written by beginners and those who aren't primarily programmers.

I was (pleasantly) surprised by Claude Code doing Raku - also with a limited training set (~2000 Stack Overflow, a bunch of Rosetta, 2,500 modules). I put this down to the quality of the code for the core community who are all frankly uber-gremlins.

Re: If AI writes your code, why use Python?

#517

Post by twelvechairs »

bozdemir wrote: post #509

Yea I take the step a bit further, why bother Rust ? Just go write assembly or better the executable bytes... You see ? Readability is very important :)

Surely its the same hierarchy as before. For most complex things you start with high level to get something running quickly then move towards low level when you bed down the spec and need more safety, error reporting, speed etc.

Re: If AI writes your code, why use Python?

#519

Post by aaa_aaa »

_boffin_ wrote: post #80

Read the first few comments and surprised I didn’t see it, but training data. The voluminous amount of Python in the training data. I could write in brainfuck with ai, but I presume, wouldn’t get the same results than if going with python. My follow up question: with AI now, why care about a lang until you need to?

For some people reducing infra costs matter. Python is very very slow, even if it uses native libs.
Post reply on HN