Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is forcing us to write good code

bits.logic.inc

Post reply on HN
Print view View original

11–20 of 229 posts

Re: AI is forcing us to write good code

#11

Post by sgk284 »

mrits wrote: post #6

Author should ask AI to write a small app with 100% code coverage that breaks in every path except what is covered in the tests.

I never claim that 100% coverage has anything to do with code breaking. The only claim made is that anything less than 100% does guarantee that some piece of code is not automatically exercised, which we don't allow.

It's a footnote on the post, but I expand on this with:

  100% coverage is actually the minimum bar we set. We encourage writing tests for as many scenarios as is possible, even if it means the same lines getting exercised multiple times. It gets us closer to 100% path coverage as well, though we don’t enforce (or measure) that

Re: AI is forcing us to write good code

#13

Post by thih9 »

mrits wrote: post #6

Author should ask AI to write a small app with 100% code coverage that breaks in every path except what is covered in the tests.

Example output if anyone else is curious:

    def fragile(x):
        lst = [None]
        lst[x - 42]
        return "ok"
    
    def test_fragile():
        assert fragile(42) == "ok"

Re: AI is forcing us to write good code

#14

Post by mrkeen »

iamjs wrote: post #9
badgersnake wrote: post #5

I’m increasingly finding that the type of engineer that blogs is not they type of engineer anyone should listen to.

I find that this idea of restricting degrees of freedom is absolutely critical to being productive with agents at scale. Please enlighten us as to why you think this is nonsense

Wearing seatbelts is critical for drunk-driving.

All praise drunk-driving for increased seatbelt use.

Re: AI is forcing us to write good code

#15

Post by mkozlows »

I like this. "Best practices" are always contingent on the particular constellation of technology out there; with tools that make it super-easy to write code, I can absolutely see 100% coverage paying off in a way that doesn't for human-written code -- it maximizes what LLMs are good at (cranking out code) while giving them easy targets to aim for with little judgement.

(A thing I think is under-explored is how much LLMs change where the value of tests are. Back in the artisan hand-crafted code days, unit tests were mostly useful as scaffolding: Almost all the value I got from them was during the writing of the code. If I'd deleted the unit tests before merging, I'd've gotten 90% of the value out of them. Whereas now, the AI doesn't necessarily need unit tests as scaffolding as much as I do, _but_ having them put in there makes future agentic interactions safer, because they act as reified context.)

Re: AI is forcing us to write good code

#16

Post by sgk284 »

jennyholzer3 wrote: post #10

I don't know about all this AI stuff. How are LLMs going to stay on top of new design concepts, new languages, really anything new? Can LLMs be trained to operate "fluently" with regards to a genuinely new concept? I think LLMs are good for writing certain types of "bad code", i.e. if you're learning a new language or trying to quickly create a prototype. However to me it seems like a security risk to try to write "g…

I suspect it will still fall on humans (with machine assistance?) to move the field forward and innovate, but in terms of training an LLM on genuinely new concepts, they tend to be pretty nimble on that front (in my experience).

Especially with the massive context windows modern LLMs have. The core idea that the GPT-3 paper introduced was (summarizing):

  A sufficiently large language model can perform new tasks it has never seen using only a few examples provided at inference time, without any gradient updates or fine-tuning.

Re: AI is forcing us to write good code

#18

Post by yoyohello13 »

badgersnake wrote: post #5

I’m increasingly finding that the type of engineer that blogs is not they type of engineer anyone should listen to.

The value of the blog post is negatively correlated to how good the site looks. Mailing list? Sponsors? Fancy Title? Garbage. Raw HTML dumped on a .xyz domain, Gold!

Re: AI is forcing us to write good code

#19

Post by reactordev »

sgk284 wrote: post #11
mrits wrote: post #6

Author should ask AI to write a small app with 100% code coverage that breaks in every path except what is covered in the tests.

I never claim that 100% coverage has anything to do with code breaking. The only claim made is that anything less than 100% does guarantee that some piece of code is not automatically exercised, which we don't allow. It's a footnote on the post, but I expand on this with: 100% coverage is actually the minimum bar we set. We encourage writing tests for as many scenarios as is possible, even if it means the same lines…

I feel this comment is lost on those who have never achieved it and gave up along the journey.
Post reply on HN