Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Property-Based Testing Caught a Security Bug I Never Would Have Found

kiro.dev

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#21

Post by regular_trash »

koakuma-chan wrote: post #20
kittoes wrote: post #19

Earlier quoted context omitted.

This just can't be your answer to everything... the article clearly stated that they're developing a client application for browsers. Rust advocates like yourself are really doing more harm than good by ignoring real world constraints.

You can use Rust to develop client application for browsers. Check out dioxus.

Rust is an unergonomic language that slows development in the general case (because it has lots of arcane syntax and rules, and people generally don't know it). Suggesting it as the "obvious" choice ignores the tradeoffs that come with adopting it for a project.

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#22

Post by koakuma-chan »

regular_trash wrote: post #21
koakuma-chan wrote: post #20

Earlier quoted context omitted.

You can use Rust to develop client application for browsers. Check out dioxus.

Rust is an unergonomic language that slows development in the general case (because it has lots of arcane syntax and rules, and people generally don't know it). Suggesting it as the "obvious" choice ignores the tradeoffs that come with adopting it for a project.

You can just hire people who know Rust. One Rust developer would likely end up more productive than ten Cursor baby sitters in the long run, and you would actually get a high quality app.

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#24

Post by bluGill »

philipwhiuk wrote: post #5

> We write unit tests for the happy path, maybe a few edge cases we can imagine, but what about the inputs we'd never consider? Many times we assume that LLMs are handling these scenarios by default, Do we?

The original claim for TDD is your write tests for all your edge cases. It doesn't matter about inputs you didn't consider because they are covered in the edge. If you can only accept inputs from 2-7 (inclusive) you check 1,2,7,8 - if those pass you assume the rest work.

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#25

Post by nslog »

mananaysiempre wrote: post #2

TL;DR: obj[key] with user-controlled key == "__proto__" is a gift that keeps on giving; buy our AI tool that will write subtle vulnerabilities like that which you yourself won’t catch in review but then it will also write some property-based tests that maybe will

Didn't react just have basically the same vuln

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#26

Post by hansvm »

kittoes wrote: post #19
koakuma-chan wrote: post #18

This kind of bug would never have happened if you used a proper programming language like Rust.

This just can't be your answer to everything... the article clearly stated that they're developing a client application for browsers. Rust advocates like yourself are really doing more harm than good by ignoring real world constraints.

To be fair, this particular issue wouldn't have happened in C, Python, Forth, Zig, or a host of other languages. String-based weirdness is something of a JS issue.

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#27

Post by wredcoll »

koakuma-chan wrote: post #22
regular_trash wrote: post #21

Earlier quoted context omitted.

Rust is an unergonomic language that slows development in the general case (because it has lots of arcane syntax and rules, and people generally don't know it). Suggesting it as the "obvious" choice ignores the tradeoffs that come with adopting it for a project.

You can just hire people who know Rust. One Rust developer would likely end up more productive than ten Cursor baby sitters in the long run, and you would actually get a high quality app.

This is like some kind of psy-op to get people to hate rust.

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#28

Post by cogman10 »

hansvm wrote: post #26
kittoes wrote: post #19

Earlier quoted context omitted.

This just can't be your answer to everything... the article clearly stated that they're developing a client application for browsers. Rust advocates like yourself are really doing more harm than good by ignoring real world constraints.

To be fair, this particular issue wouldn't have happened in C, Python, Forth, Zig, or a host of other languages. String-based weirdness is something of a JS issue.

This particular issue looks to be pretty uniquely a javascript problem. I don't even think hyper flexible languages like Ruby would ultimately experience this sort of problem.

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#29

Post by koakuma-chan »

wredcoll wrote: post #27
koakuma-chan wrote: post #22

Earlier quoted context omitted.

You can just hire people who know Rust. One Rust developer would likely end up more productive than ten Cursor baby sitters in the long run, and you would actually get a high quality app.

This is like some kind of psy-op to get people to hate rust.

Sure. Then good companies will be able to filter out people by only hiring those who know Rust.

Re: Property-Based Testing Caught a Security Bug I Never Would Have Found

#30

Post by Cpoll »

bluGill wrote: post #24
philipwhiuk wrote: post #5

> We write unit tests for the happy path, maybe a few edge cases we can imagine, but what about the inputs we'd never consider? Many times we assume that LLMs are handling these scenarios by default, Do we?

The original claim for TDD is your write tests for all your edge cases. It doesn't matter about inputs you didn't consider because they are covered in the edge. If you can only accept inputs from 2-7 (inclusive) you check 1,2,7,8 - if those pass you assume the rest work.

You forgot 0, -1, null, "1".
Post reply on HN