Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Build a Database in 3000 Lines with 0 Dependencies

build-your-own.org

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Build a Database in 3000 Lines with 0 Dependencies

#51

Post by necovek »

diggan wrote: post #49
iSnow wrote: post #46

Earlier quoted context omitted.

Happy that it's not just me. I tried it a couple of times, and for small problems, I could make it work, albeit with refactorings both to the code and tests. But for more complicated topics, I never fully grasped all the details before writing code, so my tests missed aspects and I had to refactor both code and tests. I kinda like the idea more than the reality of TDD.

I think in general people tend to overdo TDD if they do TDD, aiming for a 100% test coverage which just ends up doing what you and parent mentions, solidifies a design and makes it harder to change. If instead every test is well intentioned and focus on testing the public API of whatever you test, not making assumptions about the internal design, you can get well tested code that is also easy to change (assuming the…

It's extremely hard to really do TDD and get code that's hard to change. If you persevere with a design that's hard to change, every single change in your failing-test-fix-implementation TDD cycle will make you refactor all your tests, and you'll realise why the design is bad and reduce coupling instead.

What really happens is that people write code, write non-unit "unit" tests for 100% coverage, and then suffer because those non-unit tests are now dependent on more than just what you are trying to test, all of them have some duplication because of it, and any tiny change is now blocked by tests.

Re: Build a Database in 3000 Lines with 0 Dependencies

#52

Post by serbuvlad »

quantadev wrote: post #37
CSSer wrote: post #27

Earlier quoted context omitted.

This makes me wonder. Is anyone practicing TDD with genAI/LLMs? If the true value is in the tests, might as well write those and have the AI slop be the codebase itself. TDD is often criticized for being slow. I'd seriously like to compare one vs the other today. I've also heard people find it challenging to get it to write good tests.

I'd sort of invert that and say it's better to use LLMs to just generate tons more test cases for the SQL DBs. Theoretically we could use LLMs to create 100s of Thousands (unlimited really) of test cases for any SQL system, where you could pretty much certify the entire SQL capability. Maybe such a standardized test suite already exists, but it was probably written by humans.

Are you sure that LLMs, because of their probabilistic nature, would not bias against certain edge cases. Sure, LLMs can be used to great effect to write many tests for normal usage patterns, which is valuable for sure. But I'd still prefer my edge cases handled by humans where possible.

Re: Build a Database in 3000 Lines with 0 Dependencies

#53

Post by hitchstory »

diggan wrote: post #49
iSnow wrote: post #46

Earlier quoted context omitted.

Happy that it's not just me. I tried it a couple of times, and for small problems, I could make it work, albeit with refactorings both to the code and tests. But for more complicated topics, I never fully grasped all the details before writing code, so my tests missed aspects and I had to refactor both code and tests. I kinda like the idea more than the reality of TDD.

I think in general people tend to overdo TDD if they do TDD, aiming for a 100% test coverage which just ends up doing what you and parent mentions, solidifies a design and makes it harder to change. If instead every test is well intentioned and focus on testing the public API of whatever you test, not making assumptions about the internal design, you can get well tested code that is also easy to change (assuming the…

You can get 100% coverage by focusing on testing the public API too. These two things are completely orthogonal.

Re: Build a Database in 3000 Lines with 0 Dependencies

#54

Post by quantadev »

serbuvlad wrote: post #52
quantadev wrote: post #37

Earlier quoted context omitted.

I'd sort of invert that and say it's better to use LLMs to just generate tons more test cases for the SQL DBs. Theoretically we could use LLMs to create 100s of Thousands (unlimited really) of test cases for any SQL system, where you could pretty much certify the entire SQL capability. Maybe such a standardized test suite already exists, but it was probably written by humans.

Are you sure that LLMs, because of their probabilistic nature, would not bias against certain edge cases. Sure, LLMs can be used to great effect to write many tests for normal usage patterns, which is valuable for sure. But I'd still prefer my edge cases handled by humans where possible.

I'm not sure if LLMs would do better or worse at edge cases, but I agree humans WOULD need to study the edge case tests, like you said. Very good point. Interestingly though LLMs might help identify more edge cases us humans didn't see.

Re: Build a Database in 3000 Lines with 0 Dependencies

#55

Post by quantadev »

diggan wrote: post #48
quantadev wrote: post #37

Earlier quoted context omitted.

I'd sort of invert that and say it's better to use LLMs to just generate tons more test cases for the SQL DBs. Theoretically we could use LLMs to create 100s of Thousands (unlimited really) of test cases for any SQL system, where you could pretty much certify the entire SQL capability. Maybe such a standardized test suite already exists, but it was probably written by humans.

At that point, you'd get a ton more value from doing Property Testing (+ get up and running faster, with less costs). If I'd had to have either code or tests generated by a LLM, I'd manually write the test cases with a well-thought out API for whatever I test, then have the LLM write tests that implements what I thought up, rather than the opposite which sounds like a slow and painful death.

I hadn't heard of "Property Testing" if that's a sort of term of art. I'll look into it. Anyway, yeah in TDD it's hard to say which part deserves more human scrutiny the tests or the implementations.

Re: Build a Database in 3000 Lines with 0 Dependencies

#56

Post by wwarren »

miffy900 wrote: post #43
wwarren wrote: post #24

Man I got the first edition of this book and it was so bad. Hopefully this is better…

Can you elaborate on why it was so bad?

Check the reviews online, but generally random bits of Go code are presented with no build-up or explanation. B Trees are implemented immediately and without explanation. The book is incredibly short and seems just like a loose collection of notes, rather than an actual book. Very interested to check out the new version

Re: Build a Database in 3000 Lines with 0 Dependencies

#57

Post by vrighter »

CSSer wrote: post #27
cryptonector wrote: post #23

Earlier quoted context omitted.

Oh no, SQLite3 is a lot more featureful than SQLite2. The proprietary test suite is what makes SQLite3 so solid.

This makes me wonder. Is anyone practicing TDD with genAI/LLMs? If the true value is in the tests, might as well write those and have the AI slop be the codebase itself. TDD is often criticized for being slow. I'd seriously like to compare one vs the other today. I've also heard people find it challenging to get it to write good tests.

dude, if you have the llm write the tests, then you have no confidence it's testing what you think it is. Making the test worthless

Re: Build a Database in 3000 Lines with 0 Dependencies

#58

Post by CSSer »

vrighter wrote: post #57
CSSer wrote: post #27

Earlier quoted context omitted.

This makes me wonder. Is anyone practicing TDD with genAI/LLMs? If the true value is in the tests, might as well write those and have the AI slop be the codebase itself. TDD is often criticized for being slow. I'd seriously like to compare one vs the other today. I've also heard people find it challenging to get it to write good tests.

dude, if you have the llm write the tests, then you have no confidence it's testing what you think it is. Making the test worthless

Dude, I was suggesting that you, not the LLM, write the tests in this scenario.
Post reply on HN