Just write a test for it • Hacker News Board

Re: Just write a test for it

#21

Post by karparov » Sun, Mar 30, 2025, 9:53 AM UTC

> This wasn’t caught by the existing test suite (even though it runs almost 200 end-to-end tests), because it always starts from an empty database, applies all migrations and only then runs the test code.

Isn't that where the test coverage has a hole?

I somehow expected the blog post to extend testing for this. A pre-populated database which is then migrated. That seems to catch a wider class of issues than parsing sql and shielding against just checking for non-null without default.

Re: Just write a test for it

#22

Post by karparov » Sun, Mar 30, 2025, 9:55 AM UTC

kubb wrote: post #10

kleiba wrote: post #5

This is a whole ̶l̶o̶n̶g̶ blog post about someone who was surprised that he could use a crate to parse SQL queries in Rust. A bit of an underwhelming read to me personally.

It’s actually a great reminder that a good framing and sales pitch matter a whole lot. The author isn’t just recounting their story, they’re selling it as a lesson in a generic approach that will make you a better engineer. Whether they can deliver on that promise doesn’t matter. It’s the feeling that’s clickable.

Tbf, it's not their fault it made it to the HN front page.

Are we going to criticise every little innocent blog post just because somebody liked it, submitted it to HN and it got enough upvotes?

Re: Just write a test for it

#23

Post by karparov » Sun, Mar 30, 2025, 9:56 AM UTC

wesselbindt wrote: post #19

nikolayasdf123 wrote: post #16

just look at this test for this one-liner DDL. the test is way more complex than thing it tests in first place. such confusing and hard to write and read tests is the reason people avoid writing tests in the first place. make tests great again! (great = simple, short, easy to write, read, maintain. at very least no more complex than the thing it testing!)

Totally with you on the merits of simplicity, writability, readability. But I'm getting strong "rest of the owl" vibes. How would you have prevented this defect instead, in a way that you find simple?

Pre-populate the db.

Re: Just write a test for it

#24

Post by disgruntledphd2 » Sun, Mar 30, 2025, 10:02 AM UTC

bravetraveler wrote: post #12

swombat wrote: post #8

Earlier quoted context omitted.

Not even that long, but I agree on the "underwhelming"... "Oh I found some niche issue that bothered me and wrote some code to fix it." -> HN Front Page

Virtue trifecta; Rust , testing, and "just"

It's a great title of which we all need reminding.

Re: Just write a test for it

#25

Post by IanCal » Sun, Mar 30, 2025, 10:07 AM UTC

ptx wrote: post #20

vlovich123 wrote: post #17

Earlier quoted context omitted.

The author correctly notes the challenge of correctly populating some pre-migration data at each step.

Does it need to be done at each step, though? Couldn't the test data be added at just one point (where the schema is known) and just let it run through all the subsequent migrations to see if it goes boom?

Ideally imo you'd have a property based test that does the following:

Perform some arbitrary list of valid actions. (This alone is valuable)

Run the new migrations.

Perform some arbitrary list of valid actions.

Assert no crashes/errors.

I've found this great for testing APIs - just "perform some list of user actions and make sure things don't explode" can catch a lot.

Re: Just write a test for it

#26

Post by maccard » Sun, Mar 30, 2025, 10:14 AM UTC

swombat wrote: post #8

kleiba wrote: post #5

This is a whole ̶l̶o̶n̶g̶ blog post about someone who was surprised that he could use a crate to parse SQL queries in Rust. A bit of an underwhelming read to me personally.

Not even that long, but I agree on the "underwhelming"... "Oh I found some niche issue that bothered me and wrote some code to fix it." -> HN Front Page

If it’s that simple where are your front page blog posts for fixing niche issues?

Re: Just write a test for it

#27

Post by vlovich123 » Sun, Mar 30, 2025, 10:21 AM UTC

ptx wrote: post #20

vlovich123 wrote: post #17

Earlier quoted context omitted.

The author correctly notes the challenge of correctly populating some pre-migration data at each step.

Does it need to be done at each step, though? Couldn't the test data be added at just one point (where the schema is known) and just let it run through all the subsequent migrations to see if it goes boom?

Not if you could have data that could only have been inserted at some stage to behind with and then a subsequent migration only breaks that case (eg create table in step 10 and step 11 breaks that new table - if you only had data that you inserted in step 1, it would still pass all migrations. In other words, you need to have sample data that exercises the entire DDL that is added or you risk missing something.

Re: Just write a test for it

#28

Post by bilalq » Sun, Mar 30, 2025, 10:45 AM UTC

Serverless databases with branching support like Neon make this kind of thing trivial to do. You can just have a test that branches off your prod DB, runs the migrations, and then deletes the branch. This is lightweight enough to easily run on every pull request change.

No mocking or anything. This tests that your migrations are safe to run against real data.

Re: Just write a test for it

#29

Post by bobnamob » Sun, Mar 30, 2025, 10:49 AM UTC

And now you've pulled in a full sql parser as a dependency (admittedly a dev/build time dependency, but a dependency nonetheless) in a project that has no business parsing sql.

In this day and age of increasingly rampant supply chain attacks & dependency vulnerabilities, I'd definitely be second guessing the approach of "just write a test for it" if that test involved blowing up your attack/vuln surface

Re: Just write a test for it

#30

Post by dreghgh » Sun, Mar 30, 2025, 10:50 AM UTC

vlovich123 wrote: post #17

dreghgh wrote: post #13

This is the wrong approach. He should be putting data in the database schema before trying to run migrations on it. Doing that is simple and can potentially catch all sorts of bugs, including the bugs you didn't think of yet. His solution is complicated but only catches one very specific type of bug.

The author correctly notes the challenge of correctly populating some pre-migration data at each step.

You should solve the problem you need to solve, not the problem you have a cool solution for.

The problem to solve is "given my database in an arbitrary but valid state, applying a migration should succeed and leave the database in a equally valid state". Not "how do I stop people adding NOT NULL columns into tables".