Live data from Hacker News

Pgtestdb's template cloning approach to testing is fast

brandur.org

51–54 of 54 posts

Re: Pgtestdb's template cloning approach to testing is fast

#51
post #50

Earlier quoted context omitted.

tracking the table inserted to isn't reliable without some kind of trigger based registry as it requires all db interactions to go through some kind of orm or something which we don't do because it's a bad thing to do. Sometimes CTE's that modify stuff are 1000x faster than the alternative and it's hard to track what is doing modifications vs not. We do track at the psycopg2 level whether a query has INSERT in it som…

> tracking the table ... isn't reliable without [trigger] or [orm] We use sequences, which is neither of those, and has been very reliable for us. I included the disclaimer that you need a schema that follows strict sequence name => table name conventions, but that's what we have. > 20% of runtime creating users Right -- we also have ~2-3 "stable" rows of users that every test needs, so we can skip re-creating for ea…

Using sequences is clever if you always increment a sequence for every insert, but again my analysis says that unless your table clearing is much slower than mine it's hardly worth it to check, in fact checking the value of every sequence can't take much less than 5ms which is how long it takes to clear every table if empty but is less general.

Re: Pgtestdb's template cloning approach to testing is fast

#52
post #29

Earlier quoted context omitted.

It doesn't look like MySQL has any concept like a template database. You could still migrate up a pristine database, use mysqldump to produce a schema, then load that into a series of test databases. It of course won't be as quick as the low level copies that Postgres is doing with a template database, but it'd be faster than re-running all your migrations. OOC — I've been trying to gather real world anecdotes on who…

MySQL has a lot of broken "features" that sadly some existing legacy code might be relying upon and there just isn't engineering capacity required to move to a saner DB, so they have to make do. I had a legacy project on MySQL that turned out to only "work" because string lookups were case-insensitive in that particular version or our configuration. Moving to Postgres and its correct behavior suddenly exposed a lot o…

[flagged]

Re: Pgtestdb's template cloning approach to testing is fast

#53
post #29

Earlier quoted context omitted.

It doesn't look like MySQL has any concept like a template database. You could still migrate up a pristine database, use mysqldump to produce a schema, then load that into a series of test databases. It of course won't be as quick as the low level copies that Postgres is doing with a template database, but it'd be faster than re-running all your migrations. OOC — I've been trying to gather real world anecdotes on who…

MySQL has a lot of broken "features" that sadly some existing legacy code might be relying upon and there just isn't engineering capacity required to move to a saner DB, so they have to make do. I had a legacy project on MySQL that turned out to only "work" because string lookups were case-insensitive in that particular version or our configuration. Moving to Postgres and its correct behavior suddenly exposed a lot o…

Case sensitivity/insensitivity is a property of the collation, which is configurable on a per-column basis.

In Postgres the default collation is case-sensitive, and in MySQL the default collation is case-insensitive, but this does not mean one is "correct behavior" and the other is not. MySQL's out-of-the-box collation support is arguably a lot more thorough than Postgres's!

Re: Pgtestdb's template cloning approach to testing is fast

#54
post #25

Anything similar possible with MySQL?

In theory you can do filesystem-copy-speed population of large seed data using InnoDB's transportable tablespaces feature. For the test db use-case specifically though, unfortunately it's far less ergonomic than Postgres template databases.
Post reply on HN