Setting up PostgreSQL for running integration tests
1–10 of 50 posts
Re: Setting up PostgreSQL for running integration tests
#2I independently arrived at a very similar design, but also maintain a hot pool of template instances to hide the latency of spinning one up. The state of the template databases is maintained in another database on the same pg instance.
Another thing to contemplate is utilizing the multi tenancy systems you have in place for isolation. It can be a good way to test that the tenant isolation actually works.
Re: Setting up PostgreSQL for running integration tests
#3I’ve been delighted with this approach and have never had a single glitch. Highly recommended, although I think I’ve heard from some denizens of HN that they’ve got even faster performance by manually copying data files instead of using template databases. That feels ickier, and the overhead I see is only 10-20ms per test so I have no great need to push further. YMMV!
Re: Setting up PostgreSQL for running integration tests
#4Wow. So much effort in reinventing the wheel. The transaction approach is obviously not suited for integration testing as commits/rollbacks are part of the package too.
There is testcontainers + flyway/liquibase. Problem solved.
Re: Setting up PostgreSQL for running integration tests
#5I tried this and didn't find any improvements, then I realized we had `PG_DATA: /dev/shm/pgdata/data` which I guess has the same effect
Re: Setting up PostgreSQL for running integration tests
#6Interesting, we've just been looking at something similar at work. Unfortunately we've got a bunch of application config and test data in the database and no fast process for seeding it. We're now building a Postgres Docker image with a baked-in database. There were some tricks for making a small database and it's important to change `PGDATA` to something else, otherwise the `VOLUME` directive (in the official Postgres image) causes a full copy of all the files. It looks promising, though (accepting connections in ~200ms).
Re: Setting up PostgreSQL for running integration tests
#7Wow. So much effort in reinventing the wheel. The transaction approach is obviously not suited for integration testing as commits/rollbacks are part of the package too. There is testcontainers + flyway/liquibase. Problem solved.
Testcontainers are anything but fast in my experience. With the image pre-pulled, the most basic mysql testcontainer takes about 5 seconds from calling RunContainer to it being ready to go.
Re: Setting up PostgreSQL for running integration tests
#8In Ruby on Rails there is a way to do multi-request integration tests with single PostgreSQL/MariaDB database where transactions are used. The trick is to keep and share just one connection to database.
With PostgreSQL at the beginning of the test the outer transaction is opened, then connection is shared with application, test steps are performed (including Selenium etc), requests open/close nested transactions etc. Once steps are done the test closes the outer transaction and the database is back to initial state for the next test. It is very simple and handled by the framework.
In fact it can be even more sophisticated. E.g. an integration test class can preload some data (fixtures) for all the test cases of the class. For that the another nested transaction is used to not repeat the data load process for every test case.
MariaDB doesn't have nested transactions, however in that case RoR uses SAVEPOINTs mechanism. https://mariadb.com/kb/en/savepoint/
Re: Setting up PostgreSQL for running integration tests
#9Wish we can do same with MSSQL that we stucked with =|
Restoring base from snapshot take no less then 4-5 seconds.
No way to run on in memory filesystem without big pile of crutches...
Tried to do so with TestContainers only to find out that we are limited by disk IO
Re: Setting up PostgreSQL for running integration tests
#10In .NET we have TestContainers and Respawn.
https://medium.com/@kova98/easy-test-database-reset-in-net-w...