Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Setting up PostgreSQL for running integration tests

gajus.com

Post reply on HN
Print view View original

41–50 of 50 posts

Re: Setting up PostgreSQL for running integration tests

#41

Post by Traubenfuchs »

williamdclt wrote: post #28
dherikb wrote: post #26

I am using TestContainers, what basically is able to run a Docker image of PostgreSQL, abstracting a lot of details. You can find a working example of this setup for integration tests using Go, testify and PosgreSQL here: https://github.com/dherik/ddd-golang-project For Java services using MySQL, I was able to use just the H2 database (in-memory) many times. Does a decent job and it's very compatible with MySQL. If y…

The whole point of the article is to make it _fast_: can you share how fast it is to spin up your database with TestContainers?

Running an "mvnd clean package" on a very simple Spring Boot app having 4 different spring integration tests takes exactly 8 seconds on my m1 max.

This includes a full build of the application, postgres container startup and spring context startup.

Starting the postgres container seems be less than 2 seconds of those 8 seconds.

Is that fast enough?

Re: Setting up PostgreSQL for running integration tests

#43

Post by klysm »

vbezhenar wrote: post #12

There's one idea I'm not sure being described. Basically you create docker container from some postgres image. Then you run DDL scripts. Then you stop this container and commit it as a new image. And now you can create new container from this new image and use it for test. You can even parallelize tests by launching multiple containers. It should be fast enough thanks to docker overlay magic. And it should work with…

You have to pay a lot of latency for startup, and Postgres has all the built in isolation mechanisms internally so it’s easier to use

Re: Setting up PostgreSQL for running integration tests

#44

Post by klysm »

logan12358 wrote: post #6

Interesting, we've just been looking at something similar at work. Unfortunately we've got a bunch of application config and test data in the database and no fast process for seeding it. We're now building a Postgres Docker image with a baked-in database. There were some tricks for making a small database and it's important to change `PGDATA` to something else, otherwise the `VOLUME` directive (in the official Postgr…

You could create a backup with the slow process, restore it, and then the restored database into a template

Re: Setting up PostgreSQL for running integration tests

#45

Post by klysm »

mj4e wrote: post #15

With disclaimer I work at Neon, branching might be a good option here. https://neon.tech/docs/guides/branching-test-queries

I would only consider it if I can do it 100% locally, not sure if that’s the case. And I mean 100% locally without an internet connection

Re: Setting up PostgreSQL for running integration tests

#47

Post by [deleted] »

hn_throwaway_99 wrote: post #27
vbezhenar wrote: post #12

There's one idea I'm not sure being described. Basically you create docker container from some postgres image. Then you run DDL scripts. Then you stop this container and commit it as a new image. And now you can create new container from this new image and use it for test. You can even parallelize tests by launching multiple containers. It should be fast enough thanks to docker overlay magic. And it should work with…

I would think the fundamental issue with this is similar to what the author described with template databases: > However, on its own, template databases are not fast enough for our use case. The time it takes to create a new database from a template database is still too high for running thousands of tests: And then in the timing shows that this took about 2 seconds. Launching another container is surely going to be…

[deleted]

Re: Setting up PostgreSQL for running integration tests

#48

Post by carl_dr »

Traubenfuchs wrote: post #41
williamdclt wrote: post #28

Earlier quoted context omitted.

The whole point of the article is to make it _fast_: can you share how fast it is to spin up your database with TestContainers?

Running an "mvnd clean package" on a very simple Spring Boot app having 4 different spring integration tests takes exactly 8 seconds on my m1 max. This includes a full build of the application, postgres container startup and spring context startup. Starting the postgres container seems be less than 2 seconds of those 8 seconds. Is that fast enough?

From the article:

  However, on its own, template databases are not fast enough for our use case. The time it takes to create a new database from a template database is still too high for running thousands of tests:

  postgres=# CREATE DATABASE foo TEMPLATE contra;
  CREATE DATABASE
  Time: 1999.758 ms (00:02.000)
So no. They say this adds ~33 minutes to running their tests.

Re: Setting up PostgreSQL for running integration tests

#49

Post by AlfeG »

williamdclt wrote: post #28
dherikb wrote: post #26

I am using TestContainers, what basically is able to run a Docker image of PostgreSQL, abstracting a lot of details. You can find a working example of this setup for integration tests using Go, testify and PosgreSQL here: https://github.com/dherik/ddd-golang-project For Java services using MySQL, I was able to use just the H2 database (in-memory) many times. Does a decent job and it's very compatible with MySQL. If y…

The whole point of the article is to make it _fast_: can you share how fast it is to spin up your database with TestContainers?

You can utilize all technics from article with the help of TestContainers. They will help with setting up DB and providing connection strings to running images
Post reply on HN