Earlier quoted context omitted.
Why'd you run them in kubernetes? Seems like extreme overkill for launching a short lived container for an integration test. What could kubernetes possibly add to that?
Because we are a big company and would like to utilize resources better. We also want homogeneity in tech when possible (we already heavily use kubernetes, we don't want to keep docker hosts anymore). Teams of testers need to be accounted in terms of resource quotas and RBAC. What exactly do you see as an overkill in wanting to run short-lived containers in kubernetes rather than in docker (if we already have kuberne…
Testcontainers
121–130 of 260 posts
Re: Testcontainers
#122I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…
That being said, having specific requirements for the environment of your integration tests is not necessarily bad IMO. It's just a question of checking these requirement and reporting any mismatches.
Re: Testcontainers
#123I don't like layering abstractions on top of abstractions that were fine to begin with. Docker-compose is pretty much perfect for the job. An added complexity is that the before/after semantics of the test suite in things like JUnit are a bit handwavy and hard to control. Unlike testng, there's no @BeforeSuite (which is really what you want). The @BeforeAll that junit has is actually too late in the process to be messing around with docker. And more importantly, if I'm developing, I don't want my docker containers to be wasting time restarting in between tests. That's 20-30 seconds I don't want to add on top of the already lengthy runtime of compiling/building, firing up Spring and letting it do it's thing before my test runs in about 1-2 seconds.
All this is trivially solved by doing docker stuff at the right time: before your test process starts.
So, I do that using good old docker compose and a simple gradle plugin that calls it before our tests run and then again to shut it down right after. If it's already running (it simply probes the port) it skips the startup and shut down sequence and just leaves it running. It's not perfect but it's very simple. I have docker-compose up most of my working day. Sometimes for days on end. My tests don't have to wait for it to come up because it's already up. On CI (github actions), gradle starts docker compose, waits for it to come up, runs the tests, and then shuts it down.
This has another big advantage that the process of running a standalone development server for manual testing, running our integration tests, and running our production server are very similar. Exactly the same actually; the only difference configuration and some light bootstrapping logic (schema creation). Configuration basically involves telling our server the hosts and ports of all the stuff it needs to run. Which in our case is postgres, redis, and elasticsearch.
Editing the setup is easy; just edit the docker compose and modify some properties. Works with jvm based stuff and it's equally easy to replicate with other stuff.
There are a few more tricks I use to keep things fast. I have ~300 integration tests that use db, redis, and elasticsearch. They run concurrently in under 1 minute on my mac. I cannot emphesize how important fast integration tests are as a key enabler for developer productivity. Enabling this sort of thing requires some planning but it pays off hugely.
I wrote up a detailed article on how to do this some years ago. https://www.jillesvangurp.com/blog/2016-05-25-functional-tes...
That's still what I do a few projects and companies later.
Re: Testcontainers
#124Pulling up infra to run unit tests is an anti-pattern. This is a great tool for integration tests, though.
What if you are unit testing something that is dependent on infra?
Integration tests are fine, but they test something else - that your component integrates as intended with , while a unit test moreso tests that your unit behaves in accordance with its specification.
Re: Testcontainers
#125I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…
Testcontainers is not shelling out to the docker CLI; at least on Java, it is using a Java implementation of the docker network protocol, and I believe that’s the case also for the other platforms. Not sure this matters for the core argument you are making, just thought I’d point it out.
Re: Testcontainers
#126Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…
I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…
Re: Testcontainers
#127Earlier quoted context omitted.
If you are testing a microservices "ball of mud", you can (and probably should) setup a testing environment and do your integration tests right there, against real dependencies. The tool seems nice for simple dependencies and local testing but I fail to see it as a game changer.
This is useful too but expensive. Test containers provide a middle ground. For example we have pure unit tests. But also some tests that boot up Postgres. Test the db migration and gives you a db to play with for your specific “unit” test test case. No need for a complete environment with Kafka etc. It provides a cost effective stepping stone to what you describe. What would be nice if test containers could create a…
If you're not testing integration with Kafka, and the producer, your service is still lacking integration tests.
Testing classes in isolation with testcontainer is fine. But I observed that with microservice architecture the line between E2E tests and integration tests are blurred.
Microservices can and should be tested from the client perspective.
Re: Testcontainers
#128Go has a lot of in-memory versions of things for tests, which run so much quicker than leaning on docker. Similarly, I found C# has in-memory versions of deps you can lean on.
I really feel that test containers, although solving a problem, often introduces others for no great benefit
Re: Testcontainers
#129Earlier quoted context omitted.
Testcontainers is not shelling out to the docker CLI; at least on Java, it is using a Java implementation of the docker network protocol, and I believe that’s the case also for the other platforms. Not sure this matters for the core argument you are making, just thought I’d point it out.
The comment you are replying to makes so many mistakes about how Testcontainers works on Java that I'm not sure what source code the commenter is looking at.
Making comments like "this is wrong, but I'm not gonna explain why" has no place here, imho.
Re: Testcontainers
#130I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…