Live data from Hacker News

Testcontainers

testcontainers.com

121–130 of 260 posts

Re: Testcontainers

#121

Earlier quoted context omitted.

Why'd you run them in kubernetes? Seems like extreme overkill for launching a short lived container for an integration test. What could kubernetes possibly add to that?

Because we are a big company and would like to utilize resources better. We also want homogeneity in tech when possible (we already heavily use kubernetes, we don't want to keep docker hosts anymore). Teams of testers need to be accounted in terms of resource quotas and RBAC. What exactly do you see as an overkill in wanting to run short-lived containers in kubernetes rather than in docker (if we already have kuberne…

That reasoning seems more like one from policy/cargo cult rather than reasoning specific to your org. For something short lived and meant to be isolated I wouldn't want to subject them to even more infrastructural dependencies outside their control.

Re: Testcontainers

#122

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…

Came here with exactly this on my mind. Thanks for confirming my suspicion.

That being said, having specific requirements for the environment of your integration tests is not necessarily bad IMO. It's just a question of checking these requirement and reporting any mismatches.

Re: Testcontainers

#123
I never really liked testcontainers. Too complicated. And I don't want to have my tests make too many assumptions about what level of control there is over their environment. IMHO it's just the wrong place to be messing with docker.

I don't like layering abstractions on top of abstractions that were fine to begin with. Docker-compose is pretty much perfect for the job. An added complexity is that the before/after semantics of the test suite in things like JUnit are a bit handwavy and hard to control. Unlike testng, there's no @BeforeSuite (which is really what you want). The @BeforeAll that junit has is actually too late in the process to be messing around with docker. And more importantly, if I'm developing, I don't want my docker containers to be wasting time restarting in between tests. That's 20-30 seconds I don't want to add on top of the already lengthy runtime of compiling/building, firing up Spring and letting it do it's thing before my test runs in about 1-2 seconds.

All this is trivially solved by doing docker stuff at the right time: before your test process starts.

So, I do that using good old docker compose and a simple gradle plugin that calls it before our tests run and then again to shut it down right after. If it's already running (it simply probes the port) it skips the startup and shut down sequence and just leaves it running. It's not perfect but it's very simple. I have docker-compose up most of my working day. Sometimes for days on end. My tests don't have to wait for it to come up because it's already up. On CI (github actions), gradle starts docker compose, waits for it to come up, runs the tests, and then shuts it down.

This has another big advantage that the process of running a standalone development server for manual testing, running our integration tests, and running our production server are very similar. Exactly the same actually; the only difference configuration and some light bootstrapping logic (schema creation). Configuration basically involves telling our server the hosts and ports of all the stuff it needs to run. Which in our case is postgres, redis, and elasticsearch.

Editing the setup is easy; just edit the docker compose and modify some properties. Works with jvm based stuff and it's equally easy to replicate with other stuff.

There are a few more tricks I use to keep things fast. I have ~300 integration tests that use db, redis, and elasticsearch. They run concurrently in under 1 minute on my mac. I cannot emphesize how important fast integration tests are as a key enabler for developer productivity. Enabling this sort of thing requires some planning but it pays off hugely.

I wrote up a detailed article on how to do this some years ago. https://www.jillesvangurp.com/blog/2016-05-25-functional-tes...

That's still what I do a few projects and companies later.

Re: Testcontainers

#124

Pulling up infra to run unit tests is an anti-pattern. This is a great tool for integration tests, though.

What if you are unit testing something that is dependent on infra?

Arguably you're no longer testing a unit if the unit involves an integration with an external component, making it an integration test per definition.

Integration tests are fine, but they test something else - that your component integrates as intended with , while a unit test moreso tests that your unit behaves in accordance with its specification.

Re: Testcontainers

#125

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…

Testcontainers is not shelling out to the docker CLI; at least on Java, it is using a Java implementation of the docker network protocol, and I believe that’s the case also for the other platforms. Not sure this matters for the core argument you are making, just thought I’d point it out.

The comment you are replying to makes so many mistakes about how Testcontainers works on Java that I'm not sure what source code the commenter is looking at.

Re: Testcontainers

#126
post #32

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

Thanks for saying this out loud. I’m a solo dev and in my project I’m doing exactly this: 90% black box integration tests and 10% unit tests for edge cases I cannot trigger otherwise. It buys me precious time to not adjust tests after refactoring. Yet it made me feel like a heretic: everyone knows the testing pyramid and it comes from Google so I must be very wrong.

Re: Testcontainers

#127
post #99

Earlier quoted context omitted.

If you are testing a microservices "ball of mud", you can (and probably should) setup a testing environment and do your integration tests right there, against real dependencies. The tool seems nice for simple dependencies and local testing but I fail to see it as a game changer.

This is useful too but expensive. Test containers provide a middle ground. For example we have pure unit tests. But also some tests that boot up Postgres. Test the db migration and gives you a db to play with for your specific “unit” test test case. No need for a complete environment with Kafka etc. It provides a cost effective stepping stone to what you describe. What would be nice if test containers could create a…

It's not really a middle ground if you're not testing your service in the same conditions as in production environment.

If you're not testing integration with Kafka, and the producer, your service is still lacking integration tests.

Testing classes in isolation with testcontainer is fine. But I observed that with microservice architecture the line between E2E tests and integration tests are blurred.

Microservices can and should be tested from the client perspective.

Re: Testcontainers

#128
If you build inside docker, running tests that use docker is a pain.

Go has a lot of in-memory versions of things for tests, which run so much quicker than leaning on docker. Similarly, I found C# has in-memory versions of deps you can lean on.

I really feel that test containers, although solving a problem, often introduces others for no great benefit

Re: Testcontainers

#129

Earlier quoted context omitted.

Testcontainers is not shelling out to the docker CLI; at least on Java, it is using a Java implementation of the docker network protocol, and I believe that’s the case also for the other platforms. Not sure this matters for the core argument you are making, just thought I’d point it out.

The comment you are replying to makes so many mistakes about how Testcontainers works on Java that I'm not sure what source code the commenter is looking at.

Why don't you point them out, please. We already know about testcontainers not using the shell, but rather talking to the HTTP API.

Making comments like "this is wrong, but I'm not gonna explain why" has no place here, imho.

Re: Testcontainers

#130

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…

Does anyone happen to know which testcontainer implementations shell out, if any?
Post reply on HN