Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testcontainers

testcontainers.com

Post reply on HN
Print view View original

101–110 of 260 posts

Re: Testcontainers

#101

Post by m00x »

jake_morrison wrote: post #100

You can do something similar with docker compose, driving the system from the outside. Create dockerized versions of dependencies like the database, build and run tests, and then run tests against the production app container. It's particularly useful for testing a set of microservices. See https://github.com/cogini/phoenix_container_example for a full example. This blog post describes it in detail: https://www.cogin…

We use docker environments like this for tests, but it does have its issues.

You often need to add custom behavior like waiting for the app to load and start serving, healthchecks, etc. Having it all in code is pretty useful, and it's self-contained within the code itself vs having to set up the environment in different places (CI, Github actions, local dev, etc).

The negative is that code isn't portable to prod, it doesn't test your environment as well (important for staging), and you're missing out on sharing some environment settings.

I feel like it definitely has its place in the stack and in certain companies.

Re: Testcontainers

#102

Post by AdrenalinMd »

gui77aume wrote: post #77
dm03514 wrote: post #17

Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are fully initialized and ready to accept connections. Pretty much every project I create now has testcontainers for integration testing :) I setup CI so it lints, builds, unit tests then integration tests (using testcontainers) https://github.com/tur…

If you are testing a microservices "ball of mud", you can (and probably should) setup a testing environment and do your integration tests right there, against real dependencies. The tool seems nice for simple dependencies and local testing but I fail to see it as a game changer.

I agree with this. At work we use both approaches but at different levels of the test pyramid.

To test integration with 1 dependency at class level we can use test containers.

But to test the integration of the whole microservice with other microservices + dependencies we use a test environment and some test code. It's a bit like an E2E test for an API.

I would argue that the test environment is more useful if I had to choose between the two as it can test the service contract fully, unlike lower type testing which requires a lot of mocking.

Re: Testcontainers

#103

Post by sverhagen »

ants_everywhere wrote: post #53
sverhagen wrote: post #43

Earlier quoted context omitted.

1) At least in the Java world, the term "unit testing" is often confused by "things you do in JUnit", which runs both "pure" unit tests and project-level integration tests, i.e. spinning up an application context (like Spring) and testing against real REST endpoints etc. 2) While unit tests are cheaper and quicker than (project-level) integration tests, they also in many cases don't provide results as good a result a…

Yeah, certainly I see the value of those kinds of tests. And clearly as you say the simpler tests don't provide as realistic a simulation as the more expensive tests. But on the test philosophy angle, my take on what's happening is just that developers traditionally look for any reason to skip tests. I've seen this in a few different forms. - right now containers make it trivial to run all of your dependencies. That'…

I think testing is very important. But it's very hard to test everything. (It is not hard to get 100% test coverage by some metric, but that does not mean that all scenarios or even the most useful ones are covered.) So it's an economics game: how can you get the most value for the least amount of money? Or if you want me to rephrase that in a more-positive way: how can you get the most value out of the time that you have available? And I contend that a shift "up" in the pyramid (at which time it looses that shape, hence the "testing trophy") is where the current sweet spot lies. You have to use the tools that you have.

Re: Testcontainers

#104

Post by sverhagen »

lmm wrote: post #81
ants_everywhere wrote: post #53

Earlier quoted context omitted.

Yeah, certainly I see the value of those kinds of tests. And clearly as you say the simpler tests don't provide as realistic a simulation as the more expensive tests. But on the test philosophy angle, my take on what's happening is just that developers traditionally look for any reason to skip tests. I've seen this in a few different forms. - right now containers make it trivial to run all of your dependencies. That'…

Sounds like some kind of protestant work ethic mentality: testing should be hard work, the harder writing your tests was the better your soul and the better your system. I've seen plenty of projects that made oodles of mocks and fakes and unit tests and just sucked, outright didn't work at all in a way that would've been obvious if they'd done testcontainers-based integration tests or even just manual testing in prod…

Just to add, there's also a (Chicago) school of thought that pushes back against mocks and fakes, so even if you're religiously (to stick with the metaphore) writing unit tests, you might still not invest in mocks and fakes.

Re: Testcontainers

#105

Post by root_axis »

gui77aume wrote: post #77
dm03514 wrote: post #17

Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are fully initialized and ready to accept connections. Pretty much every project I create now has testcontainers for integration testing :) I setup CI so it lints, builds, unit tests then integration tests (using testcontainers) https://github.com/tur…

If you are testing a microservices "ball of mud", you can (and probably should) setup a testing environment and do your integration tests right there, against real dependencies. The tool seems nice for simple dependencies and local testing but I fail to see it as a game changer.

> and local testing

You mention this as an afterthought but that's the critical feature. Giving developers the ability to run integration tests locally is a massive win in a "ball of mud" environment. There are other ways to accomplish this locally, but the test-infrastructure-as-test-code approach is a powerful and conceptually elegant abstraction, especially when used as a tool to design testcontainers for your own services that can be imported as packages into dependent services.

Re: Testcontainers

#106

Post by imp0cat »

codeonline wrote: post #59
maxdaten wrote: post #29

Earlier quoted context omitted.

honest question: how are you writing integration tests? We are writing these as separate test suite often with the same test style. And in this scenario testcontainers are very valuable.

Why not use docker compose, bring up your infra in one container, your application in a second and your tests access it the application from a third.

Because you don't have to muck around with docker-compose. I guess some people might find that more attractive.

Re: Testcontainers

#107

Post by TheDong »

jchw wrote: post #92
paxys wrote: post #50

Earlier quoted context omitted.

FYI Docker already has a RESTful API, and programming container start/stop is trivial to do in any language. I haven't used Testcontainers before, and can kinda see the utility, but IMO it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour.

> it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour. This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? Of course, it's not that easy to write from scratch. You could make a pr…

> This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on?

It's easy to write an implementation specific to your existing project's development environment and workflow correctly.

It's hard to write a generic version that works in any environment. It's hard to write a version that lets you build a company and make money.

It's scary to depend on this generic version because it's too generic, and it's built by a for-profit company now who wants to upsell you to some "testcontainer cloud" crap, which doesn't exactly incentivize them to make the OSS version perfect.

For example, we were already using bazel, so writing a version using bazel to create a correct bare rootfs for a dependency + using runc to execute it in tests resulted in something with a roughly 3ms warm startup time that fulfilled all our needs, and cached correctly (since bazel has good caching).

A junior engineer, starry-eyed at test-containers, switched some code over to it, and the warm startup time went up by 1000x from 3ms to 3s, as did the flake-rate, and docker's caching is far worse than bazel's so the several minute cold-starts also happened even more often.

> Testcontainers is exactly the sort of thing open source is great for; it's something where everyone gets to benefit from the wisdom and battle-testing of everyone else.

You get to have a mismatched mess of solutions to everyone's problems, including solution's to problems you don't have. You get code of the quality of the average OSS programmer which, while higher than the average programmer, is still pretty crap.

Free Software is great when it's run by a small group of smart opinionated people. As soon as you try to build a company around that OSS and, god forbid, hire some enterprise sales people and product managers, it quickly becomes worse at the actual small developer problem than what even a mediocre developer could hack out in an hour.

Depending on testcontainers is fine, but if you know what you're doing, writing something purpose-built is fine too, and probably gives you something much nicer in the end.

Re: Testcontainers

#109

Post by simonkagedal »

SOLAR_FIELDS wrote: post #88

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…

Testcontainers is not shelling out to the docker CLI; at least on Java, it is using a Java implementation of the docker network protocol, and I believe that’s the case also for the other platforms.

Not sure this matters for the core argument you are making, just thought I’d point it out.

Re: Testcontainers

#110

Post by jchw »

TheDong wrote: post #107
jchw wrote: post #92

Earlier quoted context omitted.

> it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour. This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? Of course, it's not that easy to write from scratch. You could make a pr…

> This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? It's easy to write an implementation specific to your existing project's development environment and workflow correctly. It's hard to write a generic version that works in any environment. It's hard to write a version that lets you build a company and make money. It's scary to depend on this…

To be honest, it doesn't sound like you really had a good use case for Testcontainers anyways. Where it excels the most is in just pulling in some external containers, especially databases, e.g. PostgreSQL and Redis, directly in your test harness. In those cases, it works well.
Post reply on HN