Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testcontainers

testcontainers.com

Post reply on HN
Print view View original

131–140 of 260 posts

Re: Testcontainers

#131

Post by hyperadvanced »

Mavvie wrote: post #118
sverhagen wrote: post #104

Earlier quoted context omitted.

Just to add, there's also a (Chicago) school of thought that pushes back against mocks and fakes, so even if you're religiously (to stick with the metaphore) writing unit tests, you might still not invest in mocks and fakes.

Can you (or someone else) explain what the alternatives are? How can I write unit tests without mocks or fakes?

They might mean that rather than using a mock, use a real typed object/instance of a real thing and inject it into the unit that you’re testing. Admittedly, that might meet the definition of a fake/mock once you get down to the level of testing something that needs db access. Another way of interpreting that is that you can use in memory versions of your deps to mirror the interface of your dependency without needing to repeatedly, and possibly haphazardly mock certain functions of your dependency.

Re: Testcontainers

#132

Post by fiznool »

rollulus wrote: post #126
CobrastanJorji wrote: post #89

Earlier quoted context omitted.

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

Thanks for saying this out loud. I’m a solo dev and in my project I’m doing exactly this: 90% black box integration tests and 10% unit tests for edge cases I cannot trigger otherwise. It buys me precious time to not adjust tests after refactoring. Yet it made me feel like a heretic: everyone knows the testing pyramid and it comes from Google so I must be very wrong.

You might be interested in the ‘testing trophy’ as an alternative to the traditional pyramid.

https://kentcdodds.com/blog/write-tests

Re: Testcontainers

#133

Post by TheDong »

jchw wrote: post #110
TheDong wrote: post #107

Earlier quoted context omitted.

> This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? It's easy to write an implementation specific to your existing project's development environment and workflow correctly. It's hard to write a generic version that works in any environment. It's hard to write a version that lets you build a company and make money. It's scary to depend on this…

To be honest, it doesn't sound like you really had a good use case for Testcontainers anyways. Where it excels the most is in just pulling in some external containers, especially databases, e.g. PostgreSQL and Redis, directly in your test harness. In those cases, it works well.

Our use-case was redis servers, one of your supposedly good use-cases for testcontainers.

I didn't mention, but the testcontainer code also preferred to add that network io to the actual test runtime, which made measuring the test's performance harder, and meant we couldn't as safely cache the test's results. The bazel version made it easy to build the test's dependency as part of the test compilation process (as it should be) so the test runtime didn't have to do external network IO.

"Excels" is also a stretch; we had a few hundred tests launching dedicated redis servers, and with hand-rolled code, that worked fine with zero flakes. With testcontainers, it regularly flaked with some opaque docker network error or sometimes just plain a timeout because apparently launching 100s of containers in parallel is a hard problem for docker or something.

I'm sure it works well for some people, but if those people wanted to build out their own version without docker and testcontainers, specific to their development tooling and environment, it would probably work better in most cases I think.

Re: Testcontainers

#134

Post by flakes »

cyptus wrote: post #113
CobrastanJorji wrote: post #89

Earlier quoted context omitted.

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

how do you handle resetting a sql database after every integration test? Testcontainers may help here by spinning up a new instance for every test but that seems very slow

I do this a lot for Postgres testing. In my setup, I create a single database for the entire test run. Each test creates its own schema in that database and applies the latest table definitions.

With this setup, I only eat the container creation once, while allowing every test to operate in isolation from one another, be parallelized, and test against a real database.

I do a similar trick for S3 containers by applying a unique guid prefix to the buckets in each test.

Re: Testcontainers

#135

Post by imiric »

CobrastanJorji wrote: post #89
simonw wrote: post #32

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

I've heard this aversion to unit tests a few times in my career, and I'm unable to make sense of it.

Sure, integration tests "save" you from writing pesky unit tests, and changing them frequently after every refactor.

But how do you quickly locate the reason that integration test failed? There could be hundreds of moving parts involved, and any one of them malfunctioning, or any unexpected interaction between them, could cause it to fail. The error itself would likely not be clear enough, if it's covered by layers of indirection.

Unit tests give you that ability. If written correctly, they should be the first to fail (which is a good thing!), and if an integration test fails, it should ideally also be accompanied by at least one unit test failure. This way it immediately pinpoints the root cause.

The higher up the stack you test, the harder it is to debug. With E2E tests you're essentially debugging the entire system, which is why we don't exclusively write E2E tests, even though they're very useful.

To me the traditional test pyramid is still the best way to think about tests. Tests shouldn't be an afterthought or a chore. Maintaining a comprehensive and effective test suite takes as much hard work as, if not more than, maintaining the application itself, and it should test all layers of the system. But if you do have that, it gives you superpowers to safely and reliably work on any part of the system.

Re: Testcontainers

#136

Post by Jnr »

SOLAR_FIELDS wrote: post #88

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…

Similarly to CI agents, I run them in docker-in-docker container, which makes it a lot harder to break anything on the host.

Re: Testcontainers

#137

Post by nrabulinski »

rollulus wrote: post #130
SOLAR_FIELDS wrote: post #88

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck. Testcontainers is the library that convinced me that shelling out to docker as an abstracti…

Does anyone happen to know which testcontainer implementations shell out, if any?

Seems like the „community-maintained” ones they endorse, like the Rust implementation, do

I did not realize Rust wasn’t officially supported until I didn’t go to their GitHub and see in the readme that it’s a community project, and not their „official” one

Re: Testcontainers

#138

Post by comprev »

andrewxdiamond wrote: post #61
paxys wrote: post #50

Earlier quoted context omitted.

FYI Docker already has a RESTful API, and programming container start/stop is trivial to do in any language. I haven't used Testcontainers before, and can kinda see the utility, but IMO it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour.

Conveniently as of a few days ago, you don’t need to be worried about this anymore. TestContainers is no longer a third-party! https://www.docker.com/blog/docker-whale-comes-atomicjar-mak...

If only Docker used a Sperm Whale as their mascot….

Re: Testcontainers

#139

Post by imiric »

fiznool wrote: post #132
rollulus wrote: post #126

Earlier quoted context omitted.

Thanks for saying this out loud. I’m a solo dev and in my project I’m doing exactly this: 90% black box integration tests and 10% unit tests for edge cases I cannot trigger otherwise. It buys me precious time to not adjust tests after refactoring. Yet it made me feel like a heretic: everyone knows the testing pyramid and it comes from Google so I must be very wrong.

You might be interested in the ‘testing trophy’ as an alternative to the traditional pyramid. https://kentcdodds.com/blog/write-tests

This advice is so misguided that I'm concerned for our industry it's getting so much traction.

> You really want to avoid testing implementation details because it doesn't give you very much confidence that your application is working and it slows you down when refactoring. You should very rarely have to change tests when you refactor code.

Unit tests don't need to test implementation details. You could just as well make that mistake with integration or E2E tests. Black box testing is a good practice at all layers.

What unit tests do is confirm that the smallest pieces of the system work as expected in isolation. Yes, you should also test them in combination with each other, but it serves you no good if you get a green integration test, when it's likely only testing a small fraction of the functionality of the units themselves.

This whole "unit tests slow you down" mentality is incredibly toxic. You know what genuinely slows me down? A suite with hundreds of integration tests, each taking several seconds to run, and depend on external systems. But hey, testcontainers to the rescue, right?

Tests shouldn't be a chore, but an integral part of software development. These days I suppose we can offload some of that work to AI, but even that should be done very carefully to ensure that the code is high quality and actually tests what we need.

Test code is as important as application code. It's lazy to think otherwise.

Re: Testcontainers

#140

Post by hamandcheese »

imiric wrote: post #135
CobrastanJorji wrote: post #89

Earlier quoted context omitted.

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

I've heard this aversion to unit tests a few times in my career, and I'm unable to make sense of it. Sure, integration tests "save" you from writing pesky unit tests, and changing them frequently after every refactor. But how do you quickly locate the reason that integration test failed? There could be hundreds of moving parts involved, and any one of them malfunctioning, or any unexpected interaction between them, c…

> But how do you quickly locate the reason that integration test failed? There could be hundreds of moving parts involved, and any one of them malfunctioning, or any unexpected interaction between them, could cause it to fail. The error itself would likely not be clear enough, if it's covered by layers of indirection.

As long as the error is reproducible, never in my career have I had a hard time locating the source of the error. Bisection does wonders (as a general concept, not specifically referring to git bisect).

That said, I have encountered plenty of non-reproducible test failures. Moral of the story: make things reproducible, especially tests.

Easier said than done.

Post reply on HN