Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testcontainers

testcontainers.com

Post reply on HN
Print view View original

111–120 of 260 posts

Re: Testcontainers

#111

Post by Too »

jchw wrote: post #92
paxys wrote: post #50

Earlier quoted context omitted.

FYI Docker already has a RESTful API, and programming container start/stop is trivial to do in any language. I haven't used Testcontainers before, and can kinda see the utility, but IMO it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour.

> it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour. This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? Of course, it's not that easy to write from scratch. You could make a pr…

When writing something like this for yourself you would only support one database, one programming language, one unit testing framework, much less documentation and none of the edge cases that someone else requires.

The effort would not be equal to replicating the whole project. It wouldn’t be the same quality of course. Question is do you need all of it or not.

Re: Testcontainers

#112

Post by hughesjj »

CobrastanJorji wrote: post #89
simonw wrote: post #32

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

Legit. Probably an unpopular opinion but if I had to chose only one type of test (queue a long discussion with no resolution over defining exact taxonomic boundaries), I'd go with integration over unit. Especially if you're a new contributor to a project. I think it comes down to exercising the flow between... Well, integrations across components.

Even better? Take your integration test, put it on a cronjob in your VPN/vpc, use real endpoints and make bespoke auth credentials + namespace, and now you have canaries. Canaries are IMHO God tier for whole system observability.

Then take your canary, clean it up, and now you have examples for documentation.

Unit tests are for me mostly testing domain+codomain of functions and adherence to business logic, but a good type system along with discipline for actually making schemas/POJOs etc instead of just tossing around maps strings and ints everywhere already accomplishes a lot of that (still absolutely needed though!)

Re: Testcontainers

#113

Post by cyptus »

CobrastanJorji wrote: post #89
simonw wrote: post #32

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

how do you handle resetting a sql database after every integration test? Testcontainers may help here by spinning up a new instance for every test but that seems very slow

Re: Testcontainers

#114

Post by jamesdepp »

joeevans1000 wrote: post #108

Forgive the question... but... why can't 'test' containers be 'prod' containers?

Some tests might have side effects. Probably not a great idea to test the function “bill customer” on a prod deployment. That’s why containers for testing is great—it’s easy to spin up an environment that can be messed around with without consequences (even if things go wrong or your tests have side effects).

Re: Testcontainers

#115

Post by stingraycharles »

trevor-e wrote: post #64
simonw wrote: post #32

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…

Do you find this results in less overall test code to maintain since you likely have fewer but higher quality/signal tests?

Yes, you just focus on a few high level behaviors that you want to validate, instead of the units. It’s more difficult to pull these tests off, as there are more chances for them to become flaky tests, but if they work they provide much more value.

I’d prefer a dozen well written integration tests over a hundred unit tests.

Having said that, both solve different problems, ideally you have both. But when time-constrained, I always focus on integration tests with actual services underneath.

Re: Testcontainers

#116

Post by jchw »

Too wrote: post #111
jchw wrote: post #92

Earlier quoted context omitted.

> it really isn't worth it in the long term to take on a new external dependency for a bit of code that (1) is a critical part of the team's development and release process and (2) can be written in-house in maybe an hour. This seems to be quite a contradiction. If it's so easy to just write from scratch, then why would it be scary to depend on? Of course, it's not that easy to write from scratch. You could make a pr…

When writing something like this for yourself you would only support one database, one programming language, one unit testing framework, much less documentation and none of the edge cases that someone else requires. The effort would not be equal to replicating the whole project. It wouldn’t be the same quality of course. Question is do you need all of it or not.

Yeah, but that's kind of the thing, you get less battle testing most likely for also less surface area. If you pull in a project like this, it makes it vastly easier when you need more, because if it happens to already have built-in support, you don't need to spend additional engineer time on it.

To me it seems straightforwardly a win to start with Testcontainer and move to something else when it proves insufficient.

Re: Testcontainers

#117

Post by simonw »

cyptus wrote: post #113
CobrastanJorji wrote: post #89

Earlier quoted context omitted.

I love integration tests. You know why? Because I can safely refactor all I want! Unit tests are great, but if you significantly refactor how several classes talk to each other, and each of those classes had their own, isolated unit tests that mocked out all of the others, you're suddenly refactoring with no tests. But a black box integration tests? Refactor all your code, replace your databases, do whatever you want…

how do you handle resetting a sql database after every integration test? Testcontainers may help here by spinning up a new instance for every test but that seems very slow

If I'm using Django I let Django's default test harness handle that for me - it runs each test in a transaction and rolls it back at the end of the test, which is pretty fast. https://docs.djangoproject.com/en/5.0/topics/testing/overvie...

For my other projects I'm generally using SQLite where starting a new in-memory database is so fast it's effectively free.

Re: Testcontainers

#118

Post by Mavvie »

sverhagen wrote: post #104
lmm wrote: post #81

Earlier quoted context omitted.

Sounds like some kind of protestant work ethic mentality: testing should be hard work, the harder writing your tests was the better your soul and the better your system. I've seen plenty of projects that made oodles of mocks and fakes and unit tests and just sucked, outright didn't work at all in a way that would've been obvious if they'd done testcontainers-based integration tests or even just manual testing in prod…

Just to add, there's also a (Chicago) school of thought that pushes back against mocks and fakes, so even if you're religiously (to stick with the metaphore) writing unit tests, you might still not invest in mocks and fakes.

Can you (or someone else) explain what the alternatives are? How can I write unit tests without mocks or fakes?

Re: Testcontainers

#119

Post by politelemon »

marginalia_nu wrote: post #13
senorrib wrote: post #10

Pulling up infra to run unit tests is an anti-pattern. This is a great tool for integration tests, though.

What if you are unit testing something that is dependent on infra?

If you aren't mocking away the infra, that's an integration test.

Re: Testcontainers

#120

Post by mzi »

briHass wrote: post #79
simonw wrote: post #32

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…

Another technique I've found very useful is generative integration tests (kind of like fuzzing), especially for idempotent API endpoints (GETs). For example, assuming you have a test database with realistic data (or scrubbed production data), write tests that are based on generalizable business rules, e.g: the total line of an 'invoice' GET response should be the sum of all the 'sections' endpoint responses tied to t…

Running unit tests as integration tests will explode in your face. In any decent complex code base testing time will go through the roof and you will have a hard time getting the genie back in the bottle.

Testing that you actually run "sum()" is a unit test.

Post reply on HN