Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testcontainers

testcontainers.com

Post reply on HN
Print view View original

71–80 of 260 posts

Re: Testcontainers

#71

Post by deathanatos »

> Unit tests with real dependencies

That's an integration test. These are integration tests. You're literally testing multiple units (e.g., Redis, and the thing using Redis) to see if they're integrating.

Why do we even have words.

These are valuable in their own right. They're just complicated & often incredibly slow compared to a unit test. Which is why I prefer mocks, too: they're speedy. You just have to get the mock right … and that can be tricky, particularly since some APIs are just woefully underdocumented, or the documentation is just full of lies. But the mocks I've written in the past steadily improve over time. Learn to stop worrying, and love each for what they are.

(Our CI system actually used to pretty much directly support this pattern. Then we moved to Github Actions. GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.)

Re: Testcontainers

#72

Post by pylua »

I found test containers to be slow to startup last year. It wasn’t worth the effort considering how long it took to run compared to traditional spring IT h2 hibernate.

Re: Testcontainers

#73

Post by nslindtner »

Wow ... the syntax reminds me so much of aspire (microsoft new "composer"-syntax). Makes a lot of sense.

Why not keep this information in code .. often the developers are ending up doing those task anyway. (not recommended .. but seen it so many times)

Link: Microsoft aspire (https://learn.microsoft.com/en-us/dotnet/aspire/get-started/...)

Re: Testcontainers

#74

Post by gorjusborg »

simonw wrote: post #68
trevor-e wrote: post #64

Earlier quoted context omitted.

Do you find this results in less overall test code to maintain since you likely have fewer but higher quality/signal tests?

Yeah - I find that sticking to tests like this means I don't have hundreds of tiny unit tests that rely on mocks, and it's still very supportive of refactoring - I can make some pretty big changes and be confident that I've not broken anything because a given request continues to return the expected response.

The choice isn't unit tests vs . end-to-end tests, its between testing things you don't really care about and those you do.

You care about real use cases and verifying design constraints are met. You don't care about internal implementation details.

The nuance is that there are often things one cares about at multiple levels.

Re: Testcontainers

#75

Post by afro88 »

pitah1 wrote: post #70
dm03514 wrote: post #17

Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are fully initialized and ready to accept connections. Pretty much every project I create now has testcontainers for integration testing :) I setup CI so it lints, builds, unit tests then integration tests (using testcontainers) https://github.com/tur…

I think they make the biggest difference when testing data pipelines (which have historically been difficult to test). You can now easily test out compatibility between different versions of databases, verify data types, embed as part of your build, etc. I believe the next step, once using test containers, would be automating data generation and validation. Then you will have an automated pipeline of integration test…

You can automate data validation with snapshot tests. I do it this way with a data pipeline and have a function that queries the destination DBs and puts them unto json to be written validated with a snapshot

Re: Testcontainers

#76

Post by zer00eyz »

ants_everywhere wrote: post #33

> No more need for mocks or complicated environment configurations. Define your test dependencies as code, then simply run your tests and containers will be created and then deleted. Wait what? They think you don't need unit tests because you can run integration tests with containers? It's trivial to set up a docker container with one of your dependencies, but starting containers is painful and slow.

I have been doing E2E testing exclusively for close to a decade on several apps and it works great.

Note, not integration, E2E. I can go from bare vm to fully tested system in under 15 minutes. I can re run that test in 1-5 (depending on project) ...

Im creating 100's of records in that time, and fuzzing a lot of data entry. I could get it to go "even faster" if I went in and removed some of the stepwise testing... A->B->C->D could be broken out to a->b, a->c, a->d.

Because my tests are external, they would be durable across a system re-write (if I need to change language, platform etc). They can also be re-used/tweeked to test system perf under load (something unit tests could never do).

Re: Testcontainers

#77

Post by gui77aume »

dm03514 wrote: post #17

Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are fully initialized and ready to accept connections. Pretty much every project I create now has testcontainers for integration testing :) I setup CI so it lints, builds, unit tests then integration tests (using testcontainers) https://github.com/tur…

If you are testing a microservices "ball of mud", you can (and probably should) setup a testing environment and do your integration tests right there, against real dependencies. The tool seems nice for simple dependencies and local testing but I fail to see it as a game changer.

Re: Testcontainers

#78

Post by zer00eyz »

paulv wrote: post #69

Does this work with podman or is it docker only?

If your running podman you should pull your production deployment configs down and tweak those. You will get a much more complete env that way (routing, network, scale, load balance)

https://www.redhat.com/sysadmin/kubernetes-workloads-podman-... << as a for instance ;)

Re: Testcontainers

#79

Post by briHass »

simonw wrote: post #32

Not sure how I hadn't encountered this before, I LOVE this pattern. I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end. It looks like Testcontainers does all of…

Another technique I've found very useful is generative integration tests (kind of like fuzzing), especially for idempotent API endpoints (GETs).

For example, assuming you have a test database with realistic data (or scrubbed production data), write tests that are based on generalizable business rules, e.g: the total line of an 'invoice' GET response should be the sum of all the 'sections' endpoint responses tied to that invoice id. Then, just have a process that runs before the tests create a bunch of test cases (invoice IDs to try), randomly selected from all the IDs in the database. Limit the number of cases to something reasonable for total test duration.

As one would expect, overly tight assertions can often lead to many false positives, but really tough edge cases hidden in diverse/unexpected data (null refs) can be found that usually escape the artificial or 'happy path' pre-selected cases.

Re: Testcontainers

#80

Post by et1337 »

I looked at testcontainers and ended up rolling my own version. One issue I had is that Docker is a very leaky abstraction. I needed to write one test and have it run in all these scenarios:

- on a Mac

- on a Linux VM

- in a Docker container on a Linux VM, with a Docker socket mounted

The networking for each of these is completely different. I had to make some opinionated choices to get code that could run in all cases. And running inside Docker prevented the test from being able to mount arbitrary files into the test containers, which turns out to be a requirement often. I ended up writing code to build a new image for each container, using ADD to inject files.

I also wanted all the tests to run in parallel and spit out readable logs from every container (properly associated with the correct test).

Not sure if any of these things have changed in testcontainers since I last looked, but these are the things I ran into. It took maybe a month of off and on tweaking, contrary to some people here claiming it can be done in an hour. As always, the devil is in the details.

edit: I did end up stealing ryuk. That thing can’t really be improved upon.

Post reply on HN