31–40 of 225 posts
I then run unit tests against the fake io object. I don't mock internals, only boundaries. If for whatever reason i want to test it against the real db i can simply swap out the fake for the real object.
Earlier quoted context omitted.
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…
> operate this? Do you spin up a new postgres DB for each unit test? Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". The big thing is to wrap each of your tests in a transaction which gets rolled back after each test. > maintain this, eg have good, representative testing data lying around This is much harder. Maintaining good seed dat…
Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.
If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
When you write tests with mocks you almost always at some point end up with tests that test your mocks lol, and tests that test that you wrote the tests you think you wrote -- not the software itself. I’ve never been thrilled by tests that rely on mocking — it usually means you need to re-express your module interface boundary. Mocks for me fall into the class of software I affectionately call “load-bearing paint.” I…
If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…
Typically you'd keep the database container itself alive, and you would run the schema migrations once at startup. Then your test runner would apply fixtures for each test class, which should set up and tear down any data they need to run or that they create while running. Restarting the database server between each test can be very slow.
The test data is a harder problem to solve. For unit tests, you should probably be creating specific test data for each "unit" and cleaning up in between each test using whatever "fixture" mechanism your test runner supports. However, this can get really nasty if there's a lot of dependencies between your tables. (That in and of itself may be a sign of something wrong, but sometimes you can't avoid it or prioritize changing it.)
You can attempt to anonymize production data, but obviously that can go very wrong. You can also try to create some data by using the app in a dev environment and then use a dump of that database in your tests. However, that's going to be very fragile, and if you really need hundreds of tables to be populated to run one test, you've got some big problems to fix.
Property-based testing is an interesting alternative, where you basically generate random data subject to some constraints and run your tests repeatedly until you've covered a representative subset of the range of possible values. But this can be complicated to set up, and if your tests aren't fast, your tests can take a very long time to run.
I think at the end of the day, the best thing you can do is decouple the components of your application as much as possible so you can test each one without needing giant, complicated test data.
If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…
Just run PostgreSQL on your local machine, connect to that, setup a new schema for every test (fairly cheap-ish) inside a test database.
def Test1:
setupdb()
obj1 = createObj1()
obj2 = createObj2()
have = doStuff(obj1, obj2)
if have != want: ...
def Test1:
setupdb()
obj = createObj1()
have = doOtherStuff(obj1)
if have != want: ...
Mocks have a perfectly viable place in testing. They help establish boundaries and avoid side effects that are not pertinent to the logic being tested.
I would reference the testing pyramid when thinking about where to be spending time in unit tests vs. integration tests vs. end to end tests. What introduces risk is if we're mocking behaviors that aren't being tested further up the pyramid.
The problem is that mocks are normally used to avoid testing services/outside code. For example making a wrapper around a message system if you don't mock, it tests both your code and the message system. However the the overhead of keeping the mocking system up to date is a pain in the balls.
Testing both your code and the message system is exactly what you want, since if the message system is broken in a way that upstream didn't catch, you want to learn about it during testing and not production, if possible.
Unit tests are really for purposes of documentation. They show future programmers the intent and usage of a function/interface so that others can figure out what you were trying to do. Mocking is fine here as future programmers are not looking to learn about the message system here. They will refer to the message system's own documentation when they need to know something about the message system.
Integration tests are for the more classical view on testing. Beck suggested this is done by another team using different tools (e.g. UI control software), but regardless of specifics it is done as a whole system. This is where you would look for such failure points.
If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…
we have the same strategy for testing against Kafka., etc.
where we care about data, we seed the db with data for a specific group of tests. Otherwise, we just nuke the db between each test.
Prior to doing this, we'd use in-memory db for tests, and a real db for runtime, using JPA / Hibernate to make things transferrable. But this was leaky, and some things would pass in tests then fail at runtime (or vice versa)
TestContainers has been so much better, as we're running against a real version of the database, so much smaller chance of test and runtime diverging.
Earlier quoted context omitted.
> operate this? Do you spin up a new postgres DB for each unit test? Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". The big thing is to wrap each of your tests in a transaction which gets rolled back after each test. > maintain this, eg have good, representative testing data lying around This is much harder. Maintaining good seed dat…
> Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.
Earlier quoted context omitted.
Testing both your code and the message system is exactly what you want, since if the message system is broken in a way that upstream didn't catch, you want to learn about it during testing and not production, if possible.
I’m still mad about the time I was told to mock a payment gateway in tests even though they had a testing environment and then got a slew of bug reports from people whose company names had punctuation (and thus failed the name validation the payment gateway was secretly running).
You can get the best of both.