201–210 of 225 posts
Earlier quoted context omitted.
> Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.
It also wouldn't fly in many regulated environments.
Why not?
Earlier quoted context omitted.
> operate this? Do you spin up a new postgres DB for each unit test? Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". The big thing is to wrap each of your tests in a transaction which gets rolled back after each test. > maintain this, eg have good, representative testing data lying around This is much harder. Maintaining good seed dat…
> Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.
That's just a different way of saying "Every place I've ever worked at use production for testing" :-/
TBH, any place using the same credentials for test and for production have bigger problems than would ever be fixed by mocking.
If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…
No need to spin up a new server, not in a container, not in a new directory, not at all. It's pointless busywork with too many extra points of failure.
Nothing is stopping you using an existing server and creating a new DB, which takes about 1/100th the time that starting up a new server (whether in Docker or otherwise) takes.
Secondly, I don't actually do unit-testing on the database layer - there's little point in it. I test workflows against the database, not units!
What I do is create multiple 'packages' of tests, each with multiple tests. A single 'package' creates a temp db, runs its tests sequentially and then drops the temp db. Each package will setup itself with SQL statements.
This lets the tests perform tests of actual workflows, instead of testing in isolation that an object can be (de)serialised. IOW, I can test that the sequence of `addUser(); setProfilePassword(); signIn(); viewInfo(); signOut();` work as expected, and that `removeUser(); signIn();` fail with the correct error.
If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
> If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at worst. Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to y…
Bad article. Some of the advice is good, like decoupling I/O and logic where that makes sense. But the general idea of mocking being an anti-pattern is overreach. This kind of thinking is overly rigid/idealistic: > And with Postgres you can easily copy a test database with a random name from a template for each test. So there is your easy setup. > You need to test reality. Instead of mocking, invest in end-to-end (E2…
[0] https://www.postgresql.org/docs/current/manage-ag-templatedb...
If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
When you write tests with mocks you almost always at some point end up with tests that test your mocks lol, and tests that test that you wrote the tests you think you wrote -- not the software itself. I’ve never been thrilled by tests that rely on mocking — it usually means you need to re-express your module interface boundary. Mocks for me fall into the class of software I affectionately call “load-bearing paint.” I…
And then you mock the delegate?
But tests that are not actually unit tests masquerading as unit tests and vice versa is arguably the bigger problem here. Not mocking per se.
Earlier quoted context omitted.
> Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work. A team I worked with recently said the same thing. But, as I predicted, they ran into bugs because the CloudSQL Postgres was different than their Dockerized Postgres, even though it was the same core version. There will always be testing problems you can't anticipate. Especially wit…
Which bugs? I’ve been ready for 5 years as it’s the duration I’ve been testing my storage layers with 90% integration tests (the 10% being the hard to reproduce error cases, these tests have low value but are easy to test with mocks so I still test them). The only issue I’ve encountered was with time zones (shocking), and it made me ensure I got full control of the time zones in my app, in my deployment, in my local…
- Access to different parts of the database are limited in CloudSQL since it's a managed database. This makes some features of automated tooling (like migrations) not work on CloudSQL (i'm not saying migrations don't work, i'm saying some features do work, some don't). Sometimes elevated permissions can fix it, but some aspects are just walled off.
- There are schema differences from the stock Postgres (I don't remember specifics). No support for custom tablespaces.
- Import/export can often lock up a CloudSQL instance, whereas it might work fine on a local instance.
- You only get Read Committed transaction isolation.
- Operation is different (load, logging, replication, nodes, etc) and this affects performance, and performance can lead to bugs if there's an expectation of a certain run time or performance that doesn't match up with the development experience. Often times some job will have run faster on a laptop than in the cloud, and that leads to weird corner cases where production has an issue due to weird performance and it has to be triaged in the cloud and a fix applied to avoid it. Performance issues don't sound like a bug, but if you have to change your code to make it work in prod, it's a bug.
- To access CloudSQL they want you to use a proxy app, and that can have inconsistent results compared to a dev connecting directly to a Postgres instance. Even something as simple as handling reconnects is often not considered when all you have is a local instance that never needs reconnecting.
- There's a limited selection of Postgres extensions supported. And I could be wrong but I think the version of the extensions is pinned for each version of Postgres core used in CloudSQL.
To all of this you might reply "well none of that affects me", and that's fine... but it does affect other people, and that's important to note when you're telling people on the internet there will be no problems.
Earlier quoted context omitted.
> We can say that a Mock is a kind of spy, a spy is a kind of stub, and a stub is a kind of dummy. But a fake isn’t a kind of any of them. It’s a completely different kind of test double. You define fakes in this case, not mocks https://blog.cleancoder.com/uncle-bob/2014/05/14/TheLittleMo...
If you want. Replace mock by fake in my post and you get the same thing. It means I’m using fakes too when I’m not doing dependency inversion, so no mocks either For what it’s worth, I find the distinction between mocks, fakes, spies, stubs, dummies and whatever completely useless in practice, the whole point is to control some data flows to test in isolation, it’s really all that matters. Fun thing this kind of bike…
Personally, I like seeing "dummy" name passed into a function and understanding just from the name that no methods are expected to called on it during the test. Or seeing "fake_something " and understanding that a manual (perhaps test-specific) implementation is used. It is a minor point but such chunking is universally useful (there is enough stuff to keep track of).
My original point was that the architecture can facilitate so called classical unit tests over mocks (make the latter less necessary) https://martinfowler.com/articles/mocksArentStubs.html#Class...
Earlier quoted context omitted.
> Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.
> Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests. That's just a different way of saying "Every place I've ever worked at use production for testing" :-/ TBH, any place using the same credentials for test and for production have bigger problems than would ever be fixed by mocking.
... is that good? Hell no. But it's a much more common version of reality than you're assuming is happening.