Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When to Mock

enterprisecraftsmanship.com

Post reply on HN
Print view View original

61–70 of 82 posts

Re: When to Mock

#61

Post by davedx »

matwood wrote: post #18
davedx wrote: post #10

Earlier quoted context omitted.

> Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? So what we do where I work is for unit tests, we mock services and repositories, unit tests don't go anywhere near the database. For integration tests, we use an in memory database. BUT! Be careful before you embark down the path of running integration tests against a different database than the one you use when r…

> For integration tests, we use an in memory database. I think this can work for running the tests locally when speed is what you want, but you should still have the tests run on the actual database on commits. It requires keeping a database running just for tests to run (it can be a smaller instance and startup/shutdown on demand), but it will save so much pain later.

Yes I thought I covered that here:

"BUT! Be careful before you embark down the path of running integration tests against a different database than the one you use when running the application. There are SO many pitfalls"

Re: When to Mock

#62

Post by SAI_Peregrinus »

xhrpost wrote: post #23
pgt wrote: post #2

Panic closed the tab when a giant form popup appeared asking for my email. Hate to add noise to the discussion, but I hope the author sees this and understands the market doesn't want this and reduces readership.

These kind of CTAs are everywhere these days and it's rather frustrating. Wish there was some solution to filter out sites that use completely not-ignorable (like in the corner) CTAs.

> Wish there was some solution to filter out sites that use completely not-ignorable (like in the corner) CTAs.

uBlock Origin, if the default filter lists don't catch it just right-click and block the element.

Re: When to Mock

#63

Post by harrisonjackson »

taffer wrote: post #52
harrisonjackson wrote: post #44

For a long time, we ran our Django test suite against an in-memory SQLite DB. It was super fast, which encouraged more tests to be written and a CI/CD process that allowed everyone to confidently ship code often. Our production database is postgres. We kept bumping against things we wanted to do in the application code that worked well with postgres, but would fail in sqlite. We limited our development so that we cou…

Can't you use transactions to set up and tear down fixtures? That should be a lot faster than spinning up a new database every time.

Yes, that is how the test suite handles it. The CI uses a fresh db for each run of the test job but within that job, it uses 1 instance of the database.

Locally we have a flag to keep the test db alive between runs which speeds the tests up and can help with debugging.

The slowest part of the test run in the CI is building the application container and pulling down the postgres container. I'm sure there are improvements to how we are handling this but it isn't enough of an issue to prioritize it now.

Our issue with fixtures has more to do with changing application code and not having a great way to generate/regenerate the fixtures from live data. We've tried a few different libraries to do this but haven't found any that we love.

Re: When to Mock

#64

Post by harrisonjackson »

malinens wrote: post #50
harrisonjackson wrote: post #44

For a long time, we ran our Django test suite against an in-memory SQLite DB. It was super fast, which encouraged more tests to be written and a CI/CD process that allowed everyone to confidently ship code often. Our production database is postgres. We kept bumping against things we wanted to do in the application code that worked well with postgres, but would fail in sqlite. We limited our development so that we cou…

if you keep using sqlite for tests it forces Your database logic to be universal. You could confidently switch to other db like mysql any time.

This was specifically why we moved away from sqlite. We wanted to take advantage of features postgres offers that are not universal.

Re: When to Mock

#65

Post by mumblemumble »

UK-Al05 wrote: post #42
mumblemumble wrote: post #11

So, disclaimer: I'm about to focus on the finger instead of where it's pointing. Because that's what I feel like jabbering about right now. TFA isn't just about databases; it makes some interesting points on test practice in general, and is well worth a read. Anyway, A story I've seen all too often when mocking the database: A large development effort goes into creating test infrastructure. And then there end up bein…

A good compromise is you write an in-memory version of the dependency and the real version. You write a comprehensive test of integration tests against the real object. You then set it up so those same tests can run both the in-memory and real version. Any differences should show up for the interactions you have specified, and tests should fail. You can write sociable unit tests against the in-memory version, knowing…

That's what I typically do when I must. I find it yields much more readable tests than mocking does, and they're less likely to accidentally become tautological.

That said, discrepancies can still sink in, so I think it's best to also have some baseline level of tests that run against the real dependency, even if they're typically only run overnight on the CI server.

Re: When to Mock

#66

Post by tmountain »

detaro wrote: post #30
doublesCs wrote: post #27

Earlier quoted context omitted.

> Test against the same dependencies that exist in production. Interesting. I said this at my work recently and I got a condescending explanation about how production things are production , we don't touch them. If we need stuff for development, those are dev things. I now think that whether this is or isn't a good idea depends on specifics. Most often than not, I think it makes sense.

I think parent means test against the same code, not against the same instance as production. Tests don't get to talk to the production database, but should use the same database software, deployed as similarly as possible, as production does. Against test/demo versions of APIs if possible. ...

With an identical schema to boot.

Re: When to Mock

#67

Post by [deleted] »

commandlinefan wrote: post #22
mumblemumble wrote: post #11

So, disclaimer: I'm about to focus on the finger instead of where it's pointing. Because that's what I feel like jabbering about right now. TFA isn't just about databases; it makes some interesting points on test practice in general, and is well worth a read. Anyway, A story I've seen all too often when mocking the database: A large development effort goes into creating test infrastructure. And then there end up bein…

> scads of bugs that weren't caught by the unit tests Which is fine! Unit tests will never catch all the bugs - neither will type safety. Neither will code reviews. Neither will manual testing. But unit tests do catch the kinds of bugs that unit tests are good at catching, which eases the burden on the manual testers.

[deleted]

Re: When to Mock

#68

Post by harrisonjackson »

stickfigure wrote: post #60
harrisonjackson wrote: post #44

For a long time, we ran our Django test suite against an in-memory SQLite DB. It was super fast, which encouraged more tests to be written and a CI/CD process that allowed everyone to confidently ship code often. Our production database is postgres. We kept bumping against things we wanted to do in the application code that worked well with postgres, but would fail in sqlite. We limited our development so that we cou…

It isn't clear if you're already doing this, but Postgres has a 'create database from template' feature. You can initialize your template database with migrations once for a whole test suite, then clone a new database at the start of each test. It's quite fast. I run almost all of my tests this way. I agree, forcing your app into the lowest common denominator of portable SQL is crippling. JSONB columns in particular…

Nice! I did not know Postgres had that. We'll probably do something similar next time we prioritize improvements to the test runner.

Right now we are pulling a postgres:10 container down from our ECR on every run :laughcry: so definitely some low hanging fruit around that.

I think we will rebuild the postgres container up to our most recent migrations in prod branch then bake that container onto the gitlab runner AMI daily. Then the test runner can just start that and apply any migrations in the merge request and proceed with the tests.

Re: When to Mock

#69

Post by mumblemumble »

malinens wrote: post #50
harrisonjackson wrote: post #44

For a long time, we ran our Django test suite against an in-memory SQLite DB. It was super fast, which encouraged more tests to be written and a CI/CD process that allowed everyone to confidently ship code often. Our production database is postgres. We kept bumping against things we wanted to do in the application code that worked well with postgres, but would fail in sqlite. We limited our development so that we cou…

if you keep using sqlite for tests it forces Your database logic to be universal. You could confidently switch to other db like mysql any time.

It also means that you have to stick to the lowest common denominator, which is approximately equivalent to the state of the art as of a quarter century ago.

Every benefit has a cost. The benefit doesn't always justify the cost.

edit: To add to that - I've seen more than a couple major database engine migrations in my day, so it's not to say that that isn't a concern. But none of them has ever been from one SQL RDBMS to another. More common is migrating among different classes of database. MySQL to Mongo, Oracle to BigTable, Couch to Cassandra, something like that. MS Access to MS SQL Server a couple times, but even those are different enough that it was never going to be as simple as changing the connection string and having a carefree life.

The speculative future proofing that you do almost never manages to work for the future you end up actually living.

Re: When to Mock

#70

Post by swat535 »

commandlinefan wrote: post #21
jaitsu wrote: post #5

Earlier quoted context omitted.

Writing unit tests is debatable?

On HN, it's not debatable: you should definitely do it. Everywhere I've ever worked, it's also not debatable: no time or effort should be "wasted" writing unit tests, that's what QA is for.

> no time or effort should be "wasted" writing unit tests, that's what QA is for.

This is not my experience at all, everywhere I worked unit tests are required. I don't agree at all, writing unit tests is not a waste of time, this sentiment comes from people who don't know how to avoid brittle tests.

It's impossible for QA to test every single path in your code. Unless you wanna cover all of those in your slow end to end tests?

Post reply on HN