Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to test without mocking

amazingcto.com

Post reply on HN
Print view View original

61–70 of 225 posts

Re: How to test without mocking

#61

Post by skybrian »

Mocks can be useful when there is a standard protocol and you want to document and test that your code follows the protocol exactly, doing the same steps, independently of whether some other component also follows the protocol. It tests something different from whether or not two components work together after you change both of them.

It takes time to come up with good protocols that will remain stable and it might not be worth the effort to test it when the protocol design is new and still in flux, and you don’t have alternative implementations anyway. This is often the case for two internal modules in the same system. If you ever want to change the interface, you can change both of them, so an integration test will be a better way to ensure that functionality survives protocol changes.

Database access tends to be a bad thing to mock because the interface is very wide: “you can run any SQL transaction here.” You don’t want to make changing the SQL harder to do. Any equivalent SQL transaction should be allowed if it reads or writes the same data.

Compare with testing serialization: do you want to make sure the format remains stable and you can load old saves, or do you just want a round trip test? It would be premature to test backwards compatibility when you haven’t shipped and don’t have any data you want to preserve yet.

Re: How to test without mocking

#62

Post by rglover »

When I implemented the test suite for my JS framework [1], I realized that there was a ton of cruft and noise in most test set ups. The solution? Just start a mirror of the app [2] and its database(s) on different ports and run the tests against that.

Do away with mocks/stubs in favor of just calling the code you're testing, intentionally using a test-only settings file (e.g., so you can use a dev account for third-party APIs). You can easily write clean up code in your test this way and be certain what you've built works.

[1] https://cheatcode.co/joystick

[2] A mirror of the app/db creates a worry-free test env that can easily be reset without messing up your dev env.

Re: How to test without mocking

#63

Post by randomdata »

firesteelrain wrote: post #55
randomdata wrote: post #37

Earlier quoted context omitted.

Keep in mind that there are different kinds of testing. What Beck called unit tests and integration tests. Unit tests are really for purposes of documentation . They show future programmers the intent and usage of a function/interface so that others can figure out what you were trying to do. Mocking is fine here as future programmers are not looking to learn about the message system here. They will refer to the messa…

Unit tests as a form of example code based documentation is where I could see unit tests complimenting documentation, yes. However, depending on the industry, code coverage is a valuable tool to gauge the maturity of the software baseline and burning down software execution risk. One example of this is Airworthiness or Safety Critical Code.

Of course, there is no single code coverage metric. Code covered by unit tests does not count towards code covered by integration tests. They are completely separate systems. And, at least in Beck's opinion, should be carried out by completely different teams.

Re: How to test without mocking

#64

Post by boustrophedon »

_hl_ wrote: post #17
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…

I made https://github.com/boustrophedon/pgtemp to solve this for myself

Re: How to test without mocking

#65

Post by marcosdumay »

JasonSage wrote: post #32
didiskaka wrote: post #27

Earlier quoted context omitted.

> operate this? Do you spin up a new postgres DB for each unit test? Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". The big thing is to wrap each of your tests in a transaction which gets rolled back after each test. > maintain this, eg have good, representative testing data lying around This is much harder. Maintaining good seed dat…

> Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.

Do not delete develoment_test on your tests, it's supposed to be stable on your machine.

But, the one important thing is, do not give people direct access to production. And for the few that must have it, it should not be easy to connect to it.

Re: How to test without mocking

#66

Post by t43562 »

jasonlotito wrote: post #46

Go ahead. Don't mock that external service that you rely on for an API. Now you need to have multiple keys, one for each developer, or share keys separate from various environments? Does it not offer dev/test/staging/prod keys? Well, now you need to share those keys. Does it only offer Prod keys? Now you are stuck sharing that. API request limits? Now you are eating through that just to run tests. And let's not forge…

That is a fairly good reason for trying to use external systems/tools that make testing easy/cheap to do.

So a good approach would be to have tests where you can run with the mock and then run the same tests with the real system. Anything you catch with the mock saves you from using the costly system but you still get real testing.

Re: How to test without mocking

#67

Post by jeppester »

Anti-Pattern is a word that sounds smart and educated, but is rarely used against something that does not have a legit use case.

This article did not change my opinion on the subject.

The word anti-pattern is confusing in itself. "Anti" is usually a prefix for something that battles or goes against the word that it prefixes.

In my opinion a better word be a hostile or adverse pattern.

Re: How to test without mocking

#68

Post by lowbloodsugar »

“Mocks only test the happy path.”

This is a problem with the test authors, not mocks.

“All the bugs are when talking to an actual database.”

Databases have rules that need to be fillowed, and a lot of those can be tested very quickly with mocks. The combined system can have bugs, so don’t only use mocks. Mocks and unit tests are not a substitute for all the other tests you need to do.

How this person can claim to be a CTO I have no idea.

Re: How to test without mocking

#69

Post by t43562 »

bottlepalm wrote: post #58
jasonlotito wrote: post #46

Go ahead. Don't mock that external service that you rely on for an API. Now you need to have multiple keys, one for each developer, or share keys separate from various environments? Does it not offer dev/test/staging/prod keys? Well, now you need to share those keys. Does it only offer Prod keys? Now you are stuck sharing that. API request limits? Now you are eating through that just to run tests. And let's not forge…

Also the intermittent failures of your tests relying on unstable dependencies.

If your dependencies are unstable then that is very important to know! If it means you have to add forms of resilience then that's good for your code perhaps?
Post reply on HN