Live data from Hacker News

Stop mocking your system

blog.bitgloss.ro

171–178 of 178 posts

Re: Stop mocking your system

#171
Mocks and fakes are used to test your code under isolation - like an experiment, only one variable should be under test at one time.

In addition, most stateful code that can fail, does so intermittently, is susceptible to load delay at scale that can effect your customers and your wallet, may not be at the released version you expect, requires clean-up so that your storage isn't clogged with useless test data, which may or my not be mocked, effects downstream analytic services, etc.

Isolate your unit tests with fakes and in-memory test-local storage, run proprety-based tests with generated data for both inputs and returns that confor to the constraints of your domain, including possible expected error conditions, and unit test in parallel. If you do these things, you can and will have many of the benefits of integration tests, and won't waste time writing carefully crafted test data/scenarios.

Integration test when you get a bug in production, and run the integration tests in an isolated environment identical to the one you deploy to prior to deployment to a place where your users will be interacting with your code. Cleanup can then simply be destruction of the test environment. There was a talk on blue/green deployment the way AWS SRE does it a few years back that was great. While I have the notes, I can't find the talk, but it's a much more complex process than running tests against your up system.

Re: Stop mocking your system

#172

Earlier quoted context omitted.

I used to do TDD, I found that it didn't work that well for me. I still think that the code base you work on dictates how you should be testing. It also dictates which testing strategy I use. I test for different things when writing C than I do when I write Python for example. My testing strategy is different if I write a networked C application compared to if I write a Ruby on Rails app. Also the tools I have availa…

I think the point was more that many people can’t write unit tests because their code is so shoddy that it’s basically impossible to do so. It sounds like you and I share a similar methodology. I don’t write unit tests for all my code. In fact if I had to guess, I’d say I generally test less than 25%. But I do write all my code such that it’s capable of having unit tests written for it. In many codebases that’s just…

I do enjoy TDD, in that it makes it easier for me to ensure that I implement everything I need.

I completely agree that it's not always necessary, but I definitely find it a useful tool.

That being said, TDD with ML applications is a bunch trickier, as you'll normally have functions/classes which take an unfathomably looooooonnnnngggg time to run.

Re: Stop mocking your system

#173
post #123

Earlier quoted context omitted.

> The only thing you achieve with unit tests is to prove that you are still bug for bug compatible when changing your code. People love to ignore how fucking awful code is when you aren't forced to write unit tests. Unit tests before you write code help you code, they provide a tight feedback loop, they force or at least encourage modularity to be testable. Adding tests to my code after writing it forces me to challe…

I used to do TDD, I found that it didn't work that well for me. I still think that the code base you work on dictates how you should be testing. It also dictates which testing strategy I use. I test for different things when writing C than I do when I write Python for example. My testing strategy is different if I write a networked C application compared to if I write a Ruby on Rails app. Also the tools I have availa…

40-50% is perfectly acceptable for your team by the sounds of it. In other teams with varying degrees of experience and skill things are a bit more wild.

The fact that you can add tests means your code is at least somewhat scaffholdable and that is the important part for testability.

For the record I do TDD maybe 20% of the time, I mostly just use it in tough problems where I don't know what to do next.

It's a tool, not a goal.

Re: Stop mocking your system

#174

Earlier quoted context omitted.

If a test is in my way, I delete it.

Then what is the point of it being there in the first place?

As other said, if/when it comes to this it is because the code in question has change in a significant way. So the point was to test a previous version.

Re: Stop mocking your system

#175
post #51

True "unit" tests are: * faster to run * give you less confidence in the correctness of the system (per time spent writing them) * when they fail, give you more information about where the failure is The more integration-y/e2e-y a test is, the more it strays from this: slower to run, more confidence that the system is correct, less info about where the failures are. I think people have learned to undervalue the prope…

Having noodled on these comments for a few, I share your view, but I'm curious if the type of testing varies based on seniority/carefulness. I find people earlier in their careers (or people with less care) write code where the failures at boundary conditions are obvious to me, but not to them. I can see unit tests (or quick check) paying big dividends with that group while being lower ROI for more experienced / more careful teams.

Re: Stop mocking your system

#176

Earlier quoted context omitted.

I used to do TDD, I found that it didn't work that well for me. I still think that the code base you work on dictates how you should be testing. It also dictates which testing strategy I use. I test for different things when writing C than I do when I write Python for example. My testing strategy is different if I write a networked C application compared to if I write a Ruby on Rails app. Also the tools I have availa…

I think the point was more that many people can’t write unit tests because their code is so shoddy that it’s basically impossible to do so. It sounds like you and I share a similar methodology. I don’t write unit tests for all my code. In fact if I had to guess, I’d say I generally test less than 25%. But I do write all my code such that it’s capable of having unit tests written for it. In many codebases that’s just…

Well, I can agree with that!

Chances are that your code base will be untestable, at least for unit testing, if unit testing where not considered at all when writing it.

Re: Stop mocking your system

#177
post #133

Earlier quoted context omitted.

I personally use mocks or stubs only in unit tests. Everything else should be live test or recorded network responses but always run through live code. Unless the service really depends on it, I do not reccommend setting up local swarm of services. This breaks down if you shift to distributed micro services and has little benefit for a single service app (ex crud + database app, no reason to test the database in the…

Persisting data for your test paths is much easier with a database though.

Yes full QA and Prod environments should provide coverage for most cases in which you would not need to setup a database on each test run.

Re: Stop mocking your system

#178

Earlier quoted context omitted.

Then what is the point of it being there in the first place?

As other said, if/when it comes to this it is because the code in question has change in a significant way. So the point was to test a previous version.

Tests have nothing to do with the code. They are supposed to test the behaviour.
Post reply on HN