so this will make the actual http request the first time, then keep a recording? I’m not entirely clear how this works from the docs.
Polly.js – Record, replay, and stub HTTP interactions
41–50 of 91 posts
Re: Polly.js – Record, replay, and stub HTTP interactions
#42I've used mitmproxy + proxychains to do this. How is Polly different?
https://netflix.github.io/pollyjs/#/README Why Polly? Keeping fixtures and factories in parity with your APIs can be a time consuming process. Polly alleviates this by recording and maintaining actual server responses without foregoing flexibility. * Record your test suite's HTTP interactions and replay them during future test runs for fast, deterministic, accurate tests. * Use Polly's client-side server to modify or…
Not sure that tells me how it's different from replay or the other half dozen npm modules that do the same thing. It'd be nice for them to contrast their tool with existing ecosystem options considering some of them are pretty well established.
Re: Polly.js – Record, replay, and stub HTTP interactions
#43There is already famous Policy package for .NET with same name
Re: Polly.js – Record, replay, and stub HTTP interactions
#44I would love to hear from people involved in projects like this, what kind of work/ how much, was done to get it ready for and approved to be open sourced by the company. Especially at large corps like Netflix I'm sure there's a lot of hoops to jump through.
Netflix seems to also have a really strong culture around this though, so I wouldn't be surprised if it's a lot less hoopy than you'd imagine.
Exactly. They had engineers dedicated to creating an open source latency/fault tolerance library called Hystrix. Not too surprising they’re dedicating resources to other projects, too.
Re: Polly.js – Record, replay, and stub HTTP interactions
#45Earlier quoted context omitted.
Selenium is for behavior testing. Simulating clicks and form filling. This is for mocking http endpoints.
The example is form filling though: await fillIn('email', 'polly@netflix.com'); await fillIn('password', '@pollyjs'); This is exactly like selenium code I've written to login. I struggle to see the difference in purpose.
In your selenium code, the browser was talking to a database.
But sometimes that database is down, or really slow.
Polly says "browser, don't talk to the database anymore, instead here's what the database said last time".
So, yes, both Selenium and Polly poke DOM elements, but Selenium stops there, where as Polly does that + as well as tricks the browser into going through the whole test without making a real call to the database (assuming it has a previous recording of "what the database said" for that test).
Re: Polly.js – Record, replay, and stub HTTP interactions
#46Looks well done (it uses unicode art, so it must be amazing) but I have a fundamental distrust/dislike of record/replay frameworks...just seems like you're papering over an inherently bad testing approach.
E.g. sure, when replays work, they're great, but:
a) you have to do a manual recording to create them the first time (which means manually setting up test data in production that is just-right for your test case)
b) you have to manually re-record when they fail (and again means you have manually go back and restore the test data in production that is just-right for your test case...and odds are you weren't the one who originally recorded this test, so good luck guessing exactly what that was).
In both cases, you've accepted that you can't easily setup specific, isolated test data in your upstream system, and are really just doing "slightly faster manual testing".
So, IMO, you should focus on solving the core issue: the uncontrollable upstream system.
Or, if you can't, decouple all of your automated tests from it fully, and just accept that cross-system tests against a datasource you can't control is not a fun/good way to write more than a handful of tests (e.g. ~5 smoke tests are fine, but ~100s of record/replay tests for minute boundary cases sounds terrible).
Re: Polly.js – Record, replay, and stub HTTP interactions
#47I would love to hear from people involved in projects like this, what kind of work/ how much, was done to get it ready for and approved to be open sourced by the company. Especially at large corps like Netflix I'm sure there's a lot of hoops to jump through.
I'm sure the first project took some time to setup, but Netflix has released dozens of projects since. So very low hoops.
Re: Polly.js – Record, replay, and stub HTTP interactions
#48Earlier quoted context omitted.
Netflix seems to also have a really strong culture around this though, so I wouldn't be surprised if it's a lot less hoopy than you'd imagine.
Not to shamelessly plug but if you're in the Bay Area on June 28, we're giving a talk that's a bit about performance, a bit about Netflix engineering culture: https://jstalks2018.splashthat.com/ .
If I’m a student can I sign up to attend?
Re: Polly.js – Record, replay, and stub HTTP interactions
#49Looks well done (it uses unicode art, so it must be amazing) but I have a fundamental distrust/dislike of record/replay frameworks...just seems like you're papering over an inherently bad testing approach. E.g. sure, when replays work, they're great, but: a) you have to do a manual recording to create them the first time (which means manually setting up test data in production that is just-right for your test case) b…
How is this distinct from other http stubbing libraries?
Re: Polly.js – Record, replay, and stub HTTP interactions
#50Earlier quoted context omitted.
I still don't get why we do this [mocking http endpoints]. Sure this makes the problem of mocking the server less painful. Well done. But I'd take completely integrated tests over these any day. Sure they're slower but that's more or less irrelevant with feature toggling, staged roll-out and continuous production monitoring. It's totally possible to completely avoid mocking http endpoints thus making these tools comp…
It's not always easy. Especially not if your API is stateful.
Or if it's not your API. I've been looking for something like this to make mocking OAuth flow a lot easier.