Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A routine gem update ended up creating $73k worth of subscriptions

serpapi.com

Post reply on HN
Print view View original

101–110 of 336 posts

Re: A routine gem update ended up creating $73k worth of subscriptions

#101

Post by kortex »

Railsify wrote: post #42
ricardobeat wrote: post #8

That sounds like a major, incredibly dangerous update to the DB driver. Their 7.1, 7.2, 7.3 versions seem to all have breaking changes [1]. Yet they are in obvious violation of SemVer expectations, which they declare to follow [2]: > Mongoid follows versioning guidelines as outlined by the Semantic Versioning Specification, so you can expect only backwards incompatible changes in major versions [sic] [1] https://docs…

Semantic Versioning does not mean you can update without reading the changelog. Blame here remains with the developer who updated without reading the changelog.

Do you or your team review every item of every changelog of every dependency in your stack (recursively for their dependencies) for every minor update, and/or 100% coverage for all of your assumptions of how they are used?

Sure it was a mistake by OP in TFA but it's an honest mistake any of us could make. And it's a really Bad Move by the Mongoid devs. It'd be a bad move in 1.2->1.3, but by 7.3? I'd just be like "whelp, this is how this function behaves now in perpetuity, I guess just document that it has different semantics than AR" even if an 8.0 was released. This is the kind of change which causes really subtle bugs. It's rarely worth it. Just introduce .chain_or() or something.

Re: A routine gem update ended up creating $73k worth of subscriptions

#102

Post by lmeyerov »

asdfasgasdgasdg wrote: post #29
lmeyerov wrote: post #22

Earlier quoted context omitted.

Breaking changes are things, no project is ever done -- ye olde django is still doing big moves like adding async colors, and that's great! But... CHANGELOG.md with BREAKING CHANGE sections and a major version bump, maybe a DEPRECATED lint and runtime warn() ahead of time, and bam, no surprises for professional teams. For more than that, service contracts are things :)

I don't think you can ever make a breaking change like the one described. There's just no way to audit the correctness of all your users after that change. You need to leave the old thing with the old behavior and only add the new behavior to a new thing.

It's the users who need to audit, the social contract of OSS is just not to be sneaky/sloppy about it (when posing as a serious project).

Semvar is beautiful bc it lets you be explicit. Likewise, end users can judge "wow major version 27 in as many months, maybe not so good for us." We had a gov customer today upfront about needing slow updates, and same deal -- maybe our SaaS and OSS libs are too fast moving, so they are probably better off with our enterprise distros.

Re: A routine gem update ended up creating $73k worth of subscriptions

#103

Post by deviantintegral »

dragonwriter wrote: post #30
ricardobeat wrote: post #8

That sounds like a major, incredibly dangerous update to the DB driver. Their 7.1, 7.2, 7.3 versions seem to all have breaking changes [1]. Yet they are in obvious violation of SemVer expectations, which they declare to follow [2]: > Mongoid follows versioning guidelines as outlined by the Semantic Versioning Specification, so you can expect only backwards incompatible changes in major versions [sic] [1] https://docs…

> Mongoid follows versioning guidelines as outlined by the Semantic Versioning Specification, so you can expect only backwards incompatible changes in major versions I will note that this reverses the direction of implications. In SemVer, you should expect breaking changes only in major versions. (All version changes with breaking changes should be major, but nonbreaking changes can occur in major or minor versions.)…

They may actually mean what they say. It’s not uncommon for projects to release the last minor version at the same time as the next major. For example, Drupal 9.0.0 was functionally equivalent to Drupal 8.9.0, except 9.0 had all deprecations dropped. New 9.0+ only features didn’t show up until 9.1.

Re: A routine gem update ended up creating $73k worth of subscriptions

#104

Post by jbverschoor »

Besides the issue, I always create a method like get/find find_one, find_unique, find_unique, or find_only, in cases where I expect exactly one result. I’m a query or collection. I’ll have a separate for when I expect one or zero results.

Unfortunately it’s crazy difficult to even get such a small addition committed upstream, but that’s a different story

Re: A routine gem update ended up creating $73k worth of subscriptions

#105

Post by kortex »

warpech wrote: post #7
warpech wrote: post #6

Mongoid docs[1] seem to be pretty cool about this change: "As of Mongoid 7.1, logical operators (and, or, nor and not) have been changed to have the the same semantics as those of ActiveRecord. To obtain the semantics of or as it behaved in Mongoid 7.0 and earlier, use any_of which is described below." Is it just me or is this one of the most terrible breaking changes in a popular, official library ever? [1] https://…

The SerpAPI blog author seems cool about it, too. After such a problem, I would roll back and never ever update this dependency again.

Yep, that gets a pin to "==x.y.z # pinned to prevent BREAKING CHANGE, do not update without reading " and also some defensive regression tests on their api lest someone update by mistake.

Re: A routine gem update ended up creating $73k worth of subscriptions

#106

Post by kingcharles »

yholio wrote: post #35

Here's how true professionals would handle this: "We recently became aware of some erroneous subscription renewals made by our platform and traced the root cause to a major bug in downstream database technology affecting a very small number of accounts. Nonetheless, we working hard with our database provider to resolve the issue. In the meantime, if you are affected and believe you might have an unsolicited subscript…

Actually had this recently when helping out a friend who was targeted by a shady debt collection company. The only way their lawyer would speak to us is through fax, in order to try to act as a barrier to anyone actually trying to dispute their bullshit.

Re: A routine gem update ended up creating $73k worth of subscriptions

#107

Post by rajin444 »

mickotron wrote: post #100
mchusma wrote: post #95

Stripe also bears some responsibility here, as they don't support production testing, so it's impossible to have a test suite checking for charge related behaviors in production. If you use stripe, please contact them and request this long overdue feature. (I do not think that is the primary issue, but it does not help)

Why not test with a stub? Issue would have been picked up in testing even without Stripe integration.

Yeah, the gem library changes are pretty brutal but this 100% should have been caught by a test.

Re: A routine gem update ended up creating $73k worth of subscriptions

#108

Post by whalesalad »

Whenever I am writing complex/transactional routines that can have grave consequences such as this (billing logic) I try to do a few things:

1. Build it out as a standalone operation/class/module so it has more ceremony around it, while also being limited in scope and easy to audit.

2. Continue to utilize the DB/ORM's filtering/querying to grab data as was done in this case

3. Additionally, when it comes down to performing the big io/side-effect, ensure additional checks are in place that confirm the data I am working on matches the query that was used to ask for the data. So in each customer loop I might double check that this customer is indeed needing an upgrade/etc. It can be slower, but it is worthwhile.

Re: A routine gem update ended up creating $73k worth of subscriptions

#109

Post by LAC-Tech »

Breaking change on minor version changes. Ouch.

Not a MongoDB expert, but you don't really need an 'ORM'[0] do you? I thought it spoke JSON natively like couch.

If I was in this project I might have argued strongly for just doing that. Others might have argued back telling me that we can't possibly send JSON to a thing that expects JSON that's too low level, let's rely on this library by some guy instead.

Sorry, flashbacks. Look libraries that do stuff for us are great. But let's make sure they're worth the cost of admission.

[0] yes I know it's not really an ORM because it's not a relational store but you know what I mean

Re: A routine gem update ended up creating $73k worth of subscriptions

#110

Post by kortex »

rubyist5eva wrote: post #84

Semantic Versioning is bullshit. Even "backward compatible features or bug-only fixes" can completely break your application. With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody. -- Hyrum's Law

That's not really the point. There's a gulf of difference between "we patched a CVE and now spacebar-heating is broken" and "we knowingly and deliberately changed the semantics of a query operator in a minor/patch release in a 7.x library in wide production with 3,900 stars on gitub."

Semver doesn't mean "guarantee no breaking changes in minor" - that's impossible - and people love to point this out for whatever reason. It totally misses the point. I think most engineers have a decent intuition most of the time about whether a change is breaking or not.

Post reply on HN