How Shopify Manages API Versioning and Breaking Changes • Hacker News Board

Re: How Shopify Manages API Versioning and Breaking Changes

#61

Post by pilom » Thu, Dec 19, 2019, 7:09 PM UTC

pc86 wrote: post #53

mr__y wrote: post #29

Earlier quoted context omitted.

>those apps continue to be updated and utilizing the latest features. while this is orthogonal to Shopify API or your post at all, since you mentioned the need of updates, I just wanted to use that opportunity to vent my frustration with the constant push to update everything all the time and judging any piece of software by using "when was the last update" as a metric. The problem I see is that not all apps or libs…

Well first of all I think it's a straw man to imply that anyone would want to send emails or share to Facebook out a ping utility. That's sort of a big thing that makes utilities different than applications . Edit: I deleted a couple sentences and realize now this might not convey exactly what I meant. Utilities are easy to call "done," applications are not. Applications interact with external forces who do change co…

I think the parent is saying that API's should be utilities and not applications.

Re: How Shopify Manages API Versioning and Breaking Changes

#62

Post by sbilstein » Thu, Dec 19, 2019, 7:12 PM UTC

pc86 wrote: post #53

mr__y wrote: post #29

Earlier quoted context omitted.

>those apps continue to be updated and utilizing the latest features. while this is orthogonal to Shopify API or your post at all, since you mentioned the need of updates, I just wanted to use that opportunity to vent my frustration with the constant push to update everything all the time and judging any piece of software by using "when was the last update" as a metric. The problem I see is that not all apps or libs…

Well first of all I think it's a straw man to imply that anyone would want to send emails or share to Facebook out a ping utility. That's sort of a big thing that makes utilities different than applications . Edit: I deleted a couple sentences and realize now this might not convey exactly what I meant. Utilities are easy to call "done," applications are not. Applications interact with external forces who do change co…

100%

Companies may have really good reasons to break old APIs. I’ve done this before...retiring an old framework and set of APIs that was inherently much less secure than what the bulk of customers needed.

Re: How Shopify Manages API Versioning and Breaking Changes

#63

Post by zerkten » Thu, Dec 19, 2019, 7:24 PM UTC

setr wrote: post #33

keithwhor wrote: post #25

Earlier quoted context omitted.

I don’t disagree with your assessment on semver as it currently exists for, say, NPM packages. I do think that with web APIs specifically, the surface area is a lot smaller — the HTTP interface is literally all you touch — so semver, in its purest form, is actually completely enforceable as long as you understand the API schema. Our team has talked a lot about either hard-enforcing or automatically applying semver wh…

Tbh, it doesn't matter. But the semantics should be such that it doesn't matter -- the user of the API shouldn't care whether it's implementation bounded or not. But Everytime you break that abstraction, trust in the abstraction is necessarily reduced. Bounding to implementation is just the easiest way out -- if your policies, tests and protocols consistently fail to uphold that abstraction, then the you can fallback…

How would your trust be impacted if there was better communication ahead of a breaking change you could almost agree with? i.e. the experience has been well beyond that which you currently expect and have been provided good reasons proactively.

The gap I see a lot starts with a bad versioning strategy and continues through bad API analytics to bad communications for developers.

The API provider should be able to understand all of the individual clients to the point where they can decide to progressively migrate them one-by-one, if necessary. There shouldn't be a need to use metadata like IP addresses or user-agents to try and identify the clients. Unless someone has walked through these scenarios ahead of time, it's difficult manage when issues arise.

Re: How Shopify Manages API Versioning and Breaking Changes

#64

Post by emilsedgh » Thu, Dec 19, 2019, 7:44 PM UTC

Waterluvian wrote: post #43

asdkhadsj wrote: post #36

Earlier quoted context omitted.

What does a codebase look like with these API versioning exactly? I feel like code bloat would be hard to maintain. Obviously I could take two endpoint handlers that have the same functionality and move their behavior into a shared function, but there's also tests to consider. As time went on, I'd have a pile of tests running against seemingly random versions. Eg, `test_blog_create()` might test v1 and v2 of the API…

Versioning is hard. Versioned APIs are very hard. I'm in an especially tricky spot because we've got customers that might refuse to upgrade our product for years and years, if ever. Two approaches I've taken, with a blogpost worth of thoughts on them: 1. Every version is a whole new database/webserver stack. You shut the oldest one down when the last customer migrates off it. The bloat is real, but it's dead simple a…

Every version is a whole new database/webserver stack. You shut the oldest one down when the last customer migrates off it. The bloat is real, but it's dead simple and it works.

That means data is different between API versions?

Re: How Shopify Manages API Versioning and Breaking Changes

#65

Post by Corazoor » Thu, Dec 19, 2019, 8:14 PM UTC

asdkhadsj wrote: post #36

Waterluvian wrote: post #23

Earlier quoted context omitted.

I don't think there's one right answer, but my opinion is: - Treat APIs as immutable. - Any mutation results in a wholly new API version, not a patch or minor update. - The developer will learn what changed, and how much changed by reading the patchnotes, not looking at which semver numbers changed. This is probably a healthy practice to encourage. - Don't change APIs so much. The interface should be very carefully d…

What does a codebase look like with these API versioning exactly? I feel like code bloat would be hard to maintain. Obviously I could take two endpoint handlers that have the same functionality and move their behavior into a shared function, but there's also tests to consider. As time went on, I'd have a pile of tests running against seemingly random versions. Eg, `test_blog_create()` might test v1 and v2 of the API…

There is definitely multiple ways to handle that, but splitting your API into frontend and backend seems to be a common and good approach. The backend is basically the "real" internal API, subject to changes. The frontend consists of nothing more than forwarding functions, that take their input from the outer world and translate it to backend API calls. How you organize Code reuse depends on the language, but inheritance is not a bad way to do it, although you might run into trouble with the size of your hirarchy. You can circumvent that with maps of functions/lambdas, like prototyping in Javascript.

For testing you then have your usual set of tests for the backend, but for the frontend you just test if the translation between public and internal API is correct. Very easy to test for in most cases. If you then organize your tests exactly like your frontend Version, i.e. some kind of inheritance, you only ever have to add tests for the changes in your new Version.

It is definitely more work than just having one ever changing API, but it's not that much of a hassle imho.

Re: How Shopify Manages API Versioning and Breaking Changes

#66

Post by dwild » Thu, Dec 19, 2019, 8:24 PM UTC

mr__y wrote: post #29

Judson wrote: post #3

It's interesting to compare and contrast this method of API management with Stripe. As far as I understand, the Stripe api would continue to work indefinitely so long as you lock your api version, whereas Shopify would eventually break the app as they essentially backport breaking changes to older api versions. Initially, I thought Stripe's method was superior, and would provide the best API experience, but realized…

>those apps continue to be updated and utilizing the latest features. while this is orthogonal to Shopify API or your post at all, since you mentioned the need of updates, I just wanted to use that opportunity to vent my frustration with the constant push to update everything all the time and judging any piece of software by using "when was the last update" as a metric. The problem I see is that not all apps or libs…

> I just wanted to use that opportunity to vent my frustration with the constant push to update everything all the time and judging any piece of software by using "when was the last update" as a metric.

The update in theses cases are not necessary to keep the most recent feature, but to stay connected to the interface that does change with time.

Someone needs to update the interface between the two while things evolve. Like it or not, but that's need to be done. If the tool is used and like by a few, then why not push that requirement toward the ones theses few that used it and like it, instead of supporting everything, even what's no longer used?

Re: How Shopify Manages API Versioning and Breaking Changes

#67

Post by uyuioi » Thu, Dec 19, 2019, 9:01 PM UTC

The Shopify developer experience is terrible. No fluffy blog can change that fact.

Shopify says the main product is the store owner. But the developers pick up all of the slack of Shopify.

Recurring payments. App. Store backups. App. Theme backups. App. Order editing. Came late 2019. Checkout. So locked down. Where’s the API?! Slate tooling. Abandoned. Starter themes. Abandoned. Storefront SDK. Terrible documentation. More than 1 variant image? App. Metafields. App. Wholesale. App. Mailchimp. Removed.

People talk about google abandoning products. Shopify abandons nearly all developer tooling and is so locked down that it’s a constant “app for that” for the basics.

The interesting thing is theres been more than a few store owners I know that use Shopify. They’ve asked how to move off of Shopify.

I guess fulfilment is more important though. Right?!

Re: How Shopify Manages API Versioning and Breaking Changes

#68

Post by auv5 » Thu, Dec 19, 2019, 9:02 PM UTC

sour-taste wrote: post #49

How do they handle security fixes in old releases? If releasing a security patch requires backporting it to 5 different active releases then I'm unconvinced that this is a useful strategy.

Great question! As mentioned in the article, when making a change you would typically add an `ApiChange.in_effect?` check to see if your new functionality should execute. When we implement security fixes, we do not include this check and the fix retroactively applies to all API versions.

Re: How Shopify Manages API Versioning and Breaking Changes

#69

Post by uyuioi » Thu, Dec 19, 2019, 9:04 PM UTC

xal wrote: post #45

MentallyRetired wrote: post #16

Shopify made a change to their API that was easily measurable on who it would affect, but didn't email us. Refused to grant us a temporary exemption (they would do it for $2000/m they said). The end result? They've sunk my business. I've replaced shopify now by writing my own but it's too late. My customers have all gone to my competitors and we're looking at pivoting.

Can you share more details? I run Shopify and that's not something we would do.

I believe you may have lost a connection to the developer experience over time Tobi and developers keep Shopify relevant. Look at any store it’s packed full of slow loading plugins picking up some of the very basics. It might be that 2020 is better suited to making some of the tooling better and if we can hope for consistent.

Re: How Shopify Manages API Versioning and Breaking Changes

#70

Post by wvh » Thu, Dec 19, 2019, 9:43 PM UTC

mr__y wrote: post #29

Judson wrote: post #3

It's interesting to compare and contrast this method of API management with Stripe. As far as I understand, the Stripe api would continue to work indefinitely so long as you lock your api version, whereas Shopify would eventually break the app as they essentially backport breaking changes to older api versions. Initially, I thought Stripe's method was superior, and would provide the best API experience, but realized…

>those apps continue to be updated and utilizing the latest features. while this is orthogonal to Shopify API or your post at all, since you mentioned the need of updates, I just wanted to use that opportunity to vent my frustration with the constant push to update everything all the time and judging any piece of software by using "when was the last update" as a metric. The problem I see is that not all apps or libs…

One reason would be to make sure somebody's still around. I'd make tiny updates to let dependents know that I'm still watching the code base. Where I work, code is (sadly enough) dead pretty much when workers' contracts or consultant hiring agreements end.

Sporadic updates tell those who depend on the code that stability and security updates are still being handled, even if the code is "complete". I'm not sure I'd have a lot of faith in code that looks abandoned.