Backwards compatibility in Go (2015)

What I was trying to illustrate is, that the notion of versioning is broken in and off itself. By the lessons of the article, pretty much every API change is a breaking change, so you would constantly need to increment the major version, if you take SemVer seriously, meaning minor versions don't exist, de facto. And if you now imagine that you'd need to touch your code every time one of your dependencies increments their minor version (I mean. Likely you can't, because you have no notion of how often that happens, because currently all your tools just ignore those changes), lest people can't build your software because the packaging tool needs to assume an incompatibility that needs manual resolving, you will see how useless this makes versioning.

Now, there are two ways out of this mess: One is to ignore it and just assume that, in practice, some things will are more likely to break consumers than others and just apply a reasonable case-by-case judgement. It's what's happening right now in probably ~every language for ~every tool out there. I think it's reasonable, but I personally dislike it, because for one, humans eff up all the time, so relying on them having a good notion of what breaks and applying it consistently and timely leads to pain. This whole article is born out of the idea, that these things should be codified and then automatically applied, I shouldn't even have to need to know what version my package currently is, IMHO.

The other way out is much more complicated: Transition to a notion of breakage not by versioning APIs, but by defining it in terms of pairs of packages. This also has a bunch of definite and obvious deficiencies (for example that you don't have access to all the code that imports you. Or the combinatoric explosion).

Currently, my personal hope is that this can be solved by supporting gradual code repair (see, e.g. https://github.com/golang/go/issues/18130 for what this means and how this is currently progressing) and then add good tooling (I'm working on something, but I have limited time and brain space). We'll see :)

> It's not a problem in languages that come with a package manager, or at least promote a specific one to handle dependencies.

No. It is pretty much only a problem in languages with a package manager. Languages without the notion of package versioning don't run into the issue of needing to have a notion of what a version is, clearly. A package manager just helps automatically enforce versioning, it does zero to help you figure out what the correct version is.

Yes, not coming with versioning and a package manager comes with its own disadvantages and pain, no doubt. And we can surely debate which pain outweighs or is worse. But claiming that package managers would solve this problem just isn't true (I wrote the article when the whole "SemVer and package manager for go" debate was taking off and I tried to come up with a better approach).

> It's only a problem in languages where developers thinks "you don't need that". There is this weird belief in the Go community that Go is a miraculous language where good programming practices don't apply because Go magically make them irrelevant.

Can you source that? I am one of the most vocal opponents of package managers and versioning in go, but this contradicts everything I believe about it. My believes aren't, that go doesn't need it, but that a) it comes with its own set of problems, that b) those outweigh most of the benefits, IMO and c) that by considering and discussing it carefully, we might be able to come up with an actually innovative new solution to the problems.

Nowhere do I claim that go doesn't have those problems, on the contrary. I'm just arguing that, traditionally, go is a language that prefers doing things right over doing things like they were always done.

> Yes but only in Go.

I don't believe this to be true. I'd say at least most languages/projects would have exactly the same kind of problems (some of them at runtime, some of them at compile time), it's just that in practice they don't really matter. So you don't know about it and it seems, to a casual observer, to work just fine.

> The problem is that Go apparently doesn't let you reliably add things either.

Neither does, at least, C. I'd guess that python also doesn't let you do it. I don't know enough about java, but I'd assume that it has the same problems.

What you need to understand is, that I applied an extremely nit-picky interpretation of stability. It's not like any of the things I mentioned are learned from experience or practical problems; I just understand the go type system enough and thought really hard whether there is an obviously correct and useful interpretation of API stability under its constraints and came up with a (mostly theoretical) result. Most projects in go that use SemVer apply a more generous and naive notion that works well enough in practice. C projects do exactly the same.

> Perhaps the "correct", but cumbersome thing to do is supply different versions of the API when maintaining backward compatibility and never change the old version.

This is the approach taken in good software deployments, specially if you cannot control who the users are.

A good chunk of that is that Go, by design, makes nearly all changes breaking (as you covered pretty well). I doubt they did so intentionally, but they seem to focus their decisions on small-ish scale projects at the expense of large / longer-term ones, and this is the natural consequence.

I think the focus here makes sense, and improves lots of useful things in practice (which is why they do it - Go focuses on pragmatism), I just don't really like it. Every non-side-project I've worked on has chafed under weaknesses that Go seems to embrace, because the code has had to survive and grow for a couple years. It's a sizable step up from Python tho.

As far as ways out of this mess... not sure. A lot of the problems are solved by "hit it until it compiles", which is a good thing, and often implies automated code-rewriting tools are possible. The rest (adding methods -> you may collide with an interface which you didn't before) can probably be detected so you at least have your potential-problems enumerated. There are some fairly sophisticated tools out there for doing both of these, e.g. https://github.com/facebook/codemod , and it'd be great to see more language-communities embrace (and improve) them IMO.

If you manage to limit most of your changes to "can be automatically changed / detected", you have a fair bit more freedom. With Go's limitations... maybe enough? I'd have to read and think a lot harder to figure out if there would be too many things that fall into those gaps.