Go + Services = One Goliath Project • Hacker News Board

Re: Go + Services = One Goliath Project

#221

Post by dangoor » Sun, Dec 22, 2019, 2:18 AM UTC

peterwwillis wrote: post #14

So, some potential pitfalls: - The decision seems to be primarily a software architecture one, without much mention of all the other architects whose input will shape how the finished product is run and supported. In a modern software development environment, all the other parts of the org should be consulted on greenfield work to "Shift Left" anything that may need to change down the pike. Design in a silo leads to…

I don't disagree with your pitfalls, but I do think we're working to avoid them.

To your first point, "The decision seems to be primarily a software architecture one...", this project has had involvement of the whole engineering team since the beginning. The whole org is on board with this change. It's definitely not happening in a silo.

> This is definitely the second system effect. It's going to take years to make this thing reliable and sunset the old product.

I hope not, but obviously we're not done yet, so I can't say how long it will end up taking to completely decommission the Python 2 app. What I can say is this: there are aspects to this project that are _simplifying_ our system and, for what's left moving from Python to Go, our intention is to port the business logic as close to a straight up port as we can get.

> - They're porting over the logic? Even if this is actually the right move, wouldn't a clean-room implementation potentially give better outcomes?

_That's_ second system effect, to me. We can't change everything and fix every problem now, so we're focusing on the changes that will help us move from Python to Go faster.

> - Why are they continuing to use App Engine if the writing's on the wall for 2024?

I don't think Google Cloud is disappearing in 2024, for one. Beyond that, again, we're not changing everything about our architecture. The way our data is stored is staying the same.

Re: Go + Services = One Goliath Project

#222

Post by novok » Sun, Dec 22, 2019, 2:19 AM UTC

timwaagh wrote: post #17

Seems like such a waste. Is switching to python 3 really that hard? Is hardware that expensive? If this is indeed the right call it doesn't bode well for traditional scripting languages as the web scales to fewer high traffic apps. We might start to see more jvm, go (apparently) or even rust and c(++), rather than speed of development languages like Python or Ruby. Trend seems to be the reverse though, with python th…

In my experience it's more annoying than you think, especially if you want to prevent small sneaking regressions.

Python not being statically typed also means all of the breaking type changes they made with string now being byte or similar means crashes are not revealed until possibly production if your unit tests didn't get that oneeeee edge case right.

If I'm going to do a new project in the future, I will demand it will be a statically typed language. I'm sick of dynamic languages.

Re: Go + Services = One Goliath Project

#223

Post by bb88 » Sun, Dec 22, 2019, 2:19 AM UTC

dangoor wrote: post #182

freyr wrote: post #47

Earlier quoted context omitted.

> it's similarly difficult to switch to python 3 But that's almost certainly not the case. Even if their existing codebase relies very heavily on the small subset of Python2 features that require manual porting, switching to Python3 will be much less work than rewriting everything in a new language.

The Python 2 to 3 conversion was only part of it. The libraries were a huge difference for us. I will agree that porting to Python 3 would be less work than porting to Go, but we believe that the difference is less than people would suspect.

Never announce a plan to do something. Announce the results after doing it.

If you announce a plan, you're making a bet (sometimes with your reputation) that's it's going to be successful, and telling the world about it.

If however, you didn't announce it, you could still get the benefit of success, or make it looks like learning the lessons of a failed experiment.

Re: Go + Services = One Goliath Project

#224

Post by pushrax » Sun, Dec 22, 2019, 2:20 AM UTC

tybit wrote: post #128

As much as I personally don’t enjoy writing Go I really can’t fault them. I still find it interesting that for a relatively obvious feature set of fast compiles, fast startup and fast runtime there really isn’t anything mainstream out there to compete with Go. I really hope something like Kotlin, Swift, ReasonML or even AOT JVM/.NET brings something to the table soon. Or perhaps I’ll just have to wait for WASM to rea…

How does WASM fill this gap?

> fast compiles, fast startup and fast runtime ... mainstream

C is this but it's harder to write secure/correct C, the standard library is smaller, and there's no canonical toolchain in the same way as Go.

Re: Go + Services = One Goliath Project

#225

Post by rraghur » Sun, Dec 22, 2019, 2:22 AM UTC

osdev wrote: post #22

purple-again wrote: post #9

We turned our monolith into a bunch of micro services almost 6 years ago to the day. For a long time I was very happy with the new pattern but over the years the weight of keeping everything updated along with the inevitable corners that fall behind and have...questionable..security due to how long they sit neglected has really left me wondering if I am happy with it after all. I would love hear some thoughts from ot…

Like many things in life, there are no absolutes. So rather than going to opposite ends of the spectrum, check out modular monoliths. This approach is a nice balance (IMHO), since you start off with monolith but it is broken down into cleanly separated modules. Each module can potentially become its own micro-service if or when the time comes. In terms of implementation, this can be easily done, for example we do it…

Do you also break up the data for each service up front or do you pull that out later?

Re: Go + Services = One Goliath Project

#226

Post by Solar19 » Sun, Dec 22, 2019, 2:22 AM UTC

Interesting read. It always surprises me that companies go many years running major apps and infrastructure with interpreter-based languages like Python and Ruby to begin with. It's an incredible waste of energy, compute, and for web apps sometimes, users' time.

Developers need to be a lot more disciplined about performance and efficiency. I'm glad Khan went to Go, but man all those years wasted.

Re: Go + Services = One Goliath Project

#227

Post by coldtea » Sun, Dec 22, 2019, 2:27 AM UTC

Thaxll wrote: post #178

coldtea wrote: post #44

Earlier quoted context omitted.

Why would Go be more performant than Kotlin? The JVM has been fine-tuned for 3 decades (JIT and GC) whereas Go is pretty primitive in a lot of its backend.

See for yourself: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

See what? This is an unofficial benchmark game - and based on implementations people bothered to provide. Not some scientific test, and nothing that guarantees these are the best implementations. And even if they were they're not representative of server/long running program behavior (where JIT quality and especially GC implementation) matters.

Even so, the page linked starts with "Back in April 2010, Russ Cox charitably suggested that only fannkuch-redux, fasta, k-nucleotide, mandlebrot, nbody, reverse-complement and spectral-norm were close to fair comparisons".

Since Russ Cox is one of the Go co-developers, lets see the ones he accepts are fair. Of those 5 are present in the page, on 3 of which Java wins with decent margins (fannkuch-redux: 18%, k-nucleotide: 21%, reverse-complement: 17.5%) and on 2 of which Go barely comes ahead (fasta: 6.3%, spectral-norm: 6%).

Re: Go + Services = One Goliath Project

#228

Post by geodel » Sun, Dec 22, 2019, 2:30 AM UTC

coldtea wrote: post #44

Possiblyheroin wrote: post #2

I was under the impression that Go was more performant than Kotlin. TMYK :)

Why would Go be more performant than Kotlin? The JVM has been fine-tuned for 3 decades (JIT and GC) whereas Go is pretty primitive in a lot of its backend.

With the amount of memory Java programs consume, no wonder Java GC is getting fine tuned for 3 decades and counting.

Re: Go + Services = One Goliath Project

#229

Post by solidasparagus » Sun, Dec 22, 2019, 2:31 AM UTC

dangoor wrote: post #211

purple-again wrote: post #9

We turned our monolith into a bunch of micro services almost 6 years ago to the day. For a long time I was very happy with the new pattern but over the years the weight of keeping everything updated along with the inevitable corners that fall behind and have...questionable..security due to how long they sit neglected has really left me wondering if I am happy with it after all. I would love hear some thoughts from ot…

Our approach to services at Khan Academy is likely a bit different from most. We're sticking with a monorepo (the code for all services lives in one repository). We have a single go.mod file at the top of the repo, so all services use the _same versions_ of dependencies. We're still building out our deployment system to better support multiple services, but we're planning to redeploy all of the services when library…

I don't really understand moving to microservices if you aren't going to give the service teams the autonomy to make their own decisions and move at their own pace. Microservices always seemed to me to be more of an organizational strategy than a technical one - if you have services, but they can't operate independently, it feels a bit like you are just creating a distributed monolith.

Re: Go + Services = One Goliath Project

#230

Post by Alex3917 » Sun, Dec 22, 2019, 2:31 AM UTC

dangoor wrote: post #170

Alex3917 wrote: post #123

Earlier quoted context omitted.

I worked on a fairly large codebase that needed to be rewritten from scratch when migrating from 2 to 3, primarily because all the tests were written using a test framework that was no longer maintained. So given that you might need to start over anyway, I think it's reasonable to consider other options. That said, yeah it's difficult to understand how KA's web server costs aren't already basically zero, and how thei…

> That said, yeah it's difficult to understand how KA's web server costs aren't already basically zero, and how their endpoints aren't already basically instantaneous. I find that this is the outsider's view of a great many products. Things always seem a lot simpler on the outside. In Khan Academy's case, I think a lot of folks just think of our site as being a collection of more-or-less static pages with videos on t…

Fair, but what percentage of spend is actually on web servers as opposed to database, data transfer, static asset storage, caches, CDN, etc? The features you listed are kind of what I expected, but I still wouldn't expect the web servers to be more than 15% or so of your hosting costs. I know you guys get a ton of traffic, but on most web sites at least 90% of traffic is logged out and doesn't even need to hit the web servers in the first place.