Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go + Services = One Goliath Project

engineering.khanacademy.org

Post reply on HN
Print view View original

281–290 of 449 posts

Re: Go + Services = One Goliath Project

#281

Post by bhntr3 »

> If we moved from Python to a language that is an order of magnitude faster, we can both improve how responsive our site is and decrease our server costs dramatically.

I see people say things like this a lot but my experience is that while other languages are 10x or more faster than python in some benchmarks it's very rare that computation time dominates server latency or that servers are running at 60%+ cpu across all cores.

If 90% of your service latency is not directly on the cpu and/or you haven't profiled to see that the performance bottleneck is evenly distributed across all tasks, then it's super dangerous to migrate to a new language thinking that will fix it.

I hope people inside Khan Academy know this and it's just a clickbait blog. If they really think "go is 10x faster than python so we'll only need 1 server for every 10 when we migrate" then I think they'll be disappointed.

Re: Go + Services = One Goliath Project

#282

Post by twblalock »

purple-again wrote: post #9

We turned our monolith into a bunch of micro services almost 6 years ago to the day. For a long time I was very happy with the new pattern but over the years the weight of keeping everything updated along with the inevitable corners that fall behind and have...questionable..security due to how long they sit neglected has really left me wondering if I am happy with it after all. I would love hear some thoughts from ot…

In my experience, the biggest benefit of microservices is decoupling teams.

Developer productivity is very hard to maintain in a monolithic app as the number of developers increases and the legacy code piles up. Breaking up services and giving each dev team control over their own codebases enables them to develop their own products at their own pace.

If you only have one dev team, microservices are a lot less attractive. However, there are still some benefits, such as being able to refactor parts of your codebase in isolation (including perhaps rewriting them in different languages), and the ability to individually adjust the runtime scale of different parts of your codebase.

Re: Go + Services = One Goliath Project

#283

Post by speedplane »

guitarbill wrote: post #167
ram_rar wrote: post #99

Earlier quoted context omitted.

Switching to python3 is hard, especially if you have a massive python codebase interacting with other systems. Its not as easy as importing unicode_literals. Unicode breaks in very subtle ways.

and rewriting it in another language will magically fix this and not introduce new bugs? besides, in my experience Python 3's clear separation between bytes and str makes these breakages much less subtle than it silently going wrong in Python 2. i wish them all the best, but would've been much more impressed if they'd done it, not simply announced to do it.

> in my experience Python 3's clear separation between bytes and str makes these breakages much less subtle than it silently going wrong in Python 2.

No question Python 3 is better than 2, but it's not better enough to justify the move. People will only move when they absolutely have to. That isn't progress, it's inefficiency.

Re: Go + Services = One Goliath Project

#284

Post by stoicShell »

guitarbill wrote: post #261
stoicShell wrote: post #231

Earlier quoted context omitted.

That will make for some pretty salient readings, thanks for the links. There are obviously cases of failure. I don't know the stats. I'm speaking mostly of small/medium businesses, where size and complexity are different.

that's fair, most of those are "you get what you pay for" and hardly the fault of a competent, well-meaning, but naive dev team. problem is it's much harder to get data on small businesses. i think one thing we can agree on is that the only thing users hate more than change is breakage. rewrites are a valid tool in a long-term strategy, just as debt is for finance. but for most people, incremental change has a bigger…

I think we agree on the general perspective, yes — we'd probably agree as a team with known (ad hoc) conditions and goals.

Data on small businesses is hard indeed. I'm only speaking anecdotally from the MSP / software shops perspective, the "tech guys" of most businesses who don't in-house IT. Also from a European perspective, so that might make a big difference — we're, ahem, let's say not as involved, interested, or capable in all things "technology" as a general population (I didn't say luddites but that's how it feels sometimes, compared to the vibe I get in NYC or rich Asian cities).

Re: Go + Services = One Goliath Project

#285

Post by ddxxdd »

wonjohnchoi wrote: post #272

It is crazy to see they are still using python 2. Seeing how slow the conversions to python 3 have been, was creating python 3 a good decision for python community? Can it be argued that developing python 2 further in a backward compatible way would have been better for the community? I know that evaluating this kind of thing is hard as metrics are bound to be subjective and speculative. But I am curious if there was…

I remember a long time ago, I used the "six" library (Python (2 * 3) ) to code in Python 2 with an API based on Python 3.

I'm not entirely sure how well the line of code: "import six" would fix any compatibility issues nowadays.

Re: Go + Services = One Goliath Project

#286

Post by apta »

> Go, however, used a lot less memory, which means that it can scale down to smaller instances.

I could be mistaken, but this sounds like they went ahead with the default JVM settings, where it tends to use as much memory it is allowed to (which makes sense from a utilization and efficiency perspective). If memory usage is a concern, the JVM can be tuned for such.

Re: Go + Services = One Goliath Project

#287

Post by pm90 »

bhntr3 wrote: post #281

> If we moved from Python to a language that is an order of magnitude faster, we can both improve how responsive our site is and decrease our server costs dramatically. I see people say things like this a lot but my experience is that while other languages are 10x or more faster than python in some benchmarks it's very rare that computation time dominates server latency or that servers are running at 60%+ cpu across…

* It’s not just that Go is faster to run but also faster to iterate on. If python can be neither, its offering little benefit.

* they moved from a monolith to a microservices architecture; concern that any of the services in the request path could add latency just because of the overall runtime speed is slow is a legitimate one.

* their primary deployment method is Google App Engine where you are billed by CPU used. Any change that consumed less CPUs has a tangible effect on their costs

Re: Go + Services = One Goliath Project

#288

Post by phumbe »

dangoor wrote: post #211
purple-again wrote: post #9

We turned our monolith into a bunch of micro services almost 6 years ago to the day. For a long time I was very happy with the new pattern but over the years the weight of keeping everything updated along with the inevitable corners that fall behind and have...questionable..security due to how long they sit neglected has really left me wondering if I am happy with it after all. I would love hear some thoughts from ot…

Our approach to services at Khan Academy is likely a bit different from most. We're sticking with a monorepo (the code for all services lives in one repository). We have a single go.mod file at the top of the repo, so all services use the _same versions_ of dependencies. We're still building out our deployment system to better support multiple services, but we're planning to redeploy all of the services when library…

Sounds similar to some work I’ve been doing, so thanks for unknowingly validating my design!

At my employer, I’m spearheading a wholesale reimplementation of outdated process automation software, turning everything into Django web apps.

I’ve been working with a monorepo and monolithic deployments to maintain development velocity but recently started transitioning the CI/CD pipeline to deploy each application/service in the monorepo independently. The pipeline packages common assets (including, e.g., manage.py, common HTML templates, and the dependency spec...all housed in the same monorepo) into each app directory before the deploy stage.

Meanwhile, local developers clone the entire monorepo, and when they launch localhost, all of the services come online simultaneously. (That’s the goal, at least!)

I was already excited to see my work come to fruition, and now I’ll be keeping an eye on Khan Academy, too!

Re: Go + Services = One Goliath Project

#289

Post by speedplane »

wonjohnchoi wrote: post #272

It is crazy to see they are still using python 2. Seeing how slow the conversions to python 3 have been, was creating python 3 a good decision for python community? Can it be argued that developing python 2 further in a backward compatible way would have been better for the community? I know that evaluating this kind of thing is hard as metrics are bound to be subjective and speculative. But I am curious if there was…

> was creating python 3 a good decision for python community? ... I know that evaluating this kind of thing is hard

I don't think there is any question here: Python 3 is a complete disaster. Years and years of engineering effort wasted on changing string libraries. Sadly, the Python leadership refuses to acknowledge the failing, perhaps because such an acknowledgement would challenge their omnipotence ... it would, and it should.

Re: Go + Services = One Goliath Project

#290

Post by joshuamorton »

lesserknowndan wrote: post #214
joshuamorton wrote: post #164

Earlier quoted context omitted.

This doesn't follow: for loops are there specifically for cases of the complexity that can't be handled by other constructs (like comprehensions). Anonymous functions are not, for the complex cases you give the construct a name. It's a very intentional language limitation, much as semantic whitespace is an intentional limitation. And there is actually a language difference. A for loop is a statement. A lambda is an e…

I tentatively disagree with both of you. I've sometimes wondered whether code would be universally clearer if all you could do in a loop is have a one-liner or call another function/method (mainly when looking at my own code within loops and thinking WTF!!!). Would probably cause issues when teaching programming though. It would be interesting to have a compiler switch that enforced this...

I actually like this style (or similar if not quite as extreme). I find it's much clearer what's going on.
Post reply on HN