Go + Services = One Goliath Project • Hacker News Board

Re: Go + Services = One Goliath Project

#421

Post by dangoor » Sun, Dec 22, 2019, 6:08 PM UTC

AnonymousPlanet wrote: post #374

harel wrote: post #369

Migration from python 2 to 3 is easy and fast. I've migrated multiple large apps and it took about a day each. Most libraries that matter have been migrated. Some don't even support python 2 anymore. It's practically 2020. This should not even be a consideration. After 2 to 3 is done they should consider again If they want to redo the stack but first I'd focus on this small maintenance task.

Hahaha, sorry but this is a very cute thing to say, in my view. At our company we just barely finished migrating our software with nearly a million lines of legacy Python 2 code to Python 3. This took over a year of nearly exclusive migration effort, just making our code work with both. The entire migration project started way before I joined the company several years ago. So, no, things are not as simple if you're n…

FWIW, our codebase is of similar size to that and we estimated it would take around a year to migrate, which is why spending a bit more time and ending up with a Go-based system on the other end was appealing.

Re: Go + Services = One Goliath Project

#422

Post by dangoor » Sun, Dec 22, 2019, 6:12 PM UTC

plinkplonk wrote: post #322

just trying to understand - you guys think moving a Python2 monolith to Python 3 is too painful, and so you are going to port all the code from Python2 to a completely new language (Go), change the architecture (monolith -> microservices) and move the HTTP API to React + GraphQL, all in one year? 2020 is going to be in an interesting year at Khan Academy ;-)

2020 is absolutely going to be an interesting year.

One thing that might not have come across clearly in the blog post: we're already well on the path to using React everywhere (we started using React a week after it became public… 6 years ago?). We made the decision to move to GraphQL in 2017, so we've already got a lot in our GraphQL schema. Finishing those switchovers will make our move to Go happen more quickly.

Re: Go + Services = One Goliath Project

#423

Post by harel » Sun, Dec 22, 2019, 6:14 PM UTC

StavrosK wrote: post #379

AnonymousPlanet wrote: post #374

Earlier quoted context omitted.

Hahaha, sorry but this is a very cute thing to say, in my view. At our company we just barely finished migrating our software with nearly a million lines of legacy Python 2 code to Python 3. This took over a year of nearly exclusive migration effort, just making our code work with both. The entire migration project started way before I joined the company several years ago. So, no, things are not as simple if you're n…

It very much depends on how good the codebase is. I also spent a year on and off porting a large codebase from 2 to 3, and it would have gone an order of magnitude faster if the codebase were in better shape.

The two projects i migrated were good quality code at the hundreds of thousand lines of code scale. Though I find that metric a bit inappropriate. I agree the statement was a bit generalising and not appropriate for every project and the million lines projects should have been excluded To be honest - I find a year long dedicated migration effort a bit excessive. But who am I to judge. My experience was smooth with just a few hurdles around byte/string issues and that was it.

Re: Go + Services = One Goliath Project

#424

Post by dangoor » Sun, Dec 22, 2019, 6:16 PM UTC

conradfr wrote: post #370

plinkplonk wrote: post #322

just trying to understand - you guys think moving a Python2 monolith to Python 3 is too painful, and so you are going to port all the code from Python2 to a completely new language (Go), change the architecture (monolith -> microservices) and move the HTTP API to React + GraphQL, all in one year? 2020 is going to be in an interesting year at Khan Academy ;-)

At least the article makes it seem it's not a decision taken on a whim and they did some kind of POC and planned the transition. Of course the obvious thing missing in the article is how they expect to deliver new business features while recoding everything in a new language. It's fun to read this and the Etsy thread currently on the frontpage as well.

> Of course the obvious thing missing in the article is how they expect to deliver new business features while recoding everything in a new language.

For new features, the new parts of the GraphQL schema for those features will be written in Go as part of the new services. Our frontend is already in large part a single page app in React which requests data via GraphQL, so the frontend for the features will look just the same as it would on our monolith.

Re: Go + Services = One Goliath Project

#425

Post by dangoor » Sun, Dec 22, 2019, 6:22 PM UTC

bhntr3 wrote: post #281

> If we moved from Python to a language that is an order of magnitude faster, we can both improve how responsive our site is and decrease our server costs dramatically. I see people say things like this a lot but my experience is that while other languages are 10x or more faster than python in some benchmarks it's very rare that computation time dominates server latency or that servers are running at 60%+ cpu across…

Sorry if this wasn't clear: Go is 10x faster than Python, yes, but we know that we're not going to reduce our server count by 90%. 50% is quite possible, though, given Go's superior threading and its good resource use. Moving away from the monolith should also give us new optimization possibilities.

Re: Go + Services = One Goliath Project

#426

Post by dangoor » Sun, Dec 22, 2019, 6:26 PM UTC

d_burfoot wrote: post #392

I heard about this project from a friend who works at KA. I am concerned about the strategy, and I think the following approach would yield better results: 1. Write in Go an exact reimplementation of the current Python codebase. Use the same database schema, front-end HTML/JS, test suite, and so on. To whatever extent possible, use the same names for classes and functions. Check the reimplementation correctness by us…

It looks like I need to write a followup with more details about how our migration looks!

Since we're using GraphQL federation, we have a gateway which serves up our complete GraphQL schema, pulled together from a collection of services behind the gateway. We can move individual properties over from our Python monolith to new Go services and the clients will never know. Plus, we can do side-by-side testing in the gateway by making a request to the monolith and to the new service and comparing the results (something we don't have yet, but plan to).

This is definitely not a big bang rewrite. It's about as incremental as it can be, because we can move individual GraphQL properties over and the gateway stitches the result together.

Re: Go + Services = One Goliath Project

#427

Post by dangoor » Sun, Dec 22, 2019, 6:29 PM UTC

sethammons wrote: post #339

I've been in a similar boat. We've been splitting up or converting large Python 2.6/2.7 applications into Go services (and doing the same to large Perl applications) for a long time now. Go has consistently been 10-20x performant (allowing for dramatically reduced hardware needs), easier to maintain, and more productive to produce code in than our previous Python (Twisted) and Perl (AnyEvent). Hopefully KhanAcademy h…

Does sound similar!

We have established the rule that each service owns its own data.

We've already got Stackdriver set up to give us distributed tracing and have set up standards around logging.

Re: Go + Services = One Goliath Project

#428

Post by dangoor » Sun, Dec 22, 2019, 6:33 PM UTC

todd3834 wrote: post #319

> We’ll only generate web pages via React server side rendering, eliminating the Jinja server-side templating we’ve been using I’ve been down this road. Deep down this road. Let me just give you a heads up on something I didn’t consider at the time: Most template languages do not parse every single node, one by one. In a sense they are just doing string concatenation. Not so with server side rendering and React. I’m…

Thanks for the suggestions and the offer to chat!

We've been doing SSR for quite a while now and are improving our CDN use as we go along. We already took steps to ensure that there's no user-specific information showing up in our server-side react rendering which would damage cacheability.

Our frontend infrastructure team essentially owns the React render server. I'll let them know you offered to chat.

Re: Go + Services = One Goliath Project

#429

Post by pm90 » Sun, Dec 22, 2019, 6:38 PM UTC

andy_ppp wrote: post #356

pm90 wrote: post #337

Earlier quoted context omitted.

I can easily ask the vice versa of such a bland question. Answer that first if you want a real answer instead of trying to flame.

It’s not bland it’s direct. What you’re doing is conflating personal preference with actual language features. Most developers who aren’t us would say python, being a higher level, dynamic scripting language rather than Golang which is lower level and extremely explicit about the data your program is using. Iteration in go is simply harder as you have to be more explicit rather than sketching something out. Changing…

It’s funny that none of the things you mention in your last paragraph are handled any better in Python. Without type safety, you instead have to deal with bugs caused by inane mistakes. If we’re talking about generics, the conversation has shifted from “writing scripts that does x” to “maintaining a production system” and for the latter, golangs type safety, out of the box excellent default tooling and easily grokkable concurrency primitives make it far more maintainable than Python.

Things are also easier to change in go because of the type system and interfaces. The former catches most the obvious incompatibilities, the latter ensures that abstractions don’t leak across different system boundaries; whereas in Python there is a tendency to pass a do everything objects across the system.

Error checking has improved substantially with error wrapping in go 1.13. Not only can you locate precisely where your system failed; you have to be explicit about handling errors. I do concede that pre error wrapping the error handling was garbage.

Re: Go + Services = One Goliath Project

#430

Post by dangoor » Sun, Dec 22, 2019, 6:39 PM UTC

mjpuser wrote: post #361

I think the real problem is that they didn't properly maintain their code. Rewriting it in Go won't prevent them from dealing with this in a few years for when this Go version reaches end of life. I would have liked to see an article on "introducing process" side of programming.

Thanks for the comment! A couple of things about this...

1. the Go team is working very hard to ensure that there are no such compatibility issues. Code written for Go 1.0 should still compile with Go 1.14 beta today.

2. It's possible there's more we could have done along the way, and I tend to think that statically typed languages make it easier to safely refactor more ruthlessly. But I do think we've actually done quite a bit of change incrementally along the way. Our move to React on the frontend and GraphQL on the backend have been good examples of that. Plus, we did a huge refactoring a couple of years ago to draw better boundaries in our monolith, and that has made a move to services possible.