Go + Services = One Goliath Project • Hacker News Board

Re: Go + Services = One Goliath Project

#411

Post by hu3 » Sun, Dec 22, 2019, 4:39 PM UTC

apta wrote: post #377

kureikain wrote: post #324

Go is a very weird language :-(. It's very limited when you started to do complex thing. Example, let's say you are building websocket. You will have a hard time to write type safe websocket handler to process the payload from client for all the events... I started to do Rust/Crystal and both of them are better than Go(performance, type system). Yet, whenever I build something for work, I come back to Go :-(. I told…

Java and C# provide the benefits you mentioned, while being more expressive languages and having better runtimes compared to golang.

Hardly. OP mentioned fast compilation and limited ways to code the same solution.

C# and Java are slower to compile and offer way more options to do the same thing.

Here's Uncle Bob take on testing with Go: https://m.youtube.com/watch?v=2dKZ-dWaCiU&t=36m40s

Re: Go + Services = One Goliath Project

#412

Post by zo1 » Sun, Dec 22, 2019, 5:05 PM UTC

makapuf wrote: post #354

zo1 wrote: post #43

Earlier quoted context omitted.

This is perhaps a very big misconception with python usage. Just because it has "dynamic" or "duck" typing, doesn't mean that you have to consider chaotic and unpredictable data running through your code paths. It's just not the case. In a typical codebase, it's probably 98% very specific and known data types linked to the variables in your code. With the remaining 2% being things that are just "easier" to solve with…

Re: your last sentence, what is text editor + my so bad ? How does pycharm warrant this definitive advices ?

Honestly, at this point mypy is inferior to the built-in PyCharm when it comes to speed, integration and in some cases the type-inference as well. I've also compared it to the VS python language server and it too doesn't stack up.

The other metric would be what a person coming from established, full-fledged IDEs such as VS would expect. With PyCharm, you get intellisense almost on-par with what you get from VS for C#/VB, assuming you use type-hints that is.

Not trying to be difficult, but I'm being honest about them providing a really good python experience that is for the most part free. There is no need to putz-around with VScode, json configs, plugins, mypy, etc and still end up getting a relatively inferior experience. Doubly so for new-developers.

Re: Go + Services = One Goliath Project

#413

Post by dangoor » Sun, Dec 22, 2019, 5:19 PM UTC

bb88 wrote: post #275

dangoor wrote: post #234

Earlier quoted context omitted.

We have the good fortune of not having to be super secretive about our tech. We _want_ to talk about this project as we go along and share what we learn. We've already got more interesting stuff to talk about, and it'd be a lot less interesting, I think, without this context for the overall project. As mentioned in the post, a small piece of our GraphQL schema is already in Go running in production. This blog post is…

Just because you can do something doesn't mean you should have done it. And sometimes that's only found out in hindsight.

Indeed, that's true. I just think we have more to gain by talking about what we're doing and seeing what the community thinks about some of our approaches rather than keeping it all private for the next year.

Re: Go + Services = One Goliath Project

#414

Post by dangoor » Sun, Dec 22, 2019, 5:23 PM UTC

vips7L wrote: post #276

dangoor wrote: post #237

Earlier quoted context omitted.

We're making a certain set of tradeoffs. For example, we're not adopting the "write code in whatever language you want" form of microservices that some folks adopt, because we don't feel like we're large enough to support that. Like I said, though, we do want to minimize the library footprint. The vast majority of deploys in this new world will be single service deploys, with the benefits that come with that. We alre…

Didnt you guys recently decide to write some services in kotlin? Are you rewriting those in go now since everything is going to be one language?

We have some dataflow jobs written in Kotlin (we blogged about that in June 2018[1]). We also have an internal service written in Kotlin.

We ideally want one language. But Apache Beam (which is behind Google Dataflow) doesn't yet have production support for Go. More importantly, though, we have no time pressure on switching the Kotlin code over, so that's a long way out.

[1]: https://engineering.khanacademy.org/posts/kotlin-adoption.ht...

Re: Go + Services = One Goliath Project

#415

Post by dangoor » Sun, Dec 22, 2019, 5:24 PM UTC

phumbe wrote: post #288

dangoor wrote: post #211

Earlier quoted context omitted.

Our approach to services at Khan Academy is likely a bit different from most. We're sticking with a monorepo (the code for all services lives in one repository). We have a single go.mod file at the top of the repo, so all services use the _same versions_ of dependencies. We're still building out our deployment system to better support multiple services, but we're planning to redeploy all of the services when library…

Sounds similar to some work I’ve been doing, so thanks for unknowingly validating my design! At my employer, I’m spearheading a wholesale reimplementation of outdated process automation software, turning everything into Django web apps. I’ve been working with a monorepo and monolithic deployments to maintain development velocity but recently started transitioning the CI/CD pipeline to deploy each application/service…

That does sound pretty similar (though our services all have the luxury of serving nothing other than GraphQL!).

Our current plan for local development is to continue cloning the monorepo and firing up all of the services. Go services don't take a whole lot of resources, so we think this plan will work fine for quite a while.

Re: Go + Services = One Goliath Project

#416

Post by vips7L » Sun, Dec 22, 2019, 5:27 PM UTC

dangoor wrote: post #414

vips7L wrote: post #276

Earlier quoted context omitted.

Didnt you guys recently decide to write some services in kotlin? Are you rewriting those in go now since everything is going to be one language?

We have some dataflow jobs written in Kotlin (we blogged about that in June 2018[1]). We also have an internal service written in Kotlin. We ideally want one language. But Apache Beam (which is behind Google Dataflow) doesn't yet have production support for Go. More importantly, though, we have no time pressure on switching the Kotlin code over, so that's a long way out. [1]: https://engineering.khanacademy.org/posts…

Thanks for answering! That was the specific article I remember reading.

Re: Go + Services = One Goliath Project

#417

Post by dangoor » Sun, Dec 22, 2019, 5:34 PM UTC

scottrogowski wrote: post #299

dangoor wrote: post #221

Earlier quoted context omitted.

I don't disagree with your pitfalls, but I do think we're working to avoid them. To your first point, "The decision seems to be primarily a software architecture one...", this project has had involvement of the whole engineering team since the beginning. The whole org is on board with this change. It's definitely not happening in a silo. > This is definitely the second system effect. It's going to take years to make…

> The whole org is on board with this change. It's definitely not happening in a silo. Given the seemingly strong chorus of voices responding with cautionary tales about why you might want to rethink this plan, and the number of engineers in your organization, it seems more likely that you have some dissenting voices who have either been too scared to speak up or have already been shot down.

What I meant by "the whole org is on board" isn't that there weren't other opinions. There have been multiple opinions on almost every decision we make (and we have an open process and document our decisions in a style very much like this one[1]). In the end, it's not about "shooting down" alternatives, since that's loaded language. It's about making what we think is the best choice we can with the information available to us.

Even in this thread, there's a chorus of voices sounding caution based on their limited information of what our situation looks like, but there are others who see why we're doing this, based on the same limited information.

We absolutely do know the risks of this project, which is why we're doing this as incrementally as possible.

[1]: http://thinkrelevance.com/blog/2011/11/15/documenting-archit...

Re: Go + Services = One Goliath Project

#418

Post by dangoor » Sun, Dec 22, 2019, 5:37 PM UTC

mntmoss wrote: post #320

dangoor wrote: post #206

Earlier quoted context omitted.

You might be thinking of this excellent article by Joel Spolsky: https://www.joelonsoftware.com/2000/04/06/things-you-should-... (Ironically, that post is about Netscape and the rewrite, Mozilla, ended up growing as a nonprofit far beyond Netscape's original scale.) The so-risky-it's-almost-certain-to-fail approach is a big bang rewrite. Stop the world until the rewrite is done. That's not what we're doing. A tiny bi…

So long as your data has well-defined interface boundaries on it, this is a good way to de-risk rewrites. It really depends on the current state of the codebase. If there's a tangle of dependencies, then they have to be cleared out or facaded away to a testable interface, or else that part of the port will be a shot in the dark. And if you have the interfaces in hand, the language itself becomes considerably less imp…

Given that this is mostly about our backend systems, we've got one big interface in front: our GraphQL schema.

Our system isn't perfect, but a couple of years ago we spent a good deal of effort detangling our monolith. This plan wouldn't have been an option had we not done that.

Re: Go + Services = One Goliath Project

#419

Post by dangoor » Sun, Dec 22, 2019, 5:42 PM UTC

solidasparagus wrote: post #358

dangoor wrote: post #237

Earlier quoted context omitted.

We're making a certain set of tradeoffs. For example, we're not adopting the "write code in whatever language you want" form of microservices that some folks adopt, because we don't feel like we're large enough to support that. Like I said, though, we do want to minimize the library footprint. The vast majority of deploys in this new world will be single service deploys, with the benefits that come with that. We alre…

It'll be interesting to see how it works. I would probably fight to not have shared dependencies because the relatively small benefit does not seem to merit the increased coupling between teams - if all teams needs to agree before a library can be upgraded, I can't imagine it will be very easy to keep libraries up-to-date. To a certain extent it depends on how large your engineering org is, which I don't know. I've b…

We'll see how it plays out, but the situation that I've described isn't really different from the one we have today (because we have a monolith). Hopefully, it will be better because of how the Go project is trying to get library maintainers to follow semver and avoid breaking changes. When we need to upgrade a dependency, we can do so in one diff, catching the errors with the compiler and test runs. If the upgrade seems risky, we'll watch that deploy carefully, and we already have a process for "risky" deploys. Plus, these are likely some of the easiest changes to rollback if need be, because they are unlikely to change persisted data.

Ultimately, though, if we find that this plan reduces velocity, it won't be that hard to change later.

Re: Go + Services = One Goliath Project

#420

Post by dangoor » Sun, Dec 22, 2019, 5:55 PM UTC

tony wrote: post #410

dangoor wrote: post #260

Earlier quoted context omitted.

I appreciate all of the links and the fact that, for a lot of folks, this is the state of Python 2 to 3 migration today. What you're suggesting is, imho, the best path for most (and I appreciate you mentioning the pickle incompatibility because that is a thing which would trip people up when trying to make the move). If we would have been able to have a 2&3 compatible codebase live in a week or two, we absolutely wou…

Thanks for your reply to my reply! Sure, it may take more than two weeks, and maybe golang is a good choice for your stack. Maybe a better way of saying it is python 2/3 compatibility can be gradually adopted while maintaining. Also, I can't speak for the technical specifics of your project, but I would like to speak a bit from my prior experiences: > Incompatible changes in some libraries we use What are those libra…

We did do a bunch of refactoring a couple of years ago to draw boundaries within our monolith. We almost certainly wouldn't have made this choice had we not done that work. We're also doing work in Python to smooth the way for this change. For example, we're migrating our remaining REST endpoints to GraphQL in Python before making the move to Go.

Regarding the pickles: we are essentially doing what you're suggesting. We're going to migrate them to JSON. This is one of those things that is going to be a pain to do and we'd have to do it whether we're switching to Python 3 or Go.

Also, I'll note that the Python 2 EOL is only half real. There's so much Python 2 out there that, while the PSF is no longer supporting it, there will be people supporting it as needed.

I do agree with you about Python 3 being much cleaner. But we have to do significant rewriting of our devserver, a lot of our data access code, and some other libraries that I don't remember offhand. Again, it's quite specific to our codebase. People with a "normal" Django app are unlikely to have such issues.