Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go + Services = One Goliath Project

engineering.khanacademy.org

Post reply on HN
Print view View original

231–240 of 449 posts

Re: Go + Services = One Goliath Project

#231

Post by stoicShell »

guitarbill wrote: post #212
stoicShell wrote: post #180

Earlier quoted context omitted.

The saying only applies when your product is software itself, not in most cases where software is just the means to deliver your product — here a website providing education in video form etc. Besides, the reality is that most business software out there gets rewritten every 3-15 years (really depends on use-case and conditions, but on average 4-5-6 years is a good bet). After some time it's just not worth it to keep…

the number of failed migrations, modernizations, and "tech transformations" in non-software industries, as well as the number of consulting outfits and their profits doesn't seem to back this up. [0] IT disasters now part of modern life - https://www.afr.com/technology/it-disasters-now-part-of-mode... [1] Number of IT failures at banks and other firms is unacceptable, say MPs - https://www.theguardian.com/business/20…

That will make for some pretty salient readings, thanks for the links.

There are obviously cases of failure. I don't know the stats. I'm speaking mostly of small/medium businesses, where size and complexity are different.

Re: Go + Services = One Goliath Project

#232

Post by antielectronite »

WoodTree wrote: post #199
antielectronite wrote: post #154

Earlier quoted context omitted.

I disagree with the notion that most applications are IO bound only. This is something people often say uncritically, but in my experience is false. Just using a non-native Electron or even Java application feels very sluggish and when you look at the Waterfall on slow web pages, what's slowing it down is very often unrelated to "I/O". Secondly, C/C++ is like the third or fourth most commonly listed programming langu…

C/C++ is not a language. I am also someone who has use C and C++ for years as part of my work and have mostly moved on to TypeScript because there isn’t much reason to use C or C++ anymore unless you are in one of those niches where you need to still program at that level. Most software problems are not about solving them faster, it’s about combining existing components in new ways and figuring out to orchestrate it…

I don't necessarily disagree that not everything needs to be optimized for performance, but I would just argue that the use cases for Typescript are far more niche than the use cases for C++. There's more to software than just web stuff

Re: Go + Services = One Goliath Project

#233

Post by gtirloni »

solidasparagus wrote: post #229
dangoor wrote: post #211

Earlier quoted context omitted.

Our approach to services at Khan Academy is likely a bit different from most. We're sticking with a monorepo (the code for all services lives in one repository). We have a single go.mod file at the top of the repo, so all services use the _same versions_ of dependencies. We're still building out our deployment system to better support multiple services, but we're planning to redeploy all of the services when library…

I don't really understand moving to microservices if you aren't going to give the service teams the autonomy to make their own decisions and move at their own pace. Microservices always seemed to me to be more of an organizational strategy than a technical one - if you have services, but they can't operate independently, it feels a bit like you are just creating a distributed monolith.

Choosing the dependencies' version is not all there's to decide. Teams have a lot of control of basically everything else.

Re: Go + Services = One Goliath Project

#234

Post by dangoor »

bb88 wrote: post #223
dangoor wrote: post #182

Earlier quoted context omitted.

The Python 2 to 3 conversion was only part of it. The libraries were a huge difference for us. I will agree that porting to Python 3 would be less work than porting to Go, but we believe that the difference is less than people would suspect.

Never announce a plan to do something. Announce the results after doing it. If you announce a plan, you're making a bet (sometimes with your reputation) that's it's going to be successful, and telling the world about it. If however, you didn't announce it, you could still get the benefit of success, or make it looks like learning the lessons of a failed experiment.

We have the good fortune of not having to be super secretive about our tech. We _want_ to talk about this project as we go along and share what we learn. We've already got more interesting stuff to talk about, and it'd be a lot less interesting, I think, without this context for the overall project.

As mentioned in the post, a small piece of our GraphQL schema is already in Go running in production. This blog post isn't just "we're thinking of doing this thing". It's "we've already done a bunch of research, thinking, _and_ built some of it."

Re: Go + Services = One Goliath Project

#235

Post by PudgePacket »

pushrax wrote: post #224
tybit wrote: post #128

As much as I personally don’t enjoy writing Go I really can’t fault them. I still find it interesting that for a relatively obvious feature set of fast compiles, fast startup and fast runtime there really isn’t anything mainstream out there to compete with Go. I really hope something like Kotlin, Swift, ReasonML or even AOT JVM/.NET brings something to the table soon. Or perhaps I’ll just have to wait for WASM to rea…

How does WASM fill this gap? > fast compiles, fast startup and fast runtime ... mainstream C is this but it's harder to write secure/correct C, the standard library is smaller, and there's no canonical toolchain in the same way as Go.

One doesn't just write WASM though.. WASM is like the JVM. You still need to write code in some other language.

Re: Go + Services = One Goliath Project

#236

Post by axaxs »

purple-again wrote: post #9

We turned our monolith into a bunch of micro services almost 6 years ago to the day. For a long time I was very happy with the new pattern but over the years the weight of keeping everything updated along with the inevitable corners that fall behind and have...questionable..security due to how long they sit neglected has really left me wondering if I am happy with it after all. I would love hear some thoughts from ot…

Same. I tried microservices before and while it cleaned up the code base, I didn't quite like the results down the road. Some things go stale, you multiply ops * number of microservices, a change in one can mean a change in multiple others. I'm not against services in general, but not a fan of so called microservices.

Re: Go + Services = One Goliath Project

#237

Post by dangoor »

solidasparagus wrote: post #229
dangoor wrote: post #211

Earlier quoted context omitted.

Our approach to services at Khan Academy is likely a bit different from most. We're sticking with a monorepo (the code for all services lives in one repository). We have a single go.mod file at the top of the repo, so all services use the _same versions_ of dependencies. We're still building out our deployment system to better support multiple services, but we're planning to redeploy all of the services when library…

I don't really understand moving to microservices if you aren't going to give the service teams the autonomy to make their own decisions and move at their own pace. Microservices always seemed to me to be more of an organizational strategy than a technical one - if you have services, but they can't operate independently, it feels a bit like you are just creating a distributed monolith.

We're making a certain set of tradeoffs. For example, we're not adopting the "write code in whatever language you want" form of microservices that some folks adopt, because we don't feel like we're large enough to support that.

Like I said, though, we do want to minimize the library footprint. The vast majority of deploys in this new world will be single service deploys, with the benefits that come with that. We already deploy our monolith several times a day. These services will speed that further.

Re: Go + Services = One Goliath Project

#238

Post by WoodTree »

antielectronite wrote: post #232
WoodTree wrote: post #199

Earlier quoted context omitted.

C/C++ is not a language. I am also someone who has use C and C++ for years as part of my work and have mostly moved on to TypeScript because there isn’t much reason to use C or C++ anymore unless you are in one of those niches where you need to still program at that level. Most software problems are not about solving them faster, it’s about combining existing components in new ways and figuring out to orchestrate it…

I don't necessarily disagree that not everything needs to be optimized for performance, but I would just argue that the use cases for Typescript are far more niche than the use cases for C++. There's more to software than just web stuff

Most stuff is web stuff now. And I’m not talking about front end, we do a lot of back end work in TypeScript because node is lighter than the JVM which makes it a better choice for lambdas. I’d say it also has more sophisticated static typing than Java or C++, while also allowing dynamic typing in the few cases where it is convenient. Having the front and back end written in the same language also reduces impedance between teams. A lot of our tooling is even written in it now, deprecating many Ruby scripts.

Re: Go + Services = One Goliath Project

#239

Post by Thaxll »

coldtea wrote: post #227
Thaxll wrote: post #178

Earlier quoted context omitted.

See for yourself: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

See what? This is an unofficial benchmark game - and based on implementations people bothered to provide. Not some scientific test, and nothing that guarantees these are the best implementations. And even if they were they're not representative of server/long running program behavior (where JIT quality and especially GC implementation) matters. Even so, the page linked starts with "Back in April 2010, Russ Cox charit…

I used both Java and Go in production and yes Go can be faster or Java it depends but Go usually use between 3 and 10x less memory.

https://www.techempower.com/benchmarks/ ( real world benchmark with networking / serialization ect ... )

Get over it yes Go can be faster than Java it's not a secret.

Re: Go + Services = One Goliath Project

#240

Post by tony »

You can start porting your live codebase to python 3 right now - no downtime at all. You can have a 2+3 compatible codebase live in a week or two, it'll get you about 98% the way there [1]

My projects (e.g. https://tmuxp.git-pull.com, https://unihan-etl.git-pull.com) are both python 2/3 compatible.

I learned by reading through other projects like Sphinx, Flask, Werkzeug, and SQLAlchemy:

- Flask: https://github.com/pallets/flask

- Werkzeug: https://github.com/pallets/werkzeug

- Sphinx https://github.com/sphinx-doc/sphinx/tree/v1.8.5 (see compat: https://github.com/sphinx-doc/sphinx/blob/v1.8.5/sphinx/util...)

- https://github.com/sqlalchemy/sqlalchemy

Helpful blog posts: http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python..., http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/

As for automation tools, for 2/3 compatibility, I used futurize and huge codebase to great success: https://python-future.org/futurize.html. I started with this:

    futurize --write --stage1 --unicode-literals --nobackups
and https://docs.python.org/2/library/2to3.html

    2to3-2.7 -w -n
Really helped.

Also, wire Travis / your CI system to have your tests run on python 2 and 3.

If you want to test your python 3 codebase live on a subdomain / same SQL/NoSQL DB, be careful about jobs/tasks! Pickle version mismatches and stuff. Use a separate redis/whatever DB for the deployments.

After you're finally on python 3, you can use https://github.com/asottile/pyupgrade to modern your code.

[1] For the other 2%, use conditionals in your requirements file:

    enum34==1.0.4;python_version
Post reply on HN