Dear-GitHub: Host Github by itself as an open source project • Hacker News Board

Re: Dear-GitHub: Host Github by itself as an open source project

#61

Post by joelhandwell » Tue, Jul 03, 2018, 6:11 AM UTC

tbrock wrote: post #3

Why? It’s free for open source, reasonable for enterprise and private use and I think you could build a better one starting from scratch. GitHub’s success is largely due to the network effect and it’s entrenched status as the canonical code repository. Besides libgit2, aka “the secret sauce”, is already open source. What are you waiting for?

>Why? >What are you waiting for?

If Github was made with micro services architecture, it could be split into open source "Client side + Test-backend" and closed sourced "Production-backend". Backend can be composed of some interface and multiple implementation such as test impl and prod impl. “the secret sauce” could be the "Production-backend" and MS have good in-house talents who operates Azure so no need for the help from OSS community to improve backend.

However for example +1 button took so long time to be implemented even though it seems like small change in client side code and some adjustments in database. That itch lead to the https://github.com/dear-github/dear-github open letter and signers listed here: https://docs.google.com/spreadsheets/d/1oGsg02jS-PnlIMJ3OlWI...

The last sentence of the open letter says: "Hopefully none of these are a surprise to you as we’ve told you them before. We’ve waited years now for progress on any of them. If GitHub were open source itself, we would be implementing these things ourselves as a community—we’re very good at that!"

I think that's OSS community want, including but not only I want.

Re: Dear-GitHub: Host Github by itself as an open source project

#62

Post by TheHydroImpulse » Tue, Jul 03, 2018, 6:17 AM UTC

joelhandwell wrote: post #61

tbrock wrote: post #3

Why? It’s free for open source, reasonable for enterprise and private use and I think you could build a better one starting from scratch. GitHub’s success is largely due to the network effect and it’s entrenched status as the canonical code repository. Besides libgit2, aka “the secret sauce”, is already open source. What are you waiting for?

>Why? >What are you waiting for? If Github was made with micro services architecture, it could be split into open source "Client side + Test-backend" and closed sourced "Production-backend". Backend can be composed of some interface and multiple implementation such as test impl and prod impl. “the secret sauce” could be the "Production-backend" and MS have good in-house talents who operates Azure so no need for the h…

This just isn't possible. No company in their right mind would take their monolith and rewrite a bunch of stuff just to please a couple randoms on the internet.

I think you're drastically underestimating the amount of code Github is powered by and how freaking long any type of refactor/rewrite would take. We're talking about years.

Re: Dear-GitHub: Host Github by itself as an open source project

#63

Post by joelhandwell » Tue, Jul 03, 2018, 6:19 AM UTC

TheHydroImpulse wrote: post #62

joelhandwell wrote: post #61

Earlier quoted context omitted.

>Why? >What are you waiting for? If Github was made with micro services architecture, it could be split into open source "Client side + Test-backend" and closed sourced "Production-backend". Backend can be composed of some interface and multiple implementation such as test impl and prod impl. “the secret sauce” could be the "Production-backend" and MS have good in-house talents who operates Azure so no need for the h…

This just isn't possible. No company in their right mind would take their monolith and rewrite a bunch of stuff just to please a couple randoms on the internet. I think you're drastically underestimating the amount of code Github is powered by and how freaking long any type of refactor/rewrite would take. We're talking about years.

Oh, was Github monolith? I did not read any articles about their architecture so I assumed that Github could be composed by micro services, and in that case, splitting could go easier. I edited to "If Github was made with micro services architecture,..."

Does anyone have link to any interview or article talking about granularity of their architecture?

Re: Dear-GitHub: Host Github by itself as an open source project

#64

Post by peterwwillis » Tue, Jul 03, 2018, 6:31 AM UTC

tills13 wrote: post #40

I _hate_ people using "M$" un-ironically. It wasn't funny 10 years ago when it started and it's still not funny today. How can you take that one comment seriously with that "M$" part in it.

That's cool. I hated Micro$oft for a while, for being a bunch of evil money grubbing assholes who made it harder to run the operating system I wanted, applied proprietary extensions to open standards in order to force vendor lock-in [nearly destroying the open web], and who leveraged PC manufacturer deals to tax us for software we didn't want. They literally called Open Source a cancer and lied about what the GPL specified to discourage use of the software. M$ was a monopoly, and all about the cash. It was never supposed to be funny.

Also, irony means suggesting the opposite. I don't know why anyone would use M$ ironically.

Re: Dear-GitHub: Host Github by itself as an open source project

#65

Post by joelhandwell » Tue, Jul 03, 2018, 6:34 AM UTC

TheHydroImpulse wrote: post #62

joelhandwell wrote: post #61

Earlier quoted context omitted.

>Why? >What are you waiting for? If Github was made with micro services architecture, it could be split into open source "Client side + Test-backend" and closed sourced "Production-backend". Backend can be composed of some interface and multiple implementation such as test impl and prod impl. “the secret sauce” could be the "Production-backend" and MS have good in-house talents who operates Azure so no need for the h…

This just isn't possible. No company in their right mind would take their monolith and rewrite a bunch of stuff just to please a couple randoms on the internet. I think you're drastically underestimating the amount of code Github is powered by and how freaking long any type of refactor/rewrite would take. We're talking about years.

I did a search and found http://highscalability.com/blog/2009/11/6/product-resque-git...

It's talking about a component of Github backend called Resque (Distrubuted Job Queue)

As long as I see these thing the architecture is highly distributed and it's possibly composed of micro services.

Re: Dear-GitHub: Host Github by itself as an open source project

#66

Post by joelhandwell » Tue, Jul 03, 2018, 6:46 AM UTC

TheHydroImpulse wrote: post #62

joelhandwell wrote: post #61

Earlier quoted context omitted.

>Why? >What are you waiting for? If Github was made with micro services architecture, it could be split into open source "Client side + Test-backend" and closed sourced "Production-backend". Backend can be composed of some interface and multiple implementation such as test impl and prod impl. “the secret sauce” could be the "Production-backend" and MS have good in-house talents who operates Azure so no need for the h…

This just isn't possible. No company in their right mind would take their monolith and rewrite a bunch of stuff just to please a couple randoms on the internet. I think you're drastically underestimating the amount of code Github is powered by and how freaking long any type of refactor/rewrite would take. We're talking about years.

Reading Github Engineering blog posts related to their architecture. https://goo.gl/amkJfV As far as I read so far, Github looks like composed of micro services, or at least split into many distributed services.

Re: Dear-GitHub: Host Github by itself as an open source project

#67

Post by kanwisher » Tue, Jul 03, 2018, 6:46 AM UTC

freedomben wrote: post #39

nemothekid wrote: post #10

I really don't see the value in open sourcing GitHub when Gitlab exists if you are looking for a replacement incase MS decides to go full tyrant - unless if you believe GitHub enterprise is an amazing project. If GitHub wasn't built from the start for hosted deployments, it will probably be both expensive and a pain in the neck to run. GitLab probably has it's issues, but I won't be surprised if there are assumptions…

This would be interesting to hear about from an insider. I suspect you're right. Having worked at a company that open sources most of their code, I've seen first hand how much better the code quality is when you know it's going out to the world. When you know it's gonna be hidden and only a handful of people will ever see it, you're much less hesitant to push up hacks :-) We open sourced something that wasn't origina…

What’s the business value in spending weeks of developer time refactoring things that are going open source. For some industries it’s required like blockchain. For average shop seems like more expense with no profit

Re: Dear-GitHub: Host Github by itself as an open source project

#68

Post by TheHydroImpulse » Tue, Jul 03, 2018, 6:50 AM UTC

joelhandwell wrote: post #65

TheHydroImpulse wrote: post #62

Earlier quoted context omitted.

This just isn't possible. No company in their right mind would take their monolith and rewrite a bunch of stuff just to please a couple randoms on the internet. I think you're drastically underestimating the amount of code Github is powered by and how freaking long any type of refactor/rewrite would take. We're talking about years.

I did a search and found http://highscalability.com/blog/2009/11/6/product-resque-git... It's talking about a component of Github backend called Resque (Distrubuted Job Queue) As long as I see these thing the architecture is highly distributed and it's possibly composed of micro services.

Github has been a well known Rails monolith. Absolutely they have services that power all sorts of stuff (they have a few blog posts on it) but it's just impractical to even start a discussion on splitting it up for the reasons you initially argued for.

I've been through a number of large rewrites/reworks that took monoliths much like Github (with many many many services behind it) and split them up into modular pieces and it's an insane amount of work that can take years. You simply need very good reasons (including business reasons) to do that.

Moreover, companies at these sizes just have a LOT of code all over the place. Tooling, infra, supporting services, etc... Not to mention it's just not useful to have external contributors for a business product like Github. Doing code reviews, addressing bugs that were introduced, spending time discussing things with contributors takes an incredible amount of time.

Basically if the reason you want Github open sourced (and reworked into some weird architecture you described) is so that people can contribute to fix things and add features....Github could/will just hire more devs to work on that.

Re: Dear-GitHub: Host Github by itself as an open source project

#69

Post by joelhandwell » Tue, Jul 03, 2018, 6:57 AM UTC

joelhandwell wrote: post #65

TheHydroImpulse wrote: post #62

Earlier quoted context omitted.

This just isn't possible. No company in their right mind would take their monolith and rewrite a bunch of stuff just to please a couple randoms on the internet. I think you're drastically underestimating the amount of code Github is powered by and how freaking long any type of refactor/rewrite would take. We're talking about years.

I did a search and found http://highscalability.com/blog/2009/11/6/product-resque-git... It's talking about a component of Github backend called Resque (Distrubuted Job Queue) As long as I see these thing the architecture is highly distributed and it's possibly composed of micro services.

Thanks TheHydroImpulse for the insights. So Github IS monolith. It really makes sense that splitting is too much work for just open sourcing and I do not see business gain to invest money and people into it.

Then possible path might be isolating least coupled (and small) components of client side code and open source?

Re: Dear-GitHub: Host Github by itself as an open source project

#70

Post by joelhandwell » Tue, Jul 03, 2018, 7:48 AM UTC

TheHydroImpulse wrote: post #68

joelhandwell wrote: post #65

Earlier quoted context omitted.

I did a search and found http://highscalability.com/blog/2009/11/6/product-resque-git... It's talking about a component of Github backend called Resque (Distrubuted Job Queue) As long as I see these thing the architecture is highly distributed and it's possibly composed of micro services.

Github has been a well known Rails monolith. Absolutely they have services that power all sorts of stuff (they have a few blog posts on it) but it's just impractical to even start a discussion on splitting it up for the reasons you initially argued for. I've been through a number of large rewrites/reworks that took monoliths much like Github (with many many many services behind it) and split them up into modular piec…

Thinking about the fact that Github have Github.com (production service) and Github Enterprise (self hosted), could it be like this?

Github.com = Github Core + Production Services and Infra

Github Enterprise = Github Core + Services need for Self Hosting

Maybe it's not worthy to proceed based on assumption but if there is something like "Github Core" which is shared codebase between prod and self-hosted, open sourcing the core can be an option?