Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why We Chose Rails to Build Gitlab

about.gitlab.com

Post reply on HN
Print view View original

101–110 of 165 posts

Re: Why We Chose Rails to Build Gitlab

#101

Post by noir_lord »

dcosson wrote: post #67
yingw787 wrote: post #21

What I really liked about Ruby on Rails when I took an online course on it were the schema.rb files and `rake db:migrate` command. I don't know if there's an analog in Python, but honestly I think clean, highly structured, maintainable data is way more important than the backend you use. Having the ability to quickly iterate your SQL schemas and roll them back as necessary is essential to shipping high quality featur…

Just wait til you actually work on some production code... For the most part you don't actually want to roll back schemas as you would lose data. And iterating on SQL schemas in a live app involves all kinds of tricky edge cases and backfills, knowing when to make your changes in multiple steps, being careful not to take down the whole thing because you didn't realize a certain change required a full table lock or a…

Or you inherit a DB that’s a complete disaster scheme wise and all the nice assumptions go out the window.

I’ve mostly found that migrations work best as a structure way to just run normal SQL and know a) what ran b) when.

For that they are handy.

I’ve even considered how to structure a stand alone migration running tool with a nice API since the good ones for each language tend to be fairly different.

It annoys me that the tooling around RDBMS in 2019 is still so shit.

Prime example, try to find a halfway reasonable way to debug a badly written MySQL sproc you inherited.

Re: Why We Chose Rails to Build Gitlab

#102

Post by pdimitar »

sytse wrote: post #92
pdimitar wrote: post #76

Earlier quoted context omitted.

To me, the article reads like "We REALLY don't want to abandon Rails so we rewrite the hot paths to Go". I mean no offense; I just wasn't able to find good argumentation in it for sticking with Rails. To me, the article kind of makes the argument for Go. "Developer productivity" is not a myth and it does exist. But Rails is very far from the only framework that gives you that. IMO you should seriously evaluate making…

I think a rewrite in Go would greatly increase our line-count. I think Elixir is very interesting but Phoenix still has some way to go before the ecosystem matches that of Rails. Our Gemfile.lock https://gitlab.com/gitlab-org/gitlab-ce/blob/master/Gemfile.... is more than 1000 lines!

Well, that complexity is yours to evaluate -- what I am saying is that you are seriously underestimating the Elixir community. And our forum is full of people making this same admission and saying they regret not reaching for Elixir earlier! :)

In the end, it's your call of course. But line-count doesn't say anything about a project's complexity. Go is quite verbose but with rigorous CI, tests, code reviews and linters, teams produce pretty hardcore and good software in Go, regularly.

Re: Why We Chose Rails to Build Gitlab

#103

Post by noir_lord »

hashmal wrote: post #64
yingw787 wrote: post #21

What I really liked about Ruby on Rails when I took an online course on it were the schema.rb files and `rake db:migrate` command. I don't know if there's an analog in Python, but honestly I think clean, highly structured, maintainable data is way more important than the backend you use. Having the ability to quickly iterate your SQL schemas and roll them back as necessary is essential to shipping high quality featur…

I agree, and most other commenters still miss the point about what makes Rails so great : it's not the technical part (you can point slowness, quirks in how the language/framework is used) but the human and business part. It works. It makes adapting to change easy. Yes it's a monolith, yes it encourages fat models, etc. But it works so damn well. `rake db:migrate` is not so great as a feature (other frameworks in oth…

> other frameworks in other languages have it.

Yep, Laravel (PHP) lifted it en masse (so much so that when I had to get up to speed with laravel years ago I used the rails docs to fill in the blanks) and Symfony with Doctrine is in some ways better.

Re: Why We Chose Rails to Build Gitlab

#104

Post by lkozloff »

Roritharr wrote: post #93

My biggest issue with GitLab is support. After using it for our test pipelines for a year we've switched to GitLab CI completely with our deployments and right after we've started running into issues on our selfhosted instance where for about three weeks now we're fighting with stuck pipelines where runners won't pick up jobs until you manually restart them. We've been paying customers for GitLab Enterprise (now Star…

Hi @Roritharr - Support Manager at GitLab here.

I'd like to follow up on this to see where we dropped the ball and how (or if) we can make things right. Feel free to email me at lyle[at]gitlab.com and I'll look into your ticket history and move things along.

For others, performance issues with EFS and GitLab have been frequent enough that we specifically call EFS out in our documentation: https://docs.gitlab.com/ee/administration/high_availability/...

This isn't to say that EFS doesn't have its place, it's just that git operations quickly exhaust burst credits and many of our users on EFS end up with difficult to diagnose performance problems.

This may or may not have anything to do with @Roritharr's problems, but it's worth noting!

Re: Why We Chose Rails to Build Gitlab

#105

Post by stanhu »

yannovitch wrote: post #58

"Ruby was optimized for the developer, not for running it in production," says Sid. "For the things that get hit a lot and have to be very performant or that, for example, have to wait very long on a system IO, we rewrite those in Go … We are still trying to make GitLab use less memory" Honestly, I think that Go is beginning to be where Rails was few years ago, and lots of developer friends want to hop on the Go trai…

I agree that memory requirements are too high, but I think there is a lot of room for improvement without a complete rewrite, which would be very costly. Also keep in mind that a standalone GitLab instance ships with all batteries included: PostgreSQL, Redis, etc., all of which add to the memory requirements. That being said, we are certainly looking at moving performance-critical functions into Go. See the GitLab Workhorse and Gitaly projects as examples.

There are two major ways to look at application memory usage: baseline and runtime usage. Most people pay attention to the first because that's what you see when you first deploy GitLab. However, runtime usage is just as important because a running instance could gobble up gigabytes of RAM.

Baseline memory: We've made some recent progress in reducing baseline memory usage (e.g. cutting 80-100 MB per Unicorn process in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21008). However, our Unicorn workers dominate most of the memory usage, and you can see more analysis of where our memory is going in https://gitlab.com/gitlab-org/gitlab-ce/issues/49702. https://brandur.org/ruby-memory is probably the best explanation I've seen why RAM usage in Unicorn processes often rises and doesn't come down. However, we can reduce baseline usage in a number of ways:

* Removing dependencies from the main application

* Reducing any unnecessary allocations at startup

* Moving to a multi-threaded application server (we've shipped Puma in experimental mode in https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/...)

* Improving Rails memory usage (e.g. ActiveRecord optimizations in https://github.com/rails/rails/pull/34711)

* Improving the Ruby interpreter (e.g. helping ship a compacting garbage collector, see https://gitlab.com/gitlab-org/gitlab-ce/issues/54555).

Runtime memory: Most of our runtime memory problems are often due to memory bloat or unoptimized code paths. For example, we put big dent in some of the most egregious background jobs over the last few months. For example, a change in our merge request processing in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22725... lowered 95% of our runtime usage by gigabytes. Switching to a faster XML processor in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23136 also dropped memory usage in another background job.

Obviously we're still far from where we should be, but we're making progress. If you or know anyone else who might be excited to help us, please let us know! https://about.gitlab.com/jobs/ Thanks for your feedback.

Re: Why We Chose Rails to Build Gitlab

#106

Post by adamson »

noir_lord wrote: post #101
dcosson wrote: post #67

Earlier quoted context omitted.

Just wait til you actually work on some production code... For the most part you don't actually want to roll back schemas as you would lose data. And iterating on SQL schemas in a live app involves all kinds of tricky edge cases and backfills, knowing when to make your changes in multiple steps, being careful not to take down the whole thing because you didn't realize a certain change required a full table lock or a…

Or you inherit a DB that’s a complete disaster scheme wise and all the nice assumptions go out the window. I’ve mostly found that migrations work best as a structure way to just run normal SQL and know a) what ran b) when. For that they are handy. I’ve even considered how to structure a stand alone migration running tool with a nice API since the good ones for each language tend to be fairly different. It annoys me t…

I’ve used Flywheel (https://flywaydb.org/) for this exact problem on personal projects before and been fairly happy with it. It’s intended for use in JVM-based projects, but ships with a CLI that gets you a long way.

Re: Why We Chose Rails to Build Gitlab

#107

Post by sytse »

pdimitar wrote: post #102
sytse wrote: post #92

Earlier quoted context omitted.

I think a rewrite in Go would greatly increase our line-count. I think Elixir is very interesting but Phoenix still has some way to go before the ecosystem matches that of Rails. Our Gemfile.lock https://gitlab.com/gitlab-org/gitlab-ce/blob/master/Gemfile.... is more than 1000 lines!

Well, that complexity is yours to evaluate -- what I am saying is that you are seriously underestimating the Elixir community. And our forum is full of people making this same admission and saying they regret not reaching for Elixir earlier! :) In the end, it's your call of course. But line-count doesn't say anything about a project's complexity. Go is quite verbose but with rigorous CI, tests, code reviews and linte…

I agree Go produces great software. But for the same amount of functionality it tends to require more lines than ruby. I tend to favor the more expressive language http://www.paulgraham.com/avg.html

Another thing is that there isn't a web framework like Rails in Go. Although I'm keeping an eye on go-micro and we plan to add that as a template soon.

Re: Why We Chose Rails to Build Gitlab

#108

Post by axelfontaine »

stock_toaster wrote: post #56
mey wrote: post #26

Earlier quoted context omitted.

I suggest checking out https://flywaydb.org/ It doesn't have the ORM-ness mapping rails migrate does, but it makes it really simple to organize sql scripts to move/update schema configs.

I personally tend to prefer migrate[1] as it doesn't require java. [1]: https://github.com/golang-migrate/migrate

Flyway doesn't require Java. The CLI ships with a private JRE as well as drivers for most databases out of the box. Simply untar and run.

Re: Why We Chose Rails to Build Gitlab

#109

Post by pdimitar »

sytse wrote: post #107
pdimitar wrote: post #102

Earlier quoted context omitted.

Well, that complexity is yours to evaluate -- what I am saying is that you are seriously underestimating the Elixir community. And our forum is full of people making this same admission and saying they regret not reaching for Elixir earlier! :) In the end, it's your call of course. But line-count doesn't say anything about a project's complexity. Go is quite verbose but with rigorous CI, tests, code reviews and linte…

I agree Go produces great software. But for the same amount of functionality it tends to require more lines than ruby. I tend to favor the more expressive language http://www.paulgraham.com/avg.html Another thing is that there isn't a web framework like Rails in Go. Although I'm keeping an eye on go-micro and we plan to add that as a template soon.

Yeah, fully agreed on that. Go just doesn't have as productive a framework like Rails. That's why I only use it for very small websites or even smaller API gateways. Or CLIs. Or network daemons.

I am not disputing your choices. Just wanted to give you the quick and intuitive (and likely misinformed) immediate reaction that I had from the article.

Keep making GitLab awesome. <3

Post reply on HN