Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's not Ruby that's slow, it's your database

berk.es

Post reply on HN
Print view View original

111–120 of 203 posts

Re: It's not Ruby that's slow, it's your database

#111

Post by Lukas_Skywalker »

kstrauser wrote: post #75
twotwotwo wrote: post #45

This is kind of relevant now : Mastodon is seeing a lot of growth all the sudden, the popular server for it is in Rails, and scaling up quickly is a challenge for server operators used to the levels of traffic from two months ago. Twitter did rewrite their frontend to JVM languages because it was enough of a bottleneck to be worth it. If you can get Mastodon substantially further by serving some views using something…

Ugh, tell me about it. I had an interesting weekend dealing with traffic loads that grew by about 7x in a few days. I wrote about my misadventures at https://blog.freeradical.zone/post/surviving-thriving-throug... . The gist of it is that a RoR “Sidekiq” task queue gets CPU bound after about 25-30 worker threads doing things like making REST API calls to remote servers, querying a database, insert status updates, etc…

I assume that Mastodon is using ActiveJob to push tasks to the queue. According to the creator of Sidekiq, bypassing ActiveJob would lead to significantly lower resource usage: https://github.com/mperham/sidekiq/wiki/Active+Job#performan...

I never tried it though. Just remembered the Wiki note.

Re: It's not Ruby that's slow, it's your database

#112

Post by gls2ro »

dimgl wrote: post #11
rahimnathwani wrote: post #4

"Keep all logic out of the database. It already is the slowest point. And hardest to scale up." I don't think this is true in all cases. Sure, ORMs are awesome, but sometimes you need to write SQL queries by hand, and those queries necessarily implement some business logic (even if they're just retrieving data). Nice explanation here: https://tapoueh.org/blog/2017/06/sql-and-business-logic/

> Keep all logic out of the database. It already is the slowest point. And hardest to scale up. That's... weird advice. I think a lot of what happens in Rails projects (based on my very limited experience) is that developers start to rely on this easy syntax that ActiveRecord provides and stop thinking about the queries that the ORM is creating. So you end up with these massive N+1 queries that kill performance. This…

It is good advice to learn SQL.

I would suggest enabling the following two configurations in any Rails project. It should be a must in any new Rails project IMO:

1. Enable `config.active_record.strict_loading_by_default` - this will raise ActiveRecord::StrictLoadingViolationError on almost all cases of N+1 thus forcing the developer to fix it

2. Set `config.active_record.warn_on_records_fetched_greater_than` so that you know when someone will load a lot of records, thus forcing to paginate, load only what is needed ...

I think starting with these two will help mitigate a lot of problems even when using ORM in Rails.

Re: It's not Ruby that's slow, it's your database

#113

Post by Widdershin »

derefr wrote: post #88
seanhunter wrote: post #82

The biggest database performance problem that I have seen time and time again over the 20+ years I've been using sql databases now is caused by people not understanding that databases are relational (ie designed to perform efficiently over tables, not individual rows) and so running code that essentially causes hundreds/thousands of individual queries where they could just do one big query. Any time you are doing a s…

On the other hand, weird things can happen at scale inside an RDBMS. For example: window queries, where you're trying to extract `row_number() = 1` of each window-partition. (Or "DISTINCT ON" queries, which are basically syntax sugar for this.) These can be extremely slow, because in many RDBMSes† they generate a query plan that reads+materializes all the rows of each window-partition, in order to then just discard a…

This is good to know about, used a similar window function in Postgres recently. Will double check the performance of that query.

Re: It's not Ruby that's slow, it's your database

#114

Post by kaba0 »

twotwotwo wrote: post #45

This is kind of relevant now : Mastodon is seeing a lot of growth all the sudden, the popular server for it is in Rails, and scaling up quickly is a challenge for server operators used to the levels of traffic from two months ago. Twitter did rewrite their frontend to JVM languages because it was enough of a bottleneck to be worth it. If you can get Mastodon substantially further by serving some views using something…

What’s the state of TruffleRuby? Last time I checked it was impressively fast. I know that it is really hard to port everything (and sometimes minute details can change behavior), but nowadays Truffle can even execute C code, so those FFI parts can also do as is.

Re: It's not Ruby that's slow, it's your database

#115

Post by drogus »

adenozine wrote: post #87
anonygler wrote: post #86

It’s definitely ruby that’s slow. Your database may also be slow. I love ruby. But it’s slow. It’s a great choice for a lot of things.

Got some sources? Ruby3 was faster than python at some point, iirc

I started working with Ruby almost two decades ago and I can confirm that, in fact, Ruby is slow. It might be fast enough to make it viable for most web apps, but compared to Go or Rust it will be much harder to write a fast app in Ruby. I've seen tweets and articles like this one (it's the databse not Ruby!) numerous times and it usually doesn't map to reality, especially in bigger applications. Of course you can make Rails to respond in milliseconds, but in real production apps I've usually seen 50-80% of the responses time to be spent in CPU (ie. Ruby). At the same time when the app grows and you add more middleware an empty response can easily take 5-10ms with a bit of a traffic (like: an empty controller action, no database queries etc). It's better with smaller frameworks like Sinatra, but still you can make the database much faster than the Ruby code making queries

update:

Maybe out of curiosity I'll make some benchmarks later, but even if you look at frameworks benchmarks you can see that for the same database queries there's an order of magnitude of difference between Ruby and Rust or Go frameworks: https://www.techempower.com/benchmarks/#section=data-r21&tes... (look at latency)

Re: It's not Ruby that's slow, it's your database

#116

Post by kaba0 »

JohnBooty wrote: post #74
clintonb wrote: post #54

Earlier quoted context omitted.

> It's fast enough for Stripe's API... Stripe is moving to Java.

If you need maximum requests per second for sufficiently high volumes of traffic - ie, Stripe/Twitter/FB/whatever stuff - then yeah Ruby/Python/etc doesn't cut it. If your needs get extreme enough (high frequency trading?) then Java doesn't cut it either. Most (I suspect > 99%) use cases don't fit these criteria. I know this is skewed a perception is skewed a little by the HN crowd, where many folks legitimately are…

Java is used in HFT, and for the variant where it is not fast enough, general purpose CPUs are not fast enough so not even hand-written assembly can compete anymore. They use FPGAs for that kind.

Re: It's not Ruby that's slow, it's your database

#117

Post by drogus »

joshmn wrote: post #5
likeabbas wrote: post #2

No, Ruby is slow. Its GC is ungodly slow. And you can’t truly multithread, thus you can’t properly parallelize or maximize concurrency.

How many developers will really need it though? It's fast enough for Stripe's API, it's fast enough for Shopify; hardware in exchange for productivity is a pretty fair trade.

Both Stripe and Shopify have more resources than an average small startup

Re: It's not Ruby that's slow, it's your database

#118

Post by kaba0 »

Mikeb85 wrote: post #57
likeabbas wrote: post #2

No, Ruby is slow. Its GC is ungodly slow. And you can’t truly multithread, thus you can’t properly parallelize or maximize concurrency.

> No, Ruby is slow. Its GC is ungodly slow. Ruby is quicker than Python and that's the most popular programming language there is at the moment. > And you can’t truly multithread, thus you can’t properly parallelize or maximize concurrency You definitely can. You've been able to fork processes for longer than I can remember. Ractors just became a thing, making multithreading cheaper and easier. Ruby's also had multit…

> Python and that's the most popular programming language there is at the moment.

Ordering the top 3 languages is “subjective”, let’s say that they are Javascript, python and java. And out of these, python is 10x slower than the other two.

Re: It's not Ruby that's slow, it's your database

#119

Post by rob74 »

codedokode wrote: post #101
boloust wrote: post #60

I was left confused after reading this article. 1. It claims Rust is ~10x faster than Ruby, based on a benchmark that reads a 23mb file, and then iterates over the data a single time. In my experience, Rust is between 20-100x faster than Ruby in purely CPU-bound workloads. But the author's main contention is that most work is IO-bound instead of CPU-bound, so probably not a big deal. 2. The author claims "it hardly m…

If the query selects a small number of rows and columns, and uses an index, it typically takes less than 1 ms on modern hardware. At least with MySQL. ORM however might be slow. For example, taking 100 rows from SQLite with SQLAlchemy takes 1ms, but getting them as ORM objects takes 8 ms.

And the ORM is written in... Ruby? So then we are back to "Ruby is slow".

Re: It's not Ruby that's slow, it's your database

#120

Post by thom »

pdntspa wrote: post #73
likeabbas wrote: post #2

No, Ruby is slow. Its GC is ungodly slow. And you can’t truly multithread, thus you can’t properly parallelize or maximize concurrency.

JRuby does not have GIL and offers true multithreading

I feel bad that I’ve forever tarred JRuby with the atrociously slow performance of ThoughtWorks’ Mingle. If you think Jura’s sluggish…
Post reply on HN