Live data from Hacker News

It's not Ruby that's slow, it's your database

berk.es

141–150 of 203 posts

Re: It's not Ruby that's slow, it's your database

#141
post #136
post #114

Earlier quoted context omitted.

What’s the state of TruffleRuby? Last time I checked it was impressively fast. I know that it is really hard to port everything (and sometimes minute details can change behavior), but nowadays Truffle can even execute C code, so those FFI parts can also do as is.

It just needs a small puma config tweak: https://twitter.com/eregontp/status/1588199934796365826

Wow, that is impressive, thank you! I’m eager to see some benchmarks.

Re: It's not Ruby that's slow, it's your database

#142
post #107

Earlier quoted context omitted.

I'm gonna play devil's advocate here: Ruby sucks in large codebases where conventions aren't well defined (unlike Rails, which has well understood concepts and abstractions). Having no ability to do any kind of static analysis means a large legacy codebase will contain giant rabbit holes for you to fall into any and everywhere you look -- unless, of course, the architect(s) responsible for the codebase thought very c…

You are exactly right and the pitfalls you talk about are common to all dynamically typed languages.

Which is why some dynamically typed languages, like PHP, are moving towards types.

Re: It's not Ruby that's slow, it's your database

#143
post #115

Earlier quoted context omitted.

Got some sources? Ruby3 was faster than python at some point, iirc

I started working with Ruby almost two decades ago and I can confirm that, in fact, Ruby is slow. It might be fast enough to make it viable for most web apps, but compared to Go or Rust it will be much harder to write a fast app in Ruby. I've seen tweets and articles like this one (it's the databse not Ruby!) numerous times and it usually doesn't map to reality, especially in bigger applications. Of course you can ma…

While I agree with your general point (yes a Ruby web app once basic database access patterns have been optimized is essentially CPU/GVL bound), I'd advise caution when using things like techempower benchmarks of the benchmark game.

Different languages receive very different levels of care in these, and from memory there was some big no-nos in the Rails benchmark.

One I remember for instance is that they use redis as a cache but without a connection pool [0], and AFAICT they run puma with at least 5 threads [1], so they are very likely to content on that one Redis connection.

That's just one of the many things I spotted when I looked at it a few months back.

[0] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... [1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/5429...

Edit:

To add to my point, you linked to the "20 queries" benchmark, which is essentially:

    render json: 20.times.map { |i| World.find(i) }
Somehow this average 260ms latency while the single query version average 8ms, so something definitely don't add up, since worst case scenario it would be 20x slower so ~160ms.

Another evidence is that the "cached" version is also 260ms.

And this is a dummy app barely doing anything, I've seen apps doing a ton more work in much less time.

Re: It's not Ruby that's slow, it's your database

#144
post #76

Earlier quoted context omitted.

I’ve found that all this does is make the query less readable. SQL is purpose made for writing queries, and avoids unnecessary syntax noise you get when trying to fit the query into a host language based dsl.

That really depends on the language - specifically, on whether it already has constructs that can map nicely (e.g. LINQ in C#), or macros to define them, or syntax that is generally amenable to DSLs even without macros in the picture (e.g. Lisps). SQL itself is also not a particularly well-designed query language. E.g. the order of the query doesn't reflect the natural data flow (SELECT .. FROM .. is reversed - compa…

SQL is powerful. A DSL that "fixes" things in this area getting all the other language feature interactions right isn't trivial, all the while users have to learn yet another language. Take PRQL for example: https://prql-lang.org. It looks nice, but the examples are very basic. What about window functions, grouping sets, lateral, DML, recursive SQL, pattern matching, pivot/unpivot etc. Might be doable, but perhaps, they've already made a decision that won't enable one of those features without adding new kludges.

Besides, every single "fix" will be a proprietary solution, while SQL is an ISO/IEC standard that's here to stay and universally adopted.

> A good DSL can do much better.

Stonebraker's QUEL was "better", before SQL, and yet, where is QUEL today?

Re: It's not Ruby that's slow, it's your database

#145
post #60

I was left confused after reading this article. 1. It claims Rust is ~10x faster than Ruby, based on a benchmark that reads a 23mb file, and then iterates over the data a single time. In my experience, Rust is between 20-100x faster than Ruby in purely CPU-bound workloads. But the author's main contention is that most work is IO-bound instead of CPU-bound, so probably not a big deal. 2. The author claims "it hardly m…

> I'm not sure why it just needs to be taken as a given that databases will take 150ms to return any data.

Run your database on a t2.small instance on AWS (1 vCPU, 2GiB RAM). Why would you do that, you ask? I don't know, but that's what we got on an old job.

This was also used to prove MongoDB is faster than PostgreSQL, even though Mongo was running on-prem on much better hardware.

Re: It's not Ruby that's slow, it's your database

#146
post #52

Yes, index the database. Another thing that can make Rails apps quite fast is using caching (Rails has some builtins that make that easy). In some cases you end up skipping a database lookup and running little Ruby code, which is a huge speedup when you can do it.

Caching seems to be a general purpose approach that got lost the longer my career went on. I suspect as networks and computers got faster people stopped thinking the added complexity was worth it, but it's still invaluable in bigger applications.

Caching on web apps got harder the more stuff moved to the client, I’m not sure it’s just laziness.

Re: It's not Ruby that's slow, it's your database

#147

In my experience 9/10 web application performance problems are related to database interactions. Is the query using an index? Are you hammering your DB with 300 N+1 queries to render an API response. These are usually the biggest offenders when poor performance is observed. If you're interactions with you DB are bad, it doesn't matter what programming language you use, performance will be mostly equally awful regardl…

We use Python for web apps and I heard one developer complain that Python is a slow language and asked whether we should consider a faster language ("like PHP").

I just told them that if they find a function in the web application where we are actually bottlenecking on our Python code, I'll personally rewrite that in C (although I might actually just use Rust instead).

No complaints or rewrites thus far. I don't actually think the other developers, including the one lodging the initial compaint, know where our performance bottlenecks are. They just know that "Python is slow" and repeat that.

Re: It's not Ruby that's slow, it's your database

#148

I do think this might have been true in the past, but SSDs have changed the game for databases. Databases became 10x faster over night but this hasn't sunk in to every developer yet. Also ORMs have created an environment, where bad queries are sent to the database for questionable gains (after saving time writing some initial code). After writing a nice Java ORM in the 90s myself (also wrote my own Ruby ORM framework…

> I now prefer plain SQL

I don't know if Go is capable of, but Rust has a library called SQLx that does check that your SQL query at compile time (it makes the compilation slower than normal, but it's a good trade off I think), if this could done in Go it will be amazing too.

Re: It's not Ruby that's slow, it's your database

#149

I do think this might have been true in the past, but SSDs have changed the game for databases. Databases became 10x faster over night but this hasn't sunk in to every developer yet. Also ORMs have created an environment, where bad queries are sent to the database for questionable gains (after saving time writing some initial code). After writing a nice Java ORM in the 90s myself (also wrote my own Ruby ORM framework…

> I now prefer plain SQL

I prefer plain SQL using an ORM. Usually using the ORM for insertion, SQL for extraction.

The ones I have used are not mutually exclusive (doctrine, hibernate).

Re: It's not Ruby that's slow, it's your database

#150
post #126
post #125

Earlier quoted context omitted.

Django has pretty great features for handling exactly this. Unfortunately it doesn't by default warn you if you're missing this optimization. I've had to write that warning system myself into iommi, but with that on it's super nice.

Well, Django has an ORM that makes this kind of feature necessary in the first place? ORM's are trying to hide the relational nature of your db, don't they?

Not really... At least, not a good ORM.

The actual idea behind a model based ORM system is to basically solve two problems.

The first is SQL is old and its methodology doesn't directly map onto modern programming paradigms. This one is just apparent by seeing what you get back from raw queries; tables. Want to do something with the table? Go and iterate over the rows individually. Want to get something from a foreign table? Go and do a JOIN before you even start iterating. Want to use binary json properly (the one SQL can actually index so it's not slow as molasses)? Enjoy a serialization syntax that makes you want to rip your eyes out once you get into the fine details. And that's without delving into how every "flavor" of SQL is subtly different with its own idiosyncrasies compared to everyone else. SQLite is not MySQL is not SQL server is not Postgres. Just about the only thing actually shared between them is the extremely basic CRUD tasks (and only if your definition doesn't include any complex WHERE operations). Anything to do with configuring the database is implementation specific in syntax.

ORMs broadly solve this by letting you use OOP paradigms instead; you designate a model that inherits from some base class with fields that themselves are typed to something in the database (including foreign key fields, so the relational status of your database is preserved). After that you can use traditional OOP methods and queries to read and write to the database. Most ORMs will also offer you some form to speed up any query slowdown that might occur due to unnecessary joins by letting you express them beforehand in a syntax that actually makes programmatic sense instead of trying to awkwardly map things.

The other thing ORMs solve is that by default you uh... don't check in your database scheme into a VCS. Some projects just do an SQL dump and expect you to import that. ORMs are basically a solution to that, in that they are a reference for how your database is expected to look like to be functional.

The really fancy ORMs also typically introduce (or support) a migration system so you can also track and easily update your database when your models get changed (which is well, necessary due to the disconnect between the database and the application), but that's not everywhere (SQLAlchemy for example doesn't support it).

To be clear, ActiveRecord is kinda garbage. It does have a migration system but rather than doing model agnostic queries it just references the original models, which leads to problems when setting up the database in development mode since you can't just run all migrations. Instead rake gives you the option to create a database dump of a working environment and have that be the new "canonical" version of the database. For that reason alone it's a mess. It's also incredibly slow, has (as the post mentions) lots of footguns that you need to be careful with and in some cases cannot actually be worked around easily.

ActiveRecord is arguably what killed Ruby for most webapps and I can say that pretty confidently.

Post reply on HN