Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's not Ruby that's slow, it's your database

berk.es

Post reply on HN
Print view View original

181–190 of 203 posts

Re: It's not Ruby that's slow, it's your database

#181

Post by dimgl »

CoolCold wrote: post #106
dimgl wrote: post #11

Earlier quoted context omitted.

> Keep all logic out of the database. It already is the slowest point. And hardest to scale up. That's... weird advice. I think a lot of what happens in Rails projects (based on my very limited experience) is that developers start to rely on this easy syntax that ActiveRecord provides and stop thinking about the queries that the ORM is creating. So you end up with these massive N+1 queries that kill performance. This…

> We had to completely abandon Docker for Ruby on Rails because the performance was absolutely abysmal, even with VirtioFS enabled. can you share more details on this? not sure I get how any kind of VirtioFS comes into the game here

It's a Docker for Mac issue: https://github.com/docker/roadmap/issues/7

Re: It's not Ruby that's slow, it's your database

#182

Post by aaron_m04 »

arkis22 wrote: post #34
drewbug01 wrote: post #13

I think my favorite part is buried in footnote 5: > Ironically, the performance issue becomes less articulated in this non-http, non-rails context, yet in these cases people generally dismiss ruby as option, for its performance-issues. Which, catch-22, is one of the reasons Ruby is hardly used outside of Rails (and/or Web). It's a shame that most people only write Ruby in the context of Rails. It's a lovely language,…

rails, postgres, PORO, and htmx is a monstrously productive stack

What is PORO?

Re: It's not Ruby that's slow, it's your database

#183

Post by hamandcheese »

JohnBooty wrote: post #99
hamandcheese wrote: post #95

Earlier quoted context omitted.

I explicitly do not want to manage stored procedures in the same way as typical migrations - if I did, I would wind up with many, many versions of the procedure in my code base as it evolved over time. This would make grepping or locating the latest version pretty annoying. Flyway (migration tool in Java) has a notion of “repeatable” migrations, though, which would do the trick.

This would make grepping or locating the latest version pretty annoying Wouldn't this be an issue with any database object managed via migrations? Do any of them make this easy for any database object? In ActiveRecord, you have your migrations folder(s) and then you have your `structure.sql` (essentially the raw output of mysqldump or pgdump) or the equivalent. If I need to see the literal database definition of any…

> Wouldn't this be an issue with any database object managed via migrations?

Like I mentioned, check out flyway repeatable migrations.

Re: It's not Ruby that's slow, it's your database

#184

Post by CoolCold »

dimgl wrote: post #181
CoolCold wrote: post #106

Earlier quoted context omitted.

> We had to completely abandon Docker for Ruby on Rails because the performance was absolutely abysmal, even with VirtioFS enabled. can you share more details on this? not sure I get how any kind of VirtioFS comes into the game here

It's a Docker for Mac issue: https://github.com/docker/roadmap/issues/7

Thanks, I was thinking you were talking about [production systems] Linux.

Re: It's not Ruby that's slow, it's your database

#185

Post by rubyist5eva »

JohnBooty wrote: post #79
rubyist5eva wrote: post #50

I’m a Rails developer by trade, been doing it for over a decade. Bad indexing and lazy N+1 are 90% of the performance problems in a typical rails application. The other 10% is when people do aggregates or joins in application code instead of sql. I absolutely despise ActiveRecord because it makes non-trivial aggregates a pain to write. Sequel is a much better ORM but good luck getting a team on board.

I absolutely despise ActiveRecord because it makes non-trivial aggregates a pain to write I'm a SQL guy at heart. I certainly agree with you... I loathe writing any non-trivial query in ActiveRecord's query builder. However, I find ActiveRecord does a really good job of getting out of the way when I deem it easier to write some raw SQL. Generally do one of two things. If I have a really big nasty query, I'll implemen…

I would like some kind of middle ground, which is why lately I tend to lean toward using something like Sequel and mapping my own domain entities. Mixing domain entities with persistance has proven to me to be a colossally bad idea.

I'm in the process of writing a rather large new feature for our application that heavily uses aggregates and window functions, and writing queries that try to use these sql feature while also taking advantage of scope chaining is just an exercise in pure frustration.

I think the more I lean on "advanced" SQL features, the more I just think I've probably outgrown Rails personally because it doesn't easily enable the kind of solutions that I need.

Re: It's not Ruby that's slow, it's your database

#186

Post by tpxl »

supermatt wrote: post #152
tpxl wrote: post #145

Earlier quoted context omitted.

> I'm not sure why it just needs to be taken as a given that databases will take 150ms to return any data. Run your database on a t2.small instance on AWS (1 vCPU, 2GiB RAM). Why would you do that, you ask? I don't know, but that's what we got on an old job. This was also used to prove MongoDB is faster than PostgreSQL, even though Mongo was running on-prem on much better hardware.

Nope, single digit millisecond performance for me on those nodes when the tables are cached - anything else and you are complaining about performance of the storage medium.

How much data? We were doing a few million writes/day on peak days and the nodes couldn't keep up.

Re: It's not Ruby that's slow, it's your database

#187

Post by JohnBooty »

rubyist5eva wrote: post #185
JohnBooty wrote: post #79

Earlier quoted context omitted.

I absolutely despise ActiveRecord because it makes non-trivial aggregates a pain to write I'm a SQL guy at heart. I certainly agree with you... I loathe writing any non-trivial query in ActiveRecord's query builder. However, I find ActiveRecord does a really good job of getting out of the way when I deem it easier to write some raw SQL. Generally do one of two things. If I have a really big nasty query, I'll implemen…

I would like some kind of middle ground, which is why lately I tend to lean toward using something like Sequel and mapping my own domain entities. Mixing domain entities with persistance has proven to me to be a colossally bad idea. I'm in the process of writing a rather large new feature for our application that heavily uses aggregates and window functions, and writing queries that try to use these sql feature while…

    why lately I tend to lean toward using something like 
    Sequel and mapping my own domain entities
I'm definitely itching to use something leaner like Sequel in a project. Before coming to Rails I used some nice lean C# ORMs like PetaPoco and Dapper and I really enjoyed that approach.

    I think the more I lean on "advanced" SQL features, 
    the more I just think I've probably outgrown Rails 
    personally because it doesn't easily enable the kind 
    of solutions that I need.
I definitely agree that the "treat the database as a dumb, interchangable storage solution" mentality that infects Rails and other development communities is misguided IMO. Leaning into more advanced database stuff quite frequently the way to go for non-trivial solutions.

In practice, I still find Rails pretty amenable to a data-first approach. Applications tend to be a mix of advanced database stuff and simple CRUD operations. Realistically, a lot of developers are good at one but not the other. So on a given team, a mix of developers is able to play to their strengths.

Re: It's not Ruby that's slow, it's your database

#188

Post by lukaseder »

lmm wrote: post #172
lukaseder wrote: post #144

Earlier quoted context omitted.

SQL is powerful. A DSL that "fixes" things in this area getting all the other language feature interactions right isn't trivial, all the while users have to learn yet another language. Take PRQL for example: https://prql-lang.org . It looks nice, but the examples are very basic. What about window functions, grouping sets, lateral, DML, recursive SQL, pattern matching, pivot/unpivot etc. Might be doable, but perhaps,…

> Besides, every single "fix" will be a proprietary solution, while SQL is an ISO/IEC standard that's here to stay and universally adopted. And yet in practice the fixes end up more portable. How many of the things on your list of non-basic SQL have consistent syntax across databases, yet alone consistent behaviour?

All of them

Re: It's not Ruby that's slow, it's your database

#189

Post by rakoo »

berkes wrote: post #174
kstrauser wrote: post #75

Earlier quoted context omitted.

Ugh, tell me about it. I had an interesting weekend dealing with traffic loads that grew by about 7x in a few days. I wrote about my misadventures at https://blog.freeradical.zone/post/surviving-thriving-throug... . The gist of it is that a RoR “Sidekiq” task queue gets CPU bound after about 25-30 worker threads doing things like making REST API calls to remote servers, querying a database, insert status updates, etc…

I'm experimenting with some alternatives. I prefer Rust (because I speak that) or Ruby (it's my main language) but the Go option gotosocial is my favorite. It lacks a frontend and admin UI, so not ready for prime-time, i guess. But good enough to glue onto a pinafore frontend. And to manage a small instance. The most promising rust alternative, rustodon, is ground to a halt, it seems. A lean, sinatra-based backend co…

I use pleroma, written in Elixir and hosting everything properly inside pg, for a personal instance and it just works. No magic knobs to turn, no significant delays.
Post reply on HN