Outside of Rails I used to write map/reduce jobs using ruby and a jruby jar file to deploy the job to our hadoop cluster. Startup time for each job is super slow due to jruby but developer ergonomics mattered to me more than efficiency here. Slow but who cares.
It's not Ruby that's slow, it's your database
61–70 of 203 posts
Re: It's not Ruby that's slow, it's your database
#62Earlier quoted context omitted.
Bad indexing, no indexing, and fragmented indexes seem to cause no end of issues, and there always seem to be shortages of people capable of fixing it properly.
They’re out there—but I just ran EXPLAIN ANALYZE on my company’s search for them, and the query planner’s not too happy. It has to scan the entire resume heap before doing an on-site join with candidates, all while dealing with resource contention from queries of other employers. I know recruiters and e.g. triplebyte, stackoverflow all offer indexes to speed up this search, but in our case those indexes wouldn’t fit…
Why does the index need to fit in "cash" (cache RAM)? Generally an index on disk, especially SSD, is far faster than traversing the entire dataset, because it allows the query executor to quickly narrow it down. Even if this requires some disk IO, it's a lot faster than doing all the disk IO for the entire dataset.
Re: It's not Ruby that's slow, it's your database
#63Earlier quoted context omitted.
Bad indexing, no indexing, and fragmented indexes seem to cause no end of issues, and there always seem to be shortages of people capable of fixing it properly.
They’re out there—but I just ran EXPLAIN ANALYZE on my company’s search for them, and the query planner’s not too happy. It has to scan the entire resume heap before doing an on-site join with candidates, all while dealing with resource contention from queries of other employers. I know recruiters and e.g. triplebyte, stackoverflow all offer indexes to speed up this search, but in our case those indexes wouldn’t fit…
That was very very good, but I wonder what sort of latency you experienced on the query that produced that result.
Re: It's not Ruby that's slow, it's your database
#64Earlier quoted context omitted.
There is always an alternative to writing SQL queries by hand, and it's usually a better one IME. Any ORM worth its salt will let you do the query in your blog post via the ORM, as a single query.
"Say now we want to display the album list sorted by album’s duration, shortest first." How would we do this in an ORM like ActiveRecord or Django ORM, without generating multiple queries?
In Django, you would write an annotation as documented at https://docs.djangoproject.com/en/4.1/topics/db/aggregation/...
So something like
Album.objects.annotate(duration=Sum('track__length')).order_by('duration')
Re: It's not Ruby that's slow, it's your database
#65Earlier quoted context omitted.
None of the major Python implementations do any AoT compilation. Python is probably the poster-child language of "interpreted". It's run, line-by-line through an interpreter, and turned into byte-code, which is executed by the VM. There's no compilation, and in CPython, very little optimisation. PyPy is obviously different on that last point as it's a JIT, but the larger point still stands.
> It's run, line-by-line through an interpreter, and turned into byte-code, which is executed by the VM. Just a small technical correction: you can pre-complie to .py source files to .pyc bytecode files without waiting for the interpreter to do it line by line, many distro package managers already perform this step automatically when a package is installed. Though you're right in the sense that it doesn't really impr…
There are still some basic optimizations the Python compiler makes. It has, for example, a keyhole optimizer, and some built in optimizations around integer objects to name two.
It’s nothing too fancy, but it still does the basics.
Re: It's not Ruby that's slow, it's your database
#66No, Ruby is slow. Its GC is ungodly slow. And you can’t truly multithread, thus you can’t properly parallelize or maximize concurrency.
Yeah antidotal, but I was doing some tests of an existing Rails app API compared to Hasura (erlang GraphQL server) talking to same DB, and querying same data was like 100x faster in Hasura..
anecdotal: based on personal experience
antidotal: based on curing a deadly poison
Re: It's not Ruby that's slow, it's your database
#67Database: slow. Ruby: not slow. Our next steps are clear: we must reimplement the database using ruby.
I already did that. I decided to keep things simple, so I just built a web app that proxies raw SQL to mysql and returns the results. So it's even better - your database is written in ruby AND has an http json api for wider compatibility!
Re: It's not Ruby that's slow, it's your database
#68In my experience 9/10 web application performance problems are related to database interactions. Is the query using an index? Are you hammering your DB with 300 N+1 queries to render an API response. These are usually the biggest offenders when poor performance is observed. If you're interactions with you DB are bad, it doesn't matter what programming language you use, performance will be mostly equally awful regardl…
As an ex-DBA, the lack of understanding - and even basic performance monitoring - of your average Sr Software dev is mind blowing. As a side note, the willingness of those same devs to allow queries of arbitrary size and constraints hit their db is equally frustrating. “As long as we prevent against SQL injection, who cares?”
Before NewRelic this was so much worse too.
Re: It's not Ruby that's slow, it's your database
#69Earlier quoted context omitted.
"Say now we want to display the album list sorted by album’s duration, shortest first." How would we do this in an ORM like ActiveRecord or Django ORM, without generating multiple queries?
In Django, you would write an annotation as documented at https://docs.djangoproject.com/en/4.1/topics/db/aggregation/... So something like Album.objects.annotate(duration=Sum('track__length')).order_by('duration') There are also libraries that enable you to define an annotation or aggregation inside a model, that you can then get with a call like select_properties, similar to the built-in select_related (which you u…
Thanks. I haven't used Django for ages, but do recall using annotate() in the past. At that time, I didn't need to worry about performance, so didn't look into what happens under the hood, i.e. whether the call generates more than one SQL query.
Re: It's not Ruby that's slow, it's your database
#70Earlier quoted context omitted.
How many developers will really need it though? It's fast enough for Stripe's API, it's fast enough for Shopify; hardware in exchange for productivity is a pretty fair trade.
Except most companies won't attract the same top-tier Ruby/Rails talent. For example, it looks like 1/3 of the current Rails Core Team work at Shopify, including the person who is the #1 all-time Rails contributor.
I may be misunderstanding you, but can you elaborate on why you believe you need "top-tier" Ruby/Rails talent to run a performant Rails application?
In my experience, you need no more than mid-level talent.
You need a certain % of people who are strong at working with databases to ensure good (normalized, indexed, etc) database design. You also need a certain level of organizational will to avoid letting a Rails monolith get absolutely out of control, but this is less "a need for top-level talent" and more "just stick to boring old Ruby/Rails best practices."
Those are, of course, things you need when working with any stack. I don't find them to be Ruby/Rails specific.