Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Deep Dive into Solid Queue for Ruby on Rails

blog.appsignal.com

Post reply on HN
Print view View original

31–40 of 50 posts

Re: A Deep Dive into Solid Queue for Ruby on Rails

#31

Post by RangerScience »

teaearlgraycold wrote: post #4
nkraft11 wrote: post #2

I briefly worked at a YC company that was a ruby shop. Their answer to every performance problem was to stick it on a queue. There were, I don’t know, dozens of them. Then they decided they needed to be multi-region, because reasons. But the queues weren’t set up to be multi-region, so they built an entirely new service that’s job was to decide which queue in which region jobs needed to go on. So now you had jobs cri…

Ruby and the age of “I don’t care what type this variable is, it quacks like a duck!” is over and dead. Improvements to type systems have shown there is a better way to do software development.

Typescript is literally a language describing what the quack “sounds like” so you can attempt to ensure any particular variable makes those kinds of quacks. Typescript doesn’t care that it also quacks like a dog.

Plus, Ruby has lots of easy ways for you to check typing, if you want to.

Re: A Deep Dive into Solid Queue for Ruby on Rails

#32

Post by bigtunacan »

zdragnar wrote: post #23
throwaway493943 wrote: post #14

Earlier quoted context omitted.

I'm going to be brave (but still use a throwaway) and ask the dumb question - what is wrong with putting things in queues to help with performance problems? If some endpoint is too slow to return a response to the frontend within a reasonable time, enqueueing it via a worker makes sense to me. That doesn't cover all performance issues but it handles a lot of them. You should also do things like optimize SQL queries,…

There's two primary areas that I've seen teams get bitten by this personally: 1) Designers don't understand that things are going to happen async, and the UI ends up wanting to make assumptions that everything is happening in real time. Even if it works with the current design, it's one small change away from being impossible to implement. This is a general difficulty with working in eventually consistent systems, bu…

We have 100s of queues processing millions of jobs in sidekiq queues at any given time.

These are data and compute heavy workloads that take anywhere from minutes to hours for a request to be completed, but the UI takes this into account.

Users submit a request and then continue onto whatever is the next thing they intend to do and then they can subscribe to various async notification channels.

It’s not the right choice for everything, but it’s the right choice for something’s.

Re: A Deep Dive into Solid Queue for Ruby on Rails

#34

Post by hschne »

pqdbr wrote: post #29

Great writeup. I'd love to know more about how the Supervisor works, and how it. "fork[s] a separate process for each supervised worker/dispatcher/scheduler". In a Rails app served with Puma, I've always had a hard time understanding what would be the canonical way for having a loop doing some periodic work. I know Puma has plugin support but I don't see much documentation there. Forking a process / threads is someth…

Under the hood, it uses good ol' fork and keeps track of the generated process IDs.

It's surprisingly simple. You can check out the relevant source here: https://github.com/rails/solid_queue/blob/main/lib%2Fsolid_q...

Re: A Deep Dive into Solid Queue for Ruby on Rails

#35

Post by zem »

man, that brings back memories - very early in my career I tried to use postgres as a task queue, thinking that with O(hundreds) of jobs it wasn't worth setting up something like rabbitmq. sadly I knew pretty much nothing about db design and the performance was horrible, ended up ripping it out and installing rabbitmq after all (and having a whole new set of headaches with random rabbit admin issues but at least when it worked it was fast)

Re: A Deep Dive into Solid Queue for Ruby on Rails

#36

Post by pmontra »

nkraft11 wrote: post #2

I briefly worked at a YC company that was a ruby shop. Their answer to every performance problem was to stick it on a queue. There were, I don’t know, dozens of them. Then they decided they needed to be multi-region, because reasons. But the queues weren’t set up to be multi-region, so they built an entirely new service that’s job was to decide which queue in which region jobs needed to go on. So now you had jobs cri…

We were offloading to jobs every long running activity in the Elixir/Phoenix project I've been working on years ago. There is no other way. The response to a web request must complete in a short time and free the server for further requests.

We solved debugging by sending all log lines to a centralized server. We were running on the Google cloud.

We were not multiregion though.

My current Rails project uses sidekiq a lot to send mail, generate PDFs, any activity that does not have to necessarily complete before we return the response. We keep the interactive web app up to date by websockets and with callbacks for clients using our public API. I don't think we would have done it differently in any other language.

By the way, we built our slimmer version of sidekiq for Elixir because the language plus the OTP libraries have a lot of functionality but we still need to persist jobs, retry them even after a complete reboot, exponential back off, etc.

Re: A Deep Dive into Solid Queue for Ruby on Rails

#37

Post by pmontra »

nkraft11 wrote: post #26
zdragnar wrote: post #23

Earlier quoted context omitted.

There's two primary areas that I've seen teams get bitten by this personally: 1) Designers don't understand that things are going to happen async, and the UI ends up wanting to make assumptions that everything is happening in real time. Even if it works with the current design, it's one small change away from being impossible to implement. This is a general difficulty with working in eventually consistent systems, bu…

Error handling was a huge issue, along with other weird distributed system bugs. Backed up queues, job shedding, thundering herds, you name it. When you have jobs on queues kicking off new jobs on different queues, tracing issues is just miserable. Sure, it's not a problem of ruby per se, but engineers would basically just throw their hands up and say "ruby can't handle this" and sidekiq became the One True Way™.

Maybe "ruby can't handle this" was a short form for "we can't run this in the Rails controller because the response would take too long" possibly calling 3rd party APIs, "and we would run out of threads."

Anything running in sidekiq is written in Ruby too.

Re: A Deep Dive into Solid Queue for Ruby on Rails

#38

Post by riffraff »

ch4s3 wrote: post #15
teaearlgraycold wrote: post #4

Earlier quoted context omitted.

Ruby and the age of “I don’t care what type this variable is, it quacks like a duck!” is over and dead. Improvements to type systems have shown there is a better way to do software development.

Maybe, the cycle has happened before and maybe come back around again. Dynamic typing is really nice when most of your data looks like bags of strings. Compilers and tools just don’t add a lot when you’re passing around glorified blobs of stringy json-like stuff. Type gymnastics can eat a lot of time where you could otherwise be shipping something useful.

I would argue when you're passing around stringy JSON-like thingies is when typing is most useful :)

You're not going to misuse an API that takes a Person or Cart, but mixing up two hashes cause you used two different strings as keys can happen easily.

(I do think dynamic typing is mostly fine, but I do wish ruby had optional static typing with some nice syntax instead of RBS)

Re: A Deep Dive into Solid Queue for Ruby on Rails

#39

Post by ggsp »

WillPostForFood wrote: post #22
aaronblohowiak wrote: post #19

Earlier quoted context omitted.

Resque was a staple for a long long time. In the jvm world, throw everything into Kafka is also a staple of a lot of "enterprise" shops. Or SQS for AWS places I've worked at. I think it is not a ruby language thing, but a certain kind of architecture thing.

True that it is not uncommon to use Sidekiq or Resque , but Rails 8 is going to be the first version to ship with a queuing system (SolidQueue), later this year. So queueing has been an add-on for 20 years. I don't think it is quite a staple.

Rails 8 came out in November, and `rails new` generates an app with the solid trio in the Gemfile. Been fun playing around with it for new side projects :)

Re: A Deep Dive into Solid Queue for Ruby on Rails

#40

Post by Lio »

morkalork wrote: post #10
trevorhinesley wrote: post #9

Earlier quoted context omitted.

I’ve never felt like “throw everything into a queue” was a mindset within the Ruby community, nor have we done that at my companies. And multi-region is a business decision.

Doesn't Ruby, like Python, have a GIL? I always found that one is enough to encourage some "premature scalable architecture"

That depends on the Ruby implementation.

MRI (CRuby) has a GVL which is why you might use a forking web server like Puma or Pitchfork.

JRuby and TruffleRuby though have true multi-threading and no GVL.

I’ve used the Concurrent Ruby library with JRuby and Tomcat quite a bit and find works very well for what I need.

Post reply on HN