Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Solid Queue 1.6.0 now supports fiber workers

github.com

Post reply on HN
Print view View original

31–40 of 41 posts

Re: Solid Queue 1.6.0 now supports fiber workers

#31

Post by mrinterweb »

vdombr wrote: post #8
ramon156 wrote: post #3

So fibers are a lot like threads but they're more scoped to a task that can be paused and resumed, that's kinda cool

It’s more like goroutines or other lightweight concurrency mechanisms. If threads are OS-level concurrency primitives, fibers are scheduled within Ruby itself, which makes them much more efficient than threads. In fact, I got the following results in HTTP benchmark tests: Go: * Latency under stable load: p95 0.25–5.32 ms, p99 2.20–9.92 ms * Memory: 23–31 MB RSS across HTTP scenarios Ruby: * Latency under stable load:…

That ruby fiber vs go goroutine benchmark is interesting. The 4-10x memory use doesn't surprise me, but the near performance does. I'm guessing there is more of a gap with the p50.

Re: Solid Queue 1.6.0 now supports fiber workers

#32

Post by mrinterweb »

This looks fantastic for a common async workflow I use. I often use one job to fan out multiple individual http request jobs. The reason I prefer jobs for this is easy and consistent retry logic, and durability. I want to make sure those HTTP requests eventually go through. Fibers would be much better suited for this. So much of work that goes onto work queues is IO bound, and fibers are a great fit for that.

Re: Solid Queue 1.6.0 now supports fiber workers

#33

Post by QGQBGdeZREunxLe »

ksec wrote: post #28
thibaut_barrere wrote: post #20

Earlier quoted context omitted.

Same path & agreed. The amount of things you do not have to care about with BEAM/Elixir compared to Rails is really interesting.

>BEAM/Elixir compared to Rails Compared to Ruby. Elixir land doesn't have something similar to Rails, and phoenix is not it.

What do you think is missing from Phoenix that you have in Rails?

Re: Solid Queue 1.6.0 now supports fiber workers

#35

Post by bingemaker »

QGQBGdeZREunxLe wrote: post #33
ksec wrote: post #28

Earlier quoted context omitted.

>BEAM/Elixir compared to Rails Compared to Ruby. Elixir land doesn't have something similar to Rails, and phoenix is not it.

What do you think is missing from Phoenix that you have in Rails?

Ecosystem?

Re: Solid Queue 1.6.0 now supports fiber workers

#36

Post by QGQBGdeZREunxLe »

pqdbr wrote: post #12
Lio wrote: post #2

This is a nice update. Is it possible to either have multiple ractors dispatching jobs with fibres or to set up multiple queues with different strategies? E.g. one for IO bound and one for CPU bound? With Sidekiq I’ve had luck having workers running on Truffleruby but generally don’t use it for my main rails apps.

You can, and Carmine (who coded this update) wrote exactly about this on this blog post: https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/ From his article: One backend, two modes Fiber mode isn’t universally better. CPU-bound jobs get nothing from it, and blocking libraries or C extensions that do not cooperate with Ruby’s fiber scheduler stall the reactor. And that’s fine – you don’t have to pick one. As…

That post, and the linked post, provide very good context to understand these changes.

https://paolino.me/async-ruby-is-the-future/

Re: Solid Queue 1.6.0 now supports fiber workers

#37

Post by tt_dev »

symfoniq wrote: post #13

As someone who spent 15+ years doing Ruby/Rails, it’s nice to see this land. That said, these days you’ll pry the BEAM from my cold, dead hands. It’s hard to go back to any other concurrency story.

How has agentic coding been with rails? What has your experience been like?

Took me sometime to get used to the magic imports and metaclasses (ruby I know not rails). Curious how well llms navigate this

Re: Solid Queue 1.6.0 now supports fiber workers

#38

Post by jdkoeck »

QGQBGdeZREunxLe wrote: post #33
ksec wrote: post #28

Earlier quoted context omitted.

>BEAM/Elixir compared to Rails Compared to Ruby. Elixir land doesn't have something similar to Rails, and phoenix is not it.

What do you think is missing from Phoenix that you have in Rails?

Fascist leadership?

Re: Solid Queue 1.6.0 now supports fiber workers

#39

Post by adriand »

tt_dev wrote: post #37
symfoniq wrote: post #13

As someone who spent 15+ years doing Ruby/Rails, it’s nice to see this land. That said, these days you’ll pry the BEAM from my cold, dead hands. It’s hard to go back to any other concurrency story.

How has agentic coding been with rails? What has your experience been like? Took me sometime to get used to the magic imports and metaclasses (ruby I know not rails). Curious how well llms navigate this

Not the OP, but I’m doing a ton of it and it works great. I had to decide what to use for my startup and for a few years I’d stopped using Rails. I considered using full stack node/Typescript/etc but Rails just comes with so much stuff that makes it so easy to build things. Solid Queue is one of those things, my whole data process architecture is based on async jobs and it’s really cool to have something that is native to the platform and has no dependencies beyond Postgres which I’m already using anyway.
Post reply on HN