Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mike Perham, Creator of Sidekiq: From Employment to Independence

codecodeship.com

Post reply on HN
Print view View original

11–20 of 115 posts

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#11

Post by shubhamjain »

This is one of the best businesses I have heard of. Even 6-7 years back, Sidekiq was grossing $80k/month [1]. I imagine it must be much more now. No servers to maintain, no employees, minimal support work and Mike has the complete freedom to work on updates whenever he prefers. Almost entire revenue minus payment processing costs must be profit. Imagine making $1M / yr working only 10-20hrs per week (Don't wish to presume here, but that's my estimate of the man-hours it costs to run).

> I still have 0 employees and don't plan to hire. I tune my business processes to run as lean as possible: Most of my customers are on credit card so their payments are automatic. The gem servers take about one day of maintenance per year. I can't really outsource much of my support work because it is so technical and specialized.

> My gem server is a $6 droplet on DigitalOcean. Because gems are just static files, that little droplet can handle millions of requests per day with just Apache. Oh, and I run three servers in parallel for failover purposes for a grand total of $18/mo.

I wonder what makes Sidekiq special? I don't imagine any equivalent product (background job processing) in the other languages is raking in that much, if making any revenue in the first place.

[1]: https://www.indiehackers.com/product/sidekiq

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#13

Post by Colex »

durkie wrote: post #10
phamilton wrote: post #8

Earlier quoted context omitted.

I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash mea…

how often do your workers crash? i rely heavily on sidekiq and don't think I see this very often, if ever.

it’s not uncommon to lose jobs in sidekiq if you heavily rely on it and have a lot of jobs running. If using the free version for mission critical jobs, I usually run that task as a cron job to ensure that it will re-try if the job is lost.

I have in the past monitored how many jobs were lost and, although a small percentage, it was still recurring thing.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#15

Post by aqme28 »

phamilton wrote: post #8
jorge-d wrote: post #3

Earlier quoted context omitted.

Well Sidekiq is free to use. It's only the pro version that he charges and the free version code is open source. I don't see the problem in having that kind of business model, it still allows the community to thrive and offers entreprises a way to have premium support. Plus it allows him to invest more time in maintaining the free version.

I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash mea…

When we used Sidekiq in production, not only did I never see crashes that lost us jobs, but there are also ways to protect yourself from that. I highly recommend writing your jobs to be idempotent.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#16

Post by ch4s3 »

john_fushi wrote: post #2

I guess Matsumoto should have charged for Ruby. I'm sure Mike Perham and Derek Kraan would have been glad to pay for that and that the Ruby community would be in a strong and healthy shape.

This is a very bad take. From an OSS perspective languages can attract large communities of contributors and corporate sponsors because of their broad appeal and utility. Specialized libraries will have more trouble doing both and may need alternate models to sustain themselves. From a business perspective, Mike offers not only a free version but the paid enterprise version comes with support from Mike and his team, which is something you can't get from a language owner unless you outright hire them or they run a consultancy.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#17

Post by iudqnolq »

shubhamjain wrote: post #11

This is one of the best businesses I have heard of. Even 6-7 years back, Sidekiq was grossing $80k/month [1]. I imagine it must be much more now. No servers to maintain, no employees, minimal support work and Mike has the complete freedom to work on updates whenever he prefers. Almost entire revenue minus payment processing costs must be profit. Imagine making $1M / yr working only 10-20hrs per week (Don't wish to pr…

Oban is a roughly equivalent elixir library. They claim to have hundreds of customers and their cheapest plan is $500/yr, so maybe they make an order of magnitude less? Considering the relative language popularity that seems pretty good.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#18

Post by phamilton »

durkie wrote: post #10
phamilton wrote: post #8

Earlier quoted context omitted.

I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash mea…

how often do your workers crash? i rely heavily on sidekiq and don't think I see this very often, if ever.

We process around 50M sidekiq jobs a day across a few hundred workers on a heavily autoscaled infrastructure.

Over the past week there were 2 jobs that would have been lost if not for superfetch.

It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters.

Edit for additional color: One of the most common crashes we'll see is OutOfMemory. We run in a containerized environment and if a rogue job uses too much memory (or a deploy drastically changes our memory footprint) the container will be killed. In that scenario, the job is not placed back into the queue. SuperFetch is able to recover them, albeit with really lose guarantees around "when".

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#19

Post by phamilton »

aqme28 wrote: post #15
phamilton wrote: post #8

Earlier quoted context omitted.

I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash mea…

When we used Sidekiq in production, not only did I never see crashes that lost us jobs, but there are also ways to protect yourself from that. I highly recommend writing your jobs to be idempotent.

Idempotence doesn't solve this problem. The jobs are all idempotent. The problem is that jobs will never be retried if a crash occurs.

This doesn't happen at a high rate, but it happens more than zero times per week for us. We pay for Sidekiq Pro and have superfetch enabled so we are protected. If we didn't do so we'd need to create some additional infra to detect jobs that were never properly run and re-run them.

Post reply on HN