Live data from Hacker News

Mike Perham, Creator of Sidekiq: From Employment to Independence

codecodeship.com

61–70 of 115 posts

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#61
post #43

Earlier quoted context omitted.

How are crashes under your control? Again they aren't talking about uncaught exceptions, but crashes. So maybe the server gets unplugged, the network disconnects, etc.

To me 'crash' means any unexpected termination, whether it's caused by an uncaught exception, OOM, or hardware/network issues. I guess you can say that hardware issues on your host aren't under your control, but it's under your control to find a host that doesn't have these issues. And not even a full-on ACID database is going to be 100% reliable if you yank the power cord at the wrong moment.

I hope my tone doesn't come across as rude or too argumentative, but I think your understanding is a bit inaccurate.

> it's under your control to find a host that doesn't have these issues

All hosts will have these issues, the only question is how often. If you need 100% consistency, then you can't use the free Sidekiq. Personally, I've never needed Sidekiq pro (as these kinds of crashes are extremely rare). But this will depend on your scale and use case.

> And not even a full-on ACID database is going to be 100% reliable if you yank the power cord at the wrong moment

This is only true if there's bugs in the DB, or some underlying disk corruption happens. The whole point of an ACID database is that they're atomic, durable, and consistent, even in the worst case scenario. If a power failure corrupted my SQL database I would feel very betrayed by the database.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#62
post #56

Mike Perham's Sidekiq was what I modeled my own business, EmailEngine, after. I took my decade's worth of expertise in email protocol implementations (I also run the open-source Nodemailer library) and built an email gateway app my customers can download and run on their machines or servers. Just like Sidekiq Pro, the customers need an active subscription to run it, but the app runs on their premises, and they mainta…

Unsolicited feedback, always link to your business in your posts.

I googled “emailengine” and it’s wasn’t super obvious which was your business since so much paid ads exist for that search.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#63
post #54

Earlier quoted context omitted.

You act as if some percentage less than 100% of Ruby users have extensive experience with JS as well... OK, maybe it's only, say, 96% of Ruby users, or 100% of Rails users at least

See like, this isn't constructive maligning at all, it's more undirected blanket unniceness. It costs nothing to actually put down a reason for being upset, versus having totally generic downputs. Very few things are truly rotten to the core, most bad things have some bad aspects but could be much better if ___. If something is rotten to the core the central articles of faith for why that's so should be declared. Ple…

> It costs nothing to actually put down a reason for being upset, versus having totally generic downputs

Actually, it does cost something- time and effort- but thanks to the latest AI, I can cut some of that out right now:

As an Elixir enthusiast, I'd like to point out some technical criticisms of JavaScript and its ecosystem, particularly in comparison to Elixir's approach to dependency management and community.

A) Dependency bloat: The JavaScript ecosystem, especially when working with Node.js, often suffers from excessive dependencies. Even when you require just a few dependencies for your project, you may end up pulling in a large number of indirect dependencies. This can lead to increased complexity, slower build times, a larger attack surface for potential security vulnerabilities, and an ever-growing workload spent applying security fixes that have nothing to directly do with the purpose of the web app.

In contrast, Elixir's dependency management system, powered by Mix and Hex, encourages a more conservative approach. Elixir libraries are often designed with minimal dependencies, and the community values self-contained, focused libraries.

B) Security concerns: The extensive use of third-party packages in JavaScript projects can expose your application to security risks. When you pull in a large number of packages, you become responsible for ensuring that all of those dependencies are up-to-date and secure.

Elixir's ecosystem tends to have fewer dependencies, reducing the potential attack surface. Additionally, Elixir's focus on immutability and the actor model can help minimize the impact of security vulnerabilities.

C) Lack of native concurrency and parallelism: JavaScript is single-threaded by design, which can limit its ability to take full advantage of multi-core systems. While Node.js introduced asynchronous I/O to help mitigate this issue, it can still be challenging to build highly concurrent applications in JavaScript.

Elixir, built on the Erlang VM (BEAM), provides excellent support for concurrency and parallelism through lightweight processes, message-passing, and the actor model. This allows Elixir applications to scale across multiple cores and handle a large number of simultaneous connections more efficiently than most JavaScript applications.

D) Callback hell and async/await complexity: JavaScript has traditionally relied on callbacks for asynchronous programming, which can lead to a phenomenon known as "callback hell" when dealing with nested callbacks. While the introduction of async/await has improved this situation, it can still be cumbersome and confusing for developers.

Elixir offers a cleaner approach to concurrency with its process-based model and message-passing mechanism, enabling easier-to-read and maintainable code.

E) Mutable data- One of the key differences between Elixir and JavaScript is the way they handle data. Elixir enforces immutability, meaning that once a data structure is created, it cannot be modified. Instead, operations on data structures in Elixir return new versions of the data, leaving the original untouched. This approach offers several advantages compared to JavaScript's mutable data structures:

  1) Predictability and easier reasoning: Immutability in Elixir makes it easier to reason about and understand your code. Since data structures cannot be changed once created, you don't have to worry about unintended side effects or data being altered in unexpected ways. This results in more predictable and maintainable code.

  2) Concurrency safety: Elixir's immutable data structures simplify concurrent programming by eliminating the need for locks, mutexes, or other synchronization primitives. This is because multiple processes can safely read and share the same data without the risk of data corruption due to concurrent modifications. In JavaScript, managing shared mutable data in concurrent scenarios can be error-prone and challenging.

  3) Functional programming: Immutability is a core principle of functional programming, and Elixir is heavily influenced by this paradigm. Functional programming promotes the use of pure functions that do not produce side effects, which can make code more modular, reusable, and easier to test. While JavaScript supports functional programming concepts, its mutable data structures can make it more challenging to adhere to functional programming principles.

  4) Reduced cognitive load: Immutable data in Elixir means that developers don't have to keep as much track of changing state as they read or write code. This can make it easier to understand the flow of data and logic in your application, leading to a more pleasant and efficient development experience and ultimately, fewer bugs generated per LOC.
F) Community values: The JavaScript community is known for its rapid pace of change and the constant introduction of new libraries, frameworks, and tools. While this can drive innovation, it can also lead to fragmentation and inconsistency across projects.

The Elixir community tends to be more focused on stability, long-term support, and collaboration. This can result in a more consistent and cohesive ecosystem, where tools and libraries are more likely to work together seamlessly.

In conclusion, while JavaScript has its strengths and has undoubtedly played a vital role in the growth of web development, its dependency management and ecosystem can be seen as less than ideal from an Elixir enthusiast's perspective. Elixir's dependency management system, focus on concurrency, and community values offer a more robust, sane and secure alternative.

(How's that for "actually putting down a reason" as to why Javascript is terrible? Hey, you asked!)

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#64
post #3

Earlier quoted context omitted.

Well Sidekiq is free to use. It's only the pro version that he charges and the free version code is open source. I don't see the problem in having that kind of business model, it still allows the community to thrive and offers entreprises a way to have premium support. Plus it allows him to invest more time in maintaining the free version.

I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash mea…

Exactly why we refuse to use Sidekiq. “Hey, you have to pay to guarantee your jobs won’t just vanish”.

No thanks.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#65
post #62
post #56

Mike Perham's Sidekiq was what I modeled my own business, EmailEngine, after. I took my decade's worth of expertise in email protocol implementations (I also run the open-source Nodemailer library) and built an email gateway app my customers can download and run on their machines or servers. Just like Sidekiq Pro, the customers need an active subscription to run it, but the app runs on their premises, and they mainta…

Unsolicited feedback, always link to your business in your posts. I googled “emailengine” and it’s wasn’t super obvious which was your business since so much paid ads exist for that search.

Oh, yeah, I forgot my pitch. The link is https://emailengine.app - EmailEngine acts as a mail client, basically the same way Thunderbird runs on desktop, or the iPhone Mail on the phone, but instead of a GUI it has REST API and instead of desktop notifications it sends JSON webhooks. And instead of a single email account, it can manage thousands of accounts.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#66
post #53
post #47

Earlier quoted context omitted.

Hangfire and Oban are two other background job systems which have a commercial aspect. AFAIK Hangfire has been around almost as long as Sidekiq and is still actively supported. I'm closer to $10m than $1m in annual revenue now. My take on Sidekiq's secret sauce: a job system is a distributed system. Most of Sidekiq's commercial features are available as OSS gems but the complexity sneaks up on you as you integrate 3-…

Super excited of you. - Do you have any outside hired help? I see you still have no employees, but do you contract with anyone to do customer support as an example? - How many approx customers do you have these days? $10M revenues = 10,000 customers. Is that roughly correct, if so - wahoo, congrats.

I’m closer to 2000 customers.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#67
post #61

Earlier quoted context omitted.

To me 'crash' means any unexpected termination, whether it's caused by an uncaught exception, OOM, or hardware/network issues. I guess you can say that hardware issues on your host aren't under your control, but it's under your control to find a host that doesn't have these issues. And not even a full-on ACID database is going to be 100% reliable if you yank the power cord at the wrong moment.

I hope my tone doesn't come across as rude or too argumentative, but I think your understanding is a bit inaccurate. > it's under your control to find a host that doesn't have these issues All hosts will have these issues, the only question is how often. If you need 100% consistency, then you can't use the free Sidekiq. Personally, I've never needed Sidekiq pro (as these kinds of crashes are extremely rare). But this…

It wouldn’t be corrupted, but in-flight transactions could fail to commit, just like queued jobs can be lost with sidekiq. The failure modes are similar.

I take your point that at a certain scale, hardware failure is inevitable, but if you’re running that many servers, you can afford sidekiq’s enterprise plan. It’s not something that will realistically happen if you’re just running like 20 instances on AWS. It’s perfectly reasonable to charge extra for something only large organizations with huge infrastructure budgets need.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#68

Earlier quoted context omitted.

And he only started because companies were telling him it would be easier if they could just pay him.

This is great. Reminds me of the Derek Sivers post [1], "Don't start a business until people are asking you to." 1. https://sive.rs/asking

Sorry, this is bogus.

"If I had asked people what they wanted, they would have said faster horses." or whatever the quote is.

There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there.

Personally, I am one of those people who started a business based on an idea with no validation before launching it.

I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#69

This is one of the best businesses I have heard of. Even 6-7 years back, Sidekiq was grossing $80k/month [1]. I imagine it must be much more now. No servers to maintain, no employees, minimal support work and Mike has the complete freedom to work on updates whenever he prefers. Almost entire revenue minus payment processing costs must be profit. Imagine making $1M / yr working only 10-20hrs per week (Don't wish to pr…

Just like the underlying platform it was built for (Rails) - it is so stupidly simple to deploy Sidekiq. Performance is great and a lot of complexity is well hidden. This makes it a great product overall.

Re: Mike Perham, Creator of Sidekiq: From Employment to Independence

#70

This is one of the best businesses I have heard of. Even 6-7 years back, Sidekiq was grossing $80k/month [1]. I imagine it must be much more now. No servers to maintain, no employees, minimal support work and Mike has the complete freedom to work on updates whenever he prefers. Almost entire revenue minus payment processing costs must be profit. Imagine making $1M / yr working only 10-20hrs per week (Don't wish to pr…

> I wonder what makes Sidekiq special? Time, know-how, and long term dedication are the key ingredients from my perspective. Maybe he's working 10-20 hours now, but there's support and development every day for the past 11 years. > I don't imagine any equivalent product (background job processing) in the other languages is raking in that much, if making any revenue in the first place. I guarantee, there are backgroun…

> I guarantee, there are background job processors in other languages making revenue.

I had no idea until someone just hinted me that this was self-referential... LOL, apologies for the now-embarrassing other comment I made LOL (I only know you as "sorentwo", whoops!)

Post reply on HN