Live data from Hacker News

My thoughts after using Clojure for about a month

acdw.net

171–180 of 204 posts

Re: My thoughts after using Clojure for about a month

#171

Earlier quoted context omitted.

"As a rule of thumb, if your application never has 10,000 virtual threads or more, it is unlikely to benefit from virtual threads." https://docs.oracle.com/en/java/javase/21/core/virtual-threa...

BEAM threads are kinda magicsauce tho, instructions have a cost and after a certain cost total (quantums) the scheduler can divert to another virt thread to guarantee forward progress. Also the immutability rules etc make it easier to optimize this switching.

Eh, reduction counting isn't magic. Golang manages similar preemption semantics without counting that many operations (some tight loops do have barriers inserted every so often, but that's the exception and not the rule). And reduction counting has some serious costs! It slows the runtime down a shitload (and the BEAM is already in the bottom half of interpreted language runtimes by speed) and makes lots of JIT-flavored runtime optimizations slower or harder to implement.

I like immutability too; I wish Java and Golang did more of it. It costs a lot in terms of unexpected copies in the BEAM though, there's less copy-elision optimization than you'd think. That especially bites if you're doing a ton of message passing, because of how process heaps are implemented and how garbage collection (traditional or ETS/ThreadProgress-based) works.

I think what I want is something like Golang but with goroutine-based ownership semantics (or Rust with the Go runtime and goroutines): en excellent scheduler for extremely light-weight green threads, no refcounting or reduction counting, and all the clever optimizations around channel sending and copy elision--but no ability to use a value after it's sent to a channel, and only channel-based access to shared global state. That'd get most of the benefits of process-local heaps but without the (copying, cache/memory fragmentation) drawbacks.

Re: My thoughts after using Clojure for about a month

#172

Earlier quoted context omitted.

The question is appeal to whom . Large employers want you to learn popular languages so that you're a commodity in a liquid market. But that's not a signal of economic value; it's the reverse. It's saying "I'm entirely replaceable." What you can predict is that those employers for whom clojure (or any other minority language) is either acceptable or preferred are deciding that they don't want commodity, low-margin em…

its not large employers only. you need a steady stream and large enough pool of candidates for any size of a business. if you limit yourself to a tiny almost invisible slice you are betting your business on finding candidates and relying on that small pool of talent. employees come and go and you would have to deal with this issue constantly.

Yes not only large employers, but it's easiest to think in those terms.

And it's only true that "employees come and go" is a problem if 1. you assume turnover has to be high, and 2. you don't know where to find qualified people you need. Turnover will be high if you assume you need commodity devs and don't, for instance, invest in their skills (like teaching them minority languages if that's your thing).

If I see a company hiring for "python developers" or "java developers" at this point I absolutely know what sort of problems their codebases will have, because they're in the commodity market and treating development as a cost centre to be minimised. Which leads to lower salaries, which drives higher turnover.

It's all self-fulfilling.

Re: My thoughts after using Clojure for about a month

#173

Earlier quoted context omitted.

> that would be the reason why syntax does matter, wouldn't it? nobody wants leaky abstractions! Well I thought we were describing our current reality, not our _desired_ one? Yes nobody wants leaky abstractions and yes they are everywhere. Syntax matters insofar as to discourage bad habits, is what I'd refine from my previous statements. Most programmers go for the default so defaults and syntax that steers you the r…

> That being said, has Akka started making full use of JVM's new green threads? Has Java itself started introducing immutability and STM / share-nothing as first-class citizens? If not, then by the "programmers reach for the defaults first" rule above I'd think Java is not yet ready. > OK Clojure has these amazing libraries, kudos. Has anybody rolled up their sleeves and said "Alright, BEAM VM's reign is over, I am m…

Well, I don't think you and I disagree on the premises, with the exception of you believing that I make some outrageous general claims -- which I did not.

> you make it seem as if the world of software development is in some constant battle for supremacy

I could have misinterpreted other people in the past -- very possible. But I also stopped caring about "battles for supremacy" a long time ago and at this point in life and career I simply use my experience and brain to go where my work is more productive, deterministic and fulfilling. To me immutability, share-nothing actors, strong vertical scalability (where a lot of PLs and runtimes do well, not only my favorites) and DX (like live prod REPLs and generally trivial observability) are those axii along which I thrive.

You and a few others seem to have emotional reactions to this which, I assure you, are very unnecessary. I ain't threatening neither your livelihood nor preferences.

Re: My thoughts after using Clojure for about a month

#174

Earlier quoted context omitted.

> That being said, has Akka started making full use of JVM's new green threads? Has Java itself started introducing immutability and STM / share-nothing as first-class citizens? Amazing how it doesn't even cross your mind that there are trade-offs to those choices. Green threads are awesome, but guess what, they come at a cost. Same for share-nothing semantics. > Has anybody rolled up their sleeves and said "Alright,…

Any good reason for your rude tone? If we are going to invoke the what crossed somebody's mind trope, I'd lead with that when talking to you -- did it cross your mind to speak calmly and not assume something "did not cross" somebody's mind? RE: your other similarly rude comment, I have not "appealed to authority" anywhere. I said that I have used multiple PLs / runtimes and made an informed choice... for me. I don't…

Your first comment assumed I was "speed-running to a conclusion and squinting too hard", and this was "similar to the weird childish name-calling". I think that is in the same area(or worse) than saying "Amazing how X didn't even cross your mind". And sorry, but invoking that you have experience with X, Y, Z and thus your opinion is informed after criticizing some technology IS an appeal to authority.

> If you don't intend to discuss out of position of curiosity

I'm not the one making sweeping statements on the superiority of one piece of technology. Reading your other response, I think you are the one who have little to no curiosity in understanding how you might be wrong.

Re: My thoughts after using Clojure for about a month

#175

Earlier quoted context omitted.

Any good reason for your rude tone? If we are going to invoke the what crossed somebody's mind trope, I'd lead with that when talking to you -- did it cross your mind to speak calmly and not assume something "did not cross" somebody's mind? RE: your other similarly rude comment, I have not "appealed to authority" anywhere. I said that I have used multiple PLs / runtimes and made an informed choice... for me. I don't…

Your first comment assumed I was "speed-running to a conclusion and squinting too hard", and this was "similar to the weird childish name-calling". I think that is in the same area(or worse) than saying "Amazing how X didn't even cross your mind". And sorry, but invoking that you have experience with X, Y, Z and thus your opinion is informed after criticizing some technology IS an appeal to authority. > If you don't…

I have been on the other side as well i.e. the Java / JVM. Hence: "informed".

Your negative assumptions are tiring. Sorry that you got offended by what you quoted (and I said) but I'll drop here.

Re: My thoughts after using Clojure for about a month

#176

Earlier quoted context omitted.

Web / API services during bursts. Or just when you _really_ don't want to scale horizontally. Elixir / Golang can do this very well. And they do. I have supervised, led and authored such projects that are in production to this day. Rust too but it's lower-level and you kind of have to hand-roll OTP which of course will always fail.

from experience, during bursts it's never actual web/api server that is bogged down, it's the downstream io bottlenecks. if your accepting layer is abstracted away and implemented correctly, there is very little performance difference between different concurrency approaches and all you're exposed to as developer is implementation of your handler functions.

Not the case; good abstractions are valuable, but the performance differences between runtimes are very real.

Take the example of some simple HTTPblob store service gets slammed with millions of requests when someone using the API does a backfill via some framework on their end that aggressively scales request volume up and out.

Something like, say, async Python/starlette with a coroutine per request is gonna perform slightly worse than Erlang, which in turn is gonna perform much worse than Go.

You're right that those differences are sometimes marginal when the latency of whatever IO the backend's doing dominates the equation. However, in my experience huge volume surges show issues with the runtime (the thing managing/launching multiplexed request handler routines) or the ecosystem (the backend IO libraries' ability to work with the runtime's IO multiplexing and make things like request coalescing easy or automatic) more often than you'd think.

It really takes surprisingly little volume to cripple a return-hello-world Phoenix app that indirects the "hello world" behind way too much middleware and message passing; it takes even less to kick over, say, a Gunicorn instance returning "hello world" at the bottom of the Django middleware stack. Golang with Gin, on the other hand, is surprisingly hard to cripple in the same way. And I say that as someone who likes Elixir and Python a lot more than I like Go!

Re: My thoughts after using Clojure for about a month

#177

Earlier quoted context omitted.

Clojure was explicitly designed to be dynamic. It’s a feature, not a bug. https://clojure.org/about/dynamic Until you get better at not making mistakes that the training wheels of a static type system “protect” you from, lean into the REPL as a means to build up small correct expressions into larger ones.

"Until you get better" is such an arrogant take. It's not just about skill. It's about maintainability, ease of refactor, and modeling invariants in your code in a way that they can be checked by the machine (the compiler) without every single developer having to maintain them in their head. Clojure even knows this is an issue and many people use `spec` to sort of retrofit static typing. Dynamic typing was, is and al…

[flagged]

Re: My thoughts after using Clojure for about a month

#178

Earlier quoted context omitted.

its not large employers only. you need a steady stream and large enough pool of candidates for any size of a business. if you limit yourself to a tiny almost invisible slice you are betting your business on finding candidates and relying on that small pool of talent. employees come and go and you would have to deal with this issue constantly.

Yes not only large employers, but it's easiest to think in those terms. And it's only true that "employees come and go" is a problem if 1. you assume turnover has to be high, and 2. you don't know where to find qualified people you need. Turnover will be high if you assume you need commodity devs and don't, for instance, invest in their skills (like teaching them minority languages if that's your thing). If I see a c…

We've had Python devs for over 10 years and a few of them decided to move to another country. To find someone to replace them and from even a smaller pool of talents is just setting ourselves up for failure.

Re: My thoughts after using Clojure for about a month

#179

Earlier quoted context omitted.

from experience, during bursts it's never actual web/api server that is bogged down, it's the downstream io bottlenecks. if your accepting layer is abstracted away and implemented correctly, there is very little performance difference between different concurrency approaches and all you're exposed to as developer is implementation of your handler functions.

Not the case; good abstractions are valuable, but the performance differences between runtimes are very real. Take the example of some simple HTTP blob store service gets slammed with millions of requests when someone using the API does a backfill via some framework on their end that aggressively scales request volume up and out. Something like, say, async Python/starlette with a coroutine per request is gonna perfor…

Thank you. As a guy who made a career out of Elixir (and begins to regret it recently but oh well) I agree that Elixir's throughput is not amazing. However, it can get very far and we should always optimize for the most common usages.

I've personally rewritten one hobby and one professional projects from Elixir to Golang and loved the result; as you said, extremely difficult to bring down a Golang service to its knees.

One clarification: Phoenix server behind Caddy/nginx fairs better btw. But, details. Your point stands.

I am yet to see a Rust web/API service I wrote to _ever_ buckle under pressure and just crash. It was either an application bug (like the famous Cloudflare's `.unwrap()` error from the last weeks/months) or the Linux OOM killer. Literally never crashed. But I did witness it brutally murder a MySQL cluster because it couldn't serve it fast enough. That was both fun and terrifying to watch on the dashboards.

Re: My thoughts after using Clojure for about a month

#180

Earlier quoted context omitted.

Not the case; good abstractions are valuable, but the performance differences between runtimes are very real. Take the example of some simple HTTP blob store service gets slammed with millions of requests when someone using the API does a backfill via some framework on their end that aggressively scales request volume up and out. Something like, say, async Python/starlette with a coroutine per request is gonna perfor…

Thank you. As a guy who made a career out of Elixir (and begins to regret it recently but oh well) I agree that Elixir's throughput is not amazing. However, it can get very far and we should always optimize for the most common usages. I've personally rewritten one hobby and one professional projects from Elixir to Golang and loved the result; as you said, extremely difficult to bring down a Golang service to its knee…

> I did witness it brutally murder a MySQL cluster because it couldn't serve it fast enough. That was both fun and terrifying to watch on the dashboards.

Haha yep. In my experience, everyone running CGI/process-per-request application servers is bullish on switching to a concurrent or cooperative runtime...until they realize they just removed the primary ratelimiter on downstream DB/service accesses.

The converse war stories are also amusing: people rewrite their whole app in a concurrent/asynchronous framework and nothing changes, because the DB driver is still farming out all queries to a tiny fixed-size threadpool of connections that was the bottleneck all along.

Post reply on HN