Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why Isn't Elixir More Popular?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 91 posts

Re: Ask HN: Why Isn't Elixir More Popular?

#81

Post by elcritch »

madamelic wrote: post #25
elcritch wrote: post #24

Earlier quoted context omitted.

> From metrics, computing AWS signatures takes up an absurdly large amount of CPU time. The actual processing of events is quite minimal and honestly well-architected, a lot of stuff is loaded into memory rather than read from disk. There's syncing that happens fairly frequently from the internet which refreshes the cache. Oh, sounds nice! Caching in Elixir really is nice. Okay, that makes sense. Elixir isn't fast at…

Thank you! You gave me a great term that I can jump off from (NIF). I'll have to rig up Observer. I've been using recon because I was being lazy overall.

You might also look into Elixir's broadway as provides back-pressure [1].

1: https://hexdocs.pm/broadway/Broadway.html

> I also looked into the wake-up lengths of threads. I am starting to get my head around "dirty schedulers" but I am not entirely sure how to affect those or how I can besides it doing it forever me.

Note that dirty schedulers really only affect NIFs which run longer than what the BEAM schedulers expect. I mentioned that in regards the possibility that the AWS sigs are taking longer than they should, then they'd cause havoc on the scheduler.

Re: Ask HN: Why Isn't Elixir More Popular?

#82

Post by bc58 »

DoesntMatter22 wrote: post #54
josevalim wrote: post #48

Earlier quoted context omitted.

> The database pool needs to be at 512 to 1024? Yet Rails seems to get away with far less than that and still have better performance.[2] Rails concurrency model is per process . So you will have `n_workers` pools and the amount of connections will be `n_workers * 3` based on the code you shared. The production machine has 28 total threads [1], which means Rails is starting 35 workers, according to their auto tuning…

Nice to see elixir put up decent benchmarks for once. It looks like it's about 80 percent faster. Still a far cry from "10 to 20x reduction in size and cost".

10 to 20 improvement here.

I wrote, while learning, a monitoring system in Elixir. I put everything in one "Phoenix" app. What does it do?

* Query about 60 oracle databases in parallel every five seconds.

* Query a number of KDB databases every five seconds.

* SSH to dozens of machines every ten seconds

* Ingest statistics from icinga.

* Pull ticket updates from Jira every couple of seconds.

* Provide facilities for connecting in to screen on about 100+ remote machines, and interacting with their interfaces.

* API access to internal and external commands/functions.

* more.

It does much more than this, while serving live-view sessions to users, on a dual core machine (CPU: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz x 2), while averaging about 8% CPU for everything it does. The SSH component replaces a python version that does the same thing. The Python instance, by itself, pushes a 6 core box (same xeon as above but with 6 cores, rather than 2) to 100% CPU, and spaws close to 3,000 OS tasks.

Many of the other checks that are running in that 8% CPU Elixir process replace python, node, and java checks, each of which required more ram, demanded more CPU, and were spread across eight to ten dedicated machines per check. We have been retiring machines, and reducing CPU load, so I'd say we do see the 0 to 20 reduction.

Now, my solution might have been written better, but that was what I came up with while learning Elixir.

Re: Ask HN: Why Isn't Elixir More Popular?

#83

Post by DoesntMatter22 »

bc58 wrote: post #82
DoesntMatter22 wrote: post #54

Earlier quoted context omitted.

Nice to see elixir put up decent benchmarks for once. It looks like it's about 80 percent faster. Still a far cry from "10 to 20x reduction in size and cost".

10 to 20 improvement here. I wrote, while learning, a monitoring system in Elixir. I put everything in one "Phoenix" app. What does it do? * Query about 60 oracle databases in parallel every five seconds. * Query a number of KDB databases every five seconds. * SSH to dozens of machines every ten seconds * Ingest statistics from icinga. * Pull ticket updates from Jira every couple of seconds. * Provide facilities for…

That seems like a good fit for Elixir tbh

Re: Ask HN: Why Isn't Elixir More Popular?

#84

Post by wkyleg »

kugelblitz wrote: post #75

I use Symfony (PHP based framework) and it works fine. I've been able to get into Django (Python), Laravel (PHP), Java (Spring) and even Grails (Groovy) because they either had similar concepts or even similar syntax (I mostly do web development, so this is a very biased take). Being a freelancer, I need to focus on what's marketable. Sure, Elixir will get me into a niche, but I will have way less projects to choose…

Do you really find Java backends to be that great?

I don't think they're intrinsically bad, and Kotlin or Groovy can be nice, but I've always found the community and lack of open source tools to be wanting.

I've found myself to be much more productive in JavaScript/TypeScript or Python, in part because of the languages, but also because the open source libraries have been way better

Re: Ask HN: Why Isn't Elixir More Popular?

#85

Post by josevalim »

gregors wrote: post #59
josevalim wrote: post #32

Earlier quoted context omitted.

> And they say "Oh we didn't care to put much effort into it", Yet Jose Valim himself tried to work on it and didn't fix it. This is inaccurate. I have started looking again into solutions only last week [1]. My suspicion was always the database pool size was too small but, when I tried to contribute 4+ years ago, fine tuning was hard because it took too long, so I didn't pursue it further [2]. My discontent with the…

The techempower repo is literally a gift to the world. I've seen and learned many ways of optimizing code ...almost none of which should ever be used in production or would ever make it past a PR at most companies. Still, it's amazing in the amount of non-realistic approaches used. It really is a work of art.

You are right. There are many positive things about TechEmpower and keeping it open to all and running takes an incredible amount of effort, which is definitely appreciated and I failed to recognize. That was unfair of me.

Although I still strongly disagree with the initial observation that they are "real world". Most applications in the real world receive heterogeneous traffic, from a varying and unknown number of clients, with logging, potential contention on resources, and other aspects that are not taken into consideration.

Re: Ask HN: Why Isn't Elixir More Popular?

#86

Post by toptrumps »

ipnon wrote: post #33

No manager is going to lose their job by hiring 10 Python and JavaScript engineers to accomplish what could be done with 1 Elixir engineer. And that’s how most time and effort is allocated in our tech labor market economy. It’s by working backwards from “I want to still have a job in tech next year.” The number of people who are calculating which language to write in by thinking “how can I write a fault-tolerant dist…

How is Elixir giving 10-1. That would be insanely good? Is it because it is replacing the devops and SRE work somehow? Or you can be super productive? I am skeptical. I imagine the 10-1 ratio can happen for some projects but not all. Right now the FAANGs would get their fangs into this for some downsizing!

Re: Ask HN: Why Isn't Elixir More Popular?

#87

Post by benzible »

sbaiabBzic wrote: post #11
sergiotapia wrote: post #3

It's a wonderful language. Unfortunately the only places I've been able to use it and get paid were places where I ultimately had the decision making ability of the tech stack. Why isn't it used? It's niche and betting on such a small community is risky for the majority of companies. Why use Elixir when you could hire 10 engineers to pump out javascript. That's the mentality of most. Hiring for elixir was great, it s…

> Hiring for elixir was great, it self selected people who wrote code as their craft I worked at a company that hired like this. On the whole it was good, but it wasn’t a panacea. A surprising number ended up being academic types that were extremely smart but could never actually finish anything. Amazing guys to talk to at lunch though!

> A surprising number ended up being academic types that were extremely smart but could never actually finish anything.

As a startup CTO currently building with Elixir, I can say that it's absolutely possible to hire people who know how to deliver. I completely agree with the parent comment re: self-selecting people who write code as craft. I've put together an excellent team of developers who deeply care about what they do and are focused on shipping.

Re: Ask HN: Why Isn't Elixir More Popular?

#88

Post by asa400 »

madamelic wrote: post #21
elcritch wrote: post #19

Earlier quoted context omitted.

Interesting, Elixir should scale far more than that. Are you doing a lot of non-io processing or computations? I run Elixir on raspberry pi 4s doing IoT and they easily handle say generatings graphs with hundred of thousands data points. One possibility is you're using a single process instead of parallelizing things. For example, you may want to use one process per event, etc. Though if the hardware is very underpow…

> Are you doing a lot of non-io processing or computations? Unfortunately. From metrics, computing AWS signatures takes up an absurdly large amount of CPU time. The actual processing of events is quite minimal and honestly well-architected, a lot of stuff is loaded into memory rather than read from disk. There's syncing that happens fairly frequently from the internet which refreshes the cache. The big problem is eac…

I don't know the specifics of your app so I don't feel commenting in more than generalities, but generally speaking, if you are doing work in native code, and if that native code work is CPU-bound (roughly, more than a millisecond of CPU time) you should try to do it in a dirty scheduler. If you don't, what will happen is that that CPU-bound code will interfere with the "regular" BEAM schedulers, meaning it will start to interfere with how the BEAM schedules all of the other work in your app, from regular function calls to IO to job queuing to serving requests, and whatever else.

I'm also suspicious of the `+scl true` setting as maybe being a bit of a red herring. I've been using BEAM off and on for 10 years both professionally and as a hobbyist and I've never used this myself nor seen anyone else ever need to use it. I'm sure there are circumstances where someone, somewhere has used this, but in a given Elixir app it is extremely likely that there is lower-hanging fruit than messing with scheduler flags.

In terms of queuing, are you using Oban or Broadway or something hand-built? It's common for folks to DIY this kind DAG/queuing stuff when 99.9% of the time using something like Oban or Broadway would be better than hand-rolling it.

Re: Ask HN: Why Isn't Elixir More Popular?

#89

Post by mtndew4brkfst »

innocentoldguy wrote: post #67
binary132 wrote: post #45

Most people just want to get something done and reach for the tools that will most straightforwardly and simply get them to their destination. Niche stuff doesn’t have the market fit because you’ll always have a hard time finding people to work on your product that don’t cost twice as much as everyone else, unless it’s 10x faster gtm / 10x cheaper. Seems obvious enough. Personally? I like good typesystems and very sh…

In my experience, Elixir is one of the easiest languages to pick up and get stuff done quickly. It has excellent tooling and deployments ARE 10X cheaper (if not more). I've never converted a project to Elixir and not had the associated costs decrease dramatically. Also, I don't find typing to be an issue in Elixir due to the way the language works (pattern-matching, guard clauses, immutable data, etc.). Maybe I just…

I've never converted a project to Elixir and not had the associated costs decrease dramatically.

It's useful to remind that any full rewrite without notable changes in feature scope will probably wind up performing better than the original organically developed solution, even if you never change tech stacks. That's just the nature of going in with a total picture of the problem space and knowledge of the warts already encountered.

Re: Ask HN: Why Isn't Elixir More Popular?

#90

Post by mtndew4brkfst »

victorbjorklund wrote: post #51
viccis wrote: post #27

Earlier quoted context omitted.

>How large is the pool of available candidates for this language? A recruiting risk. I worked at a company with a massive Erlang codebase. Really nasty, not really following good OTP practices, etc. HUGE hiring problem and it took 6-12 months to spin up new devs to the point that they could really have ownership over things. And this is not a system that even remotely needed this. Elixir might seem better until you w…

You def dont need to learn Erlang to code in Elixir (other than maybe the very basics to interact with some Erlang libraries like ETS).

As José answered more gracefully, you don't need to learn the syntax but you do need to learn the semantics, the performance gotchas, and for most projects I'd say you'll probably need to consume Erlang documentation which is certainly a learned skill.
Post reply on HN