Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

471–480 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#471

Earlier quoted context omitted.

Easier to pick up, harder to use. A spade is easier to pick up than a digger, but try digging foundations with a spade.

Is it though? I've been using dynamic langs my entire career, and static ones too! I feel like statically typed langs, (I'm looking at you Java), are a bit more stubborn to work with, especially around API design, prototyping, greenfield stuff. I do like langs like Haskell and Standard ML where they are statically typed, the the type system is mathematically sounds, and the types are inferred. I want my type system t…

Static typing has huge advantages over dynamic:

* Reading and understanding code is much easier because the types are written down. You spend much less time figuring out what variables can contain.

* Navigating code is much faster because tools like go-to-definition, code completion and find-all-references work reliably

* Refactoring code is a lot easier - or in large projects actually tractable. In large dynamically typed projects something as simple as renaming a variable can be an impossible task.

* Obviously the one people talk about most is catching bugs. The degree it does that depends on how strong the typing is (e.g. Rust will catch many more bugs than Java). But they will all catch the embarrassing things that dynamic languages can't like typos and missing arguments.

If you've only ever written short greenfield projects you might not appreciate some of these benefits as much as you should because you wrote all the code yourself so all the details are still in your head. It's a bit like saying seatbelts are an unnecessary pain because you haven't ever been in a crash.

> types are inferred

Some local type inference is good, but Haskell / ML style global type inference is kind of the worse of both worlds. You have to satisfy the type checker, which is harder because global solver errors are always harder, and you don't get the documentation benefits of static types because the inferred type is frequently a generic type.

Rust went with local type inference only for very good reasons.

Re: Elixir saves Pinterest $2M a year in server costs

#472
post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

>Would be curious to know how they tried to optimise the Java stack.

Fairly safe to day not at all.

The JVM is extremely fast, very efficient and very scalable (you can write java code that scales linearly with available cores). If performance or scalability is a metric to care about, it is nearly impossible to outscore java. You can, with very skilled C/C++ developers, but it's going to be difficult to find those people and it'll be a lot of work. If you need extreme performance on a reasonable budget, you can't do better than java. I know java isn't hip anymore, so this is not a popular truth, but it is.

Re: Elixir saves Pinterest $2M a year in server costs

#473

Earlier quoted context omitted.

Is it though? I've been using dynamic langs my entire career, and static ones too! I feel like statically typed langs, (I'm looking at you Java), are a bit more stubborn to work with, especially around API design, prototyping, greenfield stuff. I do like langs like Haskell and Standard ML where they are statically typed, the the type system is mathematically sounds, and the types are inferred. I want my type system t…

Static typing has huge advantages over dynamic: * Reading and understanding code is much easier because the types are written down. You spend much less time figuring out what variables can contain. * Navigating code is much faster because tools like go-to-definition, code completion and find-all-references work reliably * Refactoring code is a lot easier - or in large projects actually tractable. In large dynamically…

I want to point out goto definitions do work for a number of dynamic langs.

This is all preference. Ive also worked on ruby on rails projects that are 15 years old with 500K LoC

i admit changing stuff can be pretty sketchy if your project doesn't have great test coverage. but that all ultimately comes down to the culture of the programmers working on that code base.

I think things like dynamic/ vs weak typing or functional vs imperative have a much greater impact on code quality/ease of coding than that of static vs dynamic.

I personally think programming in Java, c# is painful, but a lang like Crystal or Standard ML to be very pleasant. and vice versa, i think vanilla javascript is painful for all the reasons you mentioned but langs like Ruby, Erlang, or Elixir to be very pleasant.

https://edward-huang.com/programming/software-development/20...

“Some languages have a greater association with defects than others, although the effect is small.” Languages associated with fewer bugs were TypeScript , Clojure , Haskell , Ruby , and Scala ; while C , C++ , Objective-C , JavaScript , PHP , and Python were associated with more bugs.

Re: Elixir saves Pinterest $2M a year in server costs

#474

Earlier quoted context omitted.

When you move from one minor rev of python to the next, some language feature changes (either syntax or semantics or features no longer work.) For instance... if you use async io in 2.x, the debugger stops working. Between 2.3, 2.5 and 2.7, the syntax of package variable scoping changed and then the semantics changed from package to class variables. If you used a feature like package variables in your code in 2.3, th…

> When you move from one minor rev of python to the next, some language feature changes Even if there was a breaking change affecting your project every minor version, to have the cadence of backward-compatibility induced changes you siggest you’d have to be switching Python versions forward about four times as fast as they are released, which, if you started with the oldest in support version at the beginning of the…

I bring up 2.3 to 2.5 to 2.7 as an example, not as a recommendation.

Re: Elixir saves Pinterest $2M a year in server costs

#475
post #456

Earlier quoted context omitted.

What I am objecting to is you asserting my experience didn't happen.

You’re the only one doing that. Nobody here has questioned that you had an unpleasant project to work on - we’re only arguing that it’s not representative of the experience now (your initial hyperbole) or even 15 years ago when the 2.3-2.5 transition would have happened. Many of us have worked on larger codebases in that timeframe with very different experiences.

You had >70kloc of python code that required no modification between 2.3, 2.5 and 2.7. I don't find that believable.

Re: Elixir saves Pinterest $2M a year in server costs

#476

Earlier quoted context omitted.

Static typing has huge advantages over dynamic: * Reading and understanding code is much easier because the types are written down. You spend much less time figuring out what variables can contain. * Navigating code is much faster because tools like go-to-definition, code completion and find-all-references work reliably * Refactoring code is a lot easier - or in large projects actually tractable. In large dynamically…

I want to point out goto definitions do work for a number of dynamic langs. This is all preference. Ive also worked on ruby on rails projects that are 15 years old with 500K LoC i admit changing stuff can be pretty sketchy if your project doesn't have great test coverage. but that all ultimately comes down to the culture of the programmers working on that code base. I think things like dynamic/ vs weak typing or func…

> I want to point out goto definitions do work for a number of dynamic langs.

Not reliably. It can work in a small subset of situations. For statically typed languages it always works.

> “Some languages have a greater association with defects than others, although the effect is small.” Languages associated with fewer bugs were TypeScript , Clojure , Haskell , Ruby , and Scala ; while C , C++ , Objective-C , JavaScript , PHP , and Python were associated with more bugs.

This is mixing up too many things. For example C++ has a relatively decent static type system, but obviously it's going to have way more defects than memory safe languages.

This is a much much better study:

https://earlbarr.com/publications/typestudy.pdf

Re: Elixir saves Pinterest $2M a year in server costs

#477

Earlier quoted context omitted.

You can do that easily in modern Java--even for older JVMs, tools like Netty and later Vertx have been around forever. Or in Node, even more easily. Elixir/BEAM do have some benefits that are worth considering for many projects. But they absolutely are not special in this regard, and that's the junior-developer trap about which the person to whom you replied was referring.

Java, kind of with Akka or similar, although even with that one always be aware of blocking. Loom should help. Node: not really unless something dramatic has changed. Using 32 cores is going to require 32 separate node (OS level) processes, and your on your own for providing communication between them, plus callbacks aren't near as intuitive as the BEAM process model (think green threads)

Re: Node - most waiting in most web applications is IO-driven, but yes you do want to run under a process controller.

Re: Elixir saves Pinterest $2M a year in server costs

#478

Earlier quoted context omitted.

I want to point out goto definitions do work for a number of dynamic langs. This is all preference. Ive also worked on ruby on rails projects that are 15 years old with 500K LoC i admit changing stuff can be pretty sketchy if your project doesn't have great test coverage. but that all ultimately comes down to the culture of the programmers working on that code base. I think things like dynamic/ vs weak typing or func…

> I want to point out goto definitions do work for a number of dynamic langs. Not reliably. It can work in a small subset of situations. For statically typed languages it always works. > “Some languages have a greater association with defects than others, although the effect is small.” Languages associated with fewer bugs were TypeScript , Clojure , Haskell , Ruby , and Scala ; while C , C++ , Objective-C , JavaScrip…

Totally hear you on the static typing benefits, but let's zoom out a bit. As I mentioned before, I've got experience with both dynamic and static languages, and I think we're missing some nuance here. Specifically, I wanna bring functional vs. imperative and strong vs. weak typing into the mix.

JavaScript's weak typing does it no favors, agreed. But that's not a universal dynamic language issue. Ruby, for example, doesn't have those type coercion headaches.

Now, about Haskell and Standard ML—these guys offer a different flavor of static typing. It's not the Java-esque rigidity; it's more flexible and, dare I say, enjoyable.

On the tooling front, I've seen dynamic languages with solid IDE support and go-to-definition features. It's not a static-only perk; it's about the ecosystem's maturity.

That study is a neat data point, but it's not the whole picture. We should consider multiple variables like paradigms and type strengths, not just the static vs. dynamic lens.

I do also want to say in the defense of Elixir (and Erlang), with the advent of the dialyzer lib, it's a gradually typed language, and soon to be (fingers crossed,) a lang with a pretty unique type system. It will be both dynamic but have the same guarantees as a statically typed lang.

https://elixir-lang.org/blog/2023/06/22/type-system-updates-...

(Small tutorial on Set theoretical types, most of this is above my understanding though) https://pnwamk.github.io/sst-tutorial/

Re: Elixir saves Pinterest $2M a year in server costs

#479
post #421
post #151

Earlier quoted context omitted.

> ... with 1/50th the effort of any other language. To junior developers reading this, such statements are never true.

To senior developers reading this, such statements are never true ;)

To non-developers reading this, I have no way of knowing whether such statements are ever true, but I am willing to believe they are not!

Re: Elixir saves Pinterest $2M a year in server costs

#480

Earlier quoted context omitted.

Java, kind of with Akka or similar, although even with that one always be aware of blocking. Loom should help. Node: not really unless something dramatic has changed. Using 32 cores is going to require 32 separate node (OS level) processes, and your on your own for providing communication between them, plus callbacks aren't near as intuitive as the BEAM process model (think green threads)

Re: Node - most waiting in most web applications is IO-driven, but yes you do want to run under a process controller.

I mean sure, but given the topic at hand is getting performance out of your 32-core server, I'm not sure that's a super relevant observation. "I've been given way more hardware than I need" is an entirely different problem I think most of us would be happy to tackle.
Post reply on HN