Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clojure at Netflix (2013) [slides]

speakerdeck.com

Post reply on HN
Print view View original

101–110 of 307 posts

Re: Clojure at Netflix (2013) [slides]

#101

Post by PopsiclePete »

projectileboy wrote: post #53

This is a bit tangential to the post, but I've seen a couple comments in this thread that amount to "Clojure is hard to read", but I think that's a really unfair comparison, because people are unwittingly comparing apples and oranges. It seems to me the reason people think Clojure is hard to read is that the language is so powerful and expressive, that when you're reading Clojure code, you're typically trying to unde…

I don't think it's hard to read, I think it's extremely difficult to analyze and almost impossible to debug.

Lazyness is one of those "cool" features, IMO, that have more cons than pros. Yes, certain algorithms look more "elegant", but it's also harder to reason about/debug.

Most of the time, you can ignore the fact that this awesome language runs on the JVM, until you see a barely-comprehensible stack trace containing nothing but `org.clojure.Iseq` something-or-other and no user code in sight.

Looking at how a "real" LISP displays errors and allows you to debug it and comparing it to Clojure makes me feel really sad inside.

Sadly, these are the reasons I tend to favor Go for work-related projects. It's less "fun" to write than a LISP, but makes pinpointing and fixing errors a breeze in comparison. And when my job is on the line, that's the killer language feature for me.

Re: Clojure at Netflix (2013) [slides]

#102

Post by zeveb »

vbuwivbiu wrote: post #77

amazed to see there are people who find myObject.myMethod(x, y, z); easier to read than (my-function x y z) - that's 2 delimiters in Clojure vs 6 in your C-style language

In fairness, it'd actually be:

    (my-method my-object x y z)
But yes, the Lisp version is so much clearer to me. I just don't get why it's not more popular.

Re: Clojure at Netflix (2013) [slides]

#103

Post by simongray »

rubber_duck wrote: post #91
projectileboy wrote: post #53

This is a bit tangential to the post, but I've seen a couple comments in this thread that amount to "Clojure is hard to read", but I think that's a really unfair comparison, because people are unwittingly comparing apples and oranges. It seems to me the reason people think Clojure is hard to read is that the language is so powerful and expressive, that when you're reading Clojure code, you're typically trying to unde…

Not my experience - for eg. I remember reading trough implementation of a clojure standard lib core.async (this was years ago) https://github.com/clojure/core.async/blob/master/src/main/c... It took me a long time to figure out what's going on in that code - I remember realizing how it would have been much simpler to read in a popular statically typed language like C#, types are a lot like compiler verified documenta…

That source file you're referencing is a fairly low-level library doing lots of interop with Java. That's not really what most Clojure code looks like.

And furthermore: it's just a library, but this library is adding the concept of go blocks (as seen in Go) to the Clojure language. Are you even able to add a new control structure like go blocks to the C# langauge by including a library?

Re: Clojure at Netflix (2013) [slides]

#104

Post by diab0lic »

shadowmint wrote: post #93
diab0lic wrote: post #63

Earlier quoted context omitted.

I am very glad you asked! I wrote and deployed (to production) some Clojure code at Netflix just yesterday. Among other things at Netflix the Mantis Query Language (MQL an SQL for streaming data) which ferries around approximately 2 trillion events every day for operational analysis (SPS alerting, quality of experience metrics, debugging production, etc) is written entirely in Clojure. This runs in nearly every criti…

Thats an interesting point that doesn’t answer any of the questions the parent asked. :) Perhaps you can answer this simpler one: Netflix has been using clojure for a long time now; has that been a positive experience broadly speaking, that means clojure is still being used for new projects, or not? Having a large successful project in clojure is lovely, but much of the community’s concern around it is that its hard…

Hey, sorry I saw that and typed a quick response just as I woke up. I'm not usually at a computer so early in the day. I'll address these now that I'm in front of a machine. :)

> Did they keep writing more Clojure?

Yes but it has never been the primary language at Netflix.

> How much more did they rewrite from Java to Clojure?

Very little, if any was rewritten from Java.

> If so, how much of their code is now in Clojure compared to Java?

A very small amount given that it isn't the primary language and Clojure code bases tend to be much smaller than Java.

> Do they use Clojure rather than Java for new code?

This is a personal choice each engineer makes when they write new code. Those who like Clojure might reach for it more often. Clojure is also easy to use within the environment at Netflix since everything was JVM based.

> What other languages do they use? Python? Erlang? Rust?

NodeJS and Javascript, Python, Ruby all have a seat at the table but the majority of back-end code at Netflix is on the JVM, the majority of that is Java.

> Among the things that seemed great with Clojure in 2013, did they find that some of these were not so great after all once the codebase grew? Any other problems?

I've always found larger Clojure code bases to be a bit unwieldy. Fortunately you can usually continue to abstract and keep the size small. If you choose your abstractions carefully you can get a lot of mileage out of this.

I've found the lack of static typing to be a bit of a pain at times especially when refactoring. My safety net for this in the project mentioned in the GP post is to have comprehensive unit tests. If I were to initiate this project today I'd likely explore using Spec to make type assertions.

Re: Clojure at Netflix (2013) [slides]

#105

Post by mikeflynn »

We have used Clojure in our data processing and API code for five years now at Studio71, and it's been a huge success for us. We have much smaller dev teams than our competitors so Clojure has helped us keep pace with a smaller code base and fast data processing, but it has been a challenge to hire new developers when we need one, but with patience it has always worked out.

Re: Clojure at Netflix (2013) [slides]

#106

Post by keymone »

blandflakes wrote: post #68
m12k wrote: post #55

I'd like to learn a modern production-ready functional language (I have some academic experience with SML), since they seem like a good way to grow as a programmer. Main contenders so far are Haskell, Scala and Clojure (Reason might go on the list soon too) - but the fact that Clojure is dynamically typed is a bit of a turn off for me. My experience with dynamic vs static typing is that as a system grows in size and…

There are a lot of people who claim they're more productive in Clojure, in spite of or because of its dynamic typing. I'll just throw my anecdatum out here: I found that dynamic typing was a major pain even on my own personal projects. I also never found REPL-driven development to mesh well with my workflow. In Scala, for example, entire classes of error that I just shouldn't be able to make don't exist. Maybe I'm ju…

> I also never found REPL-driven development to mesh well with my workflow.

> In Scala, for example, entire classes of error that I just shouldn't be able to make don't exist.

guess what programming practice allows to almost entirely avoid those classes of error?

Re: Clojure at Netflix (2013) [slides]

#107

Post by dnautics »

jb3689 wrote: post #95
udfalkso wrote: post #88

Earlier quoted context omitted.

Another option worth considering is Elixir. It's also dynamically typed, but the pattern matching helps mitigate dynamic type issues if used correctly. It's a pleasure to work with.

I love Elixir but it's worth noting that I'd be hard-pressed to recommend it for every project because of the performance and base OS integration characteristics of the BEAM. The BEAM is a truly amazing piece of software and I do think that Erlang and Elixir are the true microservices dream we all wanted, but the BEAM uses preemptive scheduling and doing things like spinning up a new OS process can be a lot more work…

What are the performance issues? People (e.g. Instagram) deploy scaled apps in Python, which is far less performant.

Bleacher report obsesses about information push latency, and they use an all elixir stack that deploys to at least a million users.

Not to mention WhatsApp.

Re: Clojure at Netflix (2013) [slides]

#108

Post by puredanger »

kamaal wrote: post #26
systems wrote: post #11

Earlier quoted context omitted.

to talk about something new, you need to have something new to talk about i dont think that scala or clojure have lost momentum i think both clojure and scala, stopped adding new features that are worthy of blogging about you cant just add feature, just to keep people talking about you clojure, is what it is, a nice language, a very smart leader (rick hickey), and a nice vibrant community (check #clojure on twitter)…

>>i dont think that scala or clojure have lost momentum i think both clojure and scala, stopped adding new features that are worthy of blogging about More like Clojure is in Cathedral mode of open source development. Most of the development is in private discussions, implementation and prototyping at Cognitect, and once they have something to share, they release it to the world. That's more a communication and market…

https://clojureverse.org/t/roadmap-to-1-10/1581/8?u=alexmill... is a pretty good overview of where Clojure has been going and will continue to go.

Re: Clojure at Netflix (2013) [slides]

#109

Post by diab0lic »

alextheparrot wrote: post #98
diab0lic wrote: post #63

Earlier quoted context omitted.

I am very glad you asked! I wrote and deployed (to production) some Clojure code at Netflix just yesterday. Among other things at Netflix the Mantis Query Language (MQL an SQL for streaming data) which ferries around approximately 2 trillion events every day for operational analysis (SPS alerting, quality of experience metrics, debugging production, etc) is written entirely in Clojure. This runs in nearly every criti…

Do you know offhand how MQL compares to the Apache Calcite [0] extension of SQL that some big data platforms are using for streaming SQL? [0] https://calcite.apache.org/docs/stream.html

Hey Alex,

I explored potentially using Calcite when we initiated this project. The syntax is very similar because both are an SQL dialect. Some differences are that MQL largely targets unstructured data (there is no schema for the streams it operates on - they're just streams of JSON blobs).

In addition to that one of the goals of MQL was to have different compiler backends which allows different call sites to operate on different levels. For example the code that runs in our API/proxy/other large services will look at an entire query and only evaluate the WHERE / SAMPLE clauses expecting something down the stream to complete the query. Conversely the client side is a full SQL -> RxJava implementation for the data stream. I can explain more of how this works if there is interest -- it allows us to only egress data that is actively used in queries so devs can log every request and only pay the cost of those for which they are querying.)

Doing our own implementation also allows us more customization, and to compile to a NodeJS backend as well which is a critical ingestion point for our operational data. We support everything on the linked page except hopping windows, subqueries, and DML (MQL is query only, we assume no structure which was true of the streams before we ever wrote a query language for it). Of course we have to implement a lot more of this ourselves which is pretty in line with Netflix's Freedom and Responsibility. We had the freedom to implement our own query language but have the responsibility to maintain it (Calcite would have been sacrificing some freedom to avoid responsibility.)

Re: Clojure at Netflix (2013) [slides]

#110

Post by brogrammer5 »

shadowmint wrote: post #93
diab0lic wrote: post #63

Earlier quoted context omitted.

I am very glad you asked! I wrote and deployed (to production) some Clojure code at Netflix just yesterday. Among other things at Netflix the Mantis Query Language (MQL an SQL for streaming data) which ferries around approximately 2 trillion events every day for operational analysis (SPS alerting, quality of experience metrics, debugging production, etc) is written entirely in Clojure. This runs in nearly every criti…

Thats an interesting point that doesn’t answer any of the questions the parent asked. :) Perhaps you can answer this simpler one: Netflix has been using clojure for a long time now; has that been a positive experience broadly speaking, that means clojure is still being used for new projects, or not? Having a large successful project in clojure is lovely, but much of the community’s concern around it is that its hard…

What's the condescending tone for?
Post reply on HN