Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clojure at Netflix (2013) [slides]

speakerdeck.com

Post reply on HN
Print view View original

221–230 of 307 posts

Re: Clojure at Netflix (2013) [slides]

#221

Post by KingMob »

jrochkind1 wrote: post #174
KingMob wrote: post #61

Earlier quoted context omitted.

I use Clojure in the large, and the lack of static typing isn't something I miss. FWIW, spec allows you make similar guarantees if you use it (technically more, since you can express more than HM type systems). I think the static/dynamic divide speaks to deep divisions in programmer personalities, but if you're open to suggestion, I would put it like this: As Haskell/OCaML are to static langs like Java, Clojure is to…

I believe you (I'm a ruby programmer, so you know I'm not strict about typing), but can you clarify -- `spec` is runtime, not statically typed, true?

Yes, spec is a run-time tool. It's capable of multiple things, but it's mostly used at dev-time to validate fn inputs and auto-generate spec-valid data for testing. It's usually disabled on production except at the edges of the system, where it frequently validates external input (e.g., making sure a JSON input conforms to an expected spec).

Clojure's Schema is similar to spec, if a little different. There's also Typed Clojure, which allows you to introduce gradual typing to a Clojure code base and get compile-time static typing (though I believe it's still alpha/beta quality).

Re: Clojure at Netflix (2013) [slides]

#222

Post by willtim »

akmiller wrote: post #216
lgas wrote: post #208

Earlier quoted context omitted.

> To be fair, I wasn't comparing the two. Well, the statement you were disagreeing with from the post you responded to was: > You can't refactor Clojure without fear like in Haskell. You go on to suggest you can achieve a similar experience in Clojure by "depending on how you write your code". This simply hasn't been my experience. Just "writing your code the right way" solves almost every problem that arises in prog…

Exactly, and I was responding to the fact that I never fear refactoring my Clojure code. Your examples of the type safety can all be mimicked with spec in Clojure. Granted spec is opt-in (but I'm guessing so is some of the more detailed type safety attributes you are talking about like NonEmpty). Anyhow, to each their own and one persons experience isn't likely to be the same as the others so I'd encourage everyone t…

Explicit runtime checks that must be manually added are not a substitute for static typing! You should follow your own advice and try out a statically typed language with good inference!

Every tool has a sweet spot and large code bases and maintenance is well outside the sweet spot of any dynamic language.

Re: Clojure at Netflix (2013) [slides]

#223

Post by willtim »

akmiller wrote: post #217
matt-noonan wrote: post #198

Earlier quoted context omitted.

In Haskell designs, developers tend to be quite careful about setting things up so that verifying types does indicate valid data. Coming from other languages (C++, Common Lisp, and Python in my case), it can be a little surprising just how often and how easily you can make this happen.

You can do similar things with spec for Clojure. As I stated in another response that is opt-in obviously so it requires more discipline perhaps but it is definitely available.

Null checks are also opt-in, that's why Tony Hoare calls null his "Billion Dollar Mistake". Explicit runtime checks are in no way "similar" to static typing! Static typing is a machine-checked proof of correctness for a certain class of properties.

Re: Clojure at Netflix (2013) [slides]

#224

Post by jcadam »

jimbokun wrote: post #159
TheMagicHorsey wrote: post #43

I love Lisp, so I thought Clojure would be a great productivity booster. And it is, if you are a one-man or one-woman shop. But try to build a team around a Clojure project and it's another story. The language basically begs you to make "magic" happen with domain-specific constructs. That might make you feel powerful as a programmer, but it's also a nightmare for new team members to get up to speed on. And then, as o…

"Most programmers are turned off by the syntax and/or immutability." Programmers who don't see the value of immutability or can't understand it are not programmers you really want to work with.

> Programmers who don't see the value of immutability or can't understand it are not programmers you really want to work with.

I'm not disagreeing with you, but since you just described easily 90%+ of all working programmers, it's kind of unavoidable.

One place I worked, one of the lead engineers knew I was into functional programming and asked me if I thought it would be a good idea to introduce Scala or Clojure into our pure Java code base. I said "No. The majority of the programmers here couldn't make the adjustment."

Put another way, if I could assemble a hand-picked team of all of the engineers I would rate as "great" who I've worked with over my 12+ year career thus far -- I'd have a team of about 7 people.

Re: Clojure at Netflix (2013) [slides]

#225

Post by matt-noonan »

akmiller wrote: post #217
matt-noonan wrote: post #198

Earlier quoted context omitted.

In Haskell designs, developers tend to be quite careful about setting things up so that verifying types does indicate valid data. Coming from other languages (C++, Common Lisp, and Python in my case), it can be a little surprising just how often and how easily you can make this happen.

You can do similar things with spec for Clojure. As I stated in another response that is opt-in obviously so it requires more discipline perhaps but it is definitely available.

I'd rather have the compiler ensure that my discipline (and my team's) never slips, and to get guarantees about all possible executions instead of just those executions that have been tested.

For these reasons and others, I don't personally find run-time checks to be an adequate replacement for compile-time checks. But there is no really convincing research on the subject, and I certainly don't begrudge your preference here.

Re: Clojure at Netflix (2013) [slides]

#226

Post by blain_the_train »

shadowmint wrote: post #93
diab0lic wrote: post #63

Earlier quoted context omitted.

I am very glad you asked! I wrote and deployed (to production) some Clojure code at Netflix just yesterday. Among other things at Netflix the Mantis Query Language (MQL an SQL for streaming data) which ferries around approximately 2 trillion events every day for operational analysis (SPS alerting, quality of experience metrics, debugging production, etc) is written entirely in Clojure. This runs in nearly every criti…

Thats an interesting point that doesn’t answer any of the questions the parent asked. :) Perhaps you can answer this simpler one: Netflix has been using clojure for a long time now; has that been a positive experience broadly speaking, that means clojure is still being used for new projects, or not? Having a large successful project in clojure is lovely, but much of the community’s concern around it is that its hard…

> Having a large successful project in clojure is lovely, but much of the community’s concern around it is that its hard to maintain, and falling in popularity, broadly speaking.

What community? The larger software community or the clojure community. I personally see no logical reason why a "large" clojure project would be harder to maintain then say a java, python, ruby, javascript, etc.. project. If anything, the guiding principles that make Clojure a well designed language at the micro level should have an exponential effect the same way poor decisions do.

Re: Clojure at Netflix (2013) [slides]

#228

Post by diab0lic »

vram22 wrote: post #183
diab0lic wrote: post #104

Earlier quoted context omitted.

Hey, sorry I saw that and typed a quick response just as I woke up. I'm not usually at a computer so early in the day. I'll address these now that I'm in front of a machine. :) > Did they keep writing more Clojure? Yes but it has never been the primary language at Netflix. > How much more did they rewrite from Java to Clojure? Very little, if any was rewritten from Java. > If so, how much of their code is now in Cloj…

>Clojure code bases tend to be much smaller than Java. What are the reasons for this? FP language vs. OOP? Less boilerplate (again maybe due to FP)? Higher-level abstractions in the language or libraries? I have seen that F# code (another FP language, although I've read F# is more from the ML family via OCaml, vs. Clojure being from the Lisp family) can be significantly shorter than equivalent C# code, for example, a…

The two sibling replies to my own do a good job of enumerating the reasons;

- Macros allow you to greatly reduce boilerplate. - Lots of built in functions that operate on very few types. - Compact intertop

And I agree with wild_preference that this isn't necessarily a good vs. bad debate. It is just a matter of fact that Clojure is concise.

Re: Clojure at Netflix (2013) [slides]

#229

Post by matt-noonan »

lgessler wrote: post #179
tanin wrote: post #166

Earlier quoted context omitted.

Is Spec simply a test? Or is it something more? I keep hearing about it, but it seems to be more than just a test case. Let's take a simple example. I have an object with the method named "get". But I call "fetch" in my code. When will I see this error? During compile time or run time?

Think of it this way: spec gives you an easy way to ensure that your data conforms to arbitrary predicate functions. Suppose you have an XML format, that has books and authors[1]. The elements look something like this: , . Obviously, you want to make sure that book's author-id attribute will always refer to an author that actually exists. This is something a (Java-style) static type system can't do: it doesn't know a…

That blog post was a nice read, thanks!

I realize this is just one example, but you might be interested to know that I've done some work making this kind of invariant enforceable via static types (though in practice, you need a combination of features that aren't found in many mainstream language besides Haskell afaik).

See the README here for motivation and some examples (in Haskell, but hopefully the idea is still clear): https://github.com/matt-noonan/justified-containers

Or the tutorial module here: https://hackage.haskell.org/package/justified-containers-0.3...

Or this paper, if you really want to go off the deep end: http://kataskeue.com/gdp.pdf

Re: Clojure at Netflix (2013) [slides]

#230

Post by jwhitlark »

tanin wrote: post #166
KingMob wrote: post #61

Earlier quoted context omitted.

I use Clojure in the large, and the lack of static typing isn't something I miss. FWIW, spec allows you make similar guarantees if you use it (technically more, since you can express more than HM type systems). I think the static/dynamic divide speaks to deep divisions in programmer personalities, but if you're open to suggestion, I would put it like this: As Haskell/OCaML are to static langs like Java, Clojure is to…

Is Spec simply a test? Or is it something more? I keep hearing about it, but it seems to be more than just a test case. Let's take a simple example. I have an object with the method named "get". But I call "fetch" in my code. When will I see this error? During compile time or run time?

At its core, spec lets you ask "Do I have what I need to do my job?", and if you have more, spec doesn't care. Type systems answer the question "Do I have exactly the machine primitives I expected, no less and no more".

It's about inspecting and verifying minimums at runtime.

Post reply on HN