Live data from Hacker News

Tour of our 250k line Clojure codebase

tech.redplanetlabs.com

151–160 of 237 posts

Re: Tour of our 250k line Clojure codebase

#151

This is the first time I'm looking closely at Clojure code and it seems wholly antithetical to the practice of software engineering. Not very readable, too many ways to create magical jank, odd coding conventions, and it looks like refactoring and maintenance would be a nightmare. I'm very glad the community has had the collective sense to not adopt this widely.

It's weird to dismiss something so easily after seeing it for the first time. Maybe you should give it more time.

Re: Tour of our 250k line Clojure codebase

#152
post #65

Earlier quoted context omitted.

> Lots of classes and interfaces but they are all small and with a single purpose. That's the side effect. You ultimately end up with more code and not less even though it saves you from type checks. There are trade offs for both.

One thing I wish for is some kind of "type tags". Being able to express concepts like List[Widget], List[Widget, Nonempty], List[Widget, Nonempty, Sorted], Vector[User, Sorted], etc. - or even, more generic, [ , NonEmpty] (where and are parameters, like in C++ templates) - without implementing an explicit new type for each. Logic verification through typing would then involve not just changing "main" types, but also…

When you have higher-kinded types you can build that kind of thing yourself. Dependent types going further in that direction. Take a look at Idris.

Re: Tour of our 250k line Clojure codebase

#153

Earlier quoted context omitted.

I personally bounce back and forth about this. My experience is probably colored by the fact that I'm doing this in C++. Boilerplate gets annoying there (and attempts to cut it down tend to produce lots of incomprehensible function templates). I like the idea of using types to encode assertions at a fine granularity. I dislike the amount of tiny little functions this creates. I also dislike that the resulting code is…

Ok yes, C++ might not be the greatest language for this. My experience here is mostly from Kotlin which is a great language for this. Nullability, extension methods, (reified) generics, data classes, delegates, etc can all help reduce boilerplate.

You should take a look at Scala. Lots of things that have to be special-case language features in Kotlin become just a straightforward use of higher-kinded types or a combination of a couple of existing language features.

Re: Tour of our 250k line Clojure codebase

#154

> Detecting an error when creating a record is much better than when using it later on, as during creation you have the context needed to debug the problem. This is a great insight no matter what language or framework you're operating in. Laziness has its virtues, but invariants, validity checks, run-time type checks, etc. should all be performed as early (and often) as possible - it's much easier to debug issues whe…

> invariants, validity checks, run-time type checks, etc. should all be performed as early (and often) as possible

Having worked on numerous large-scale distributed systems, I strongly disagree (and I would think the author would too).

There is a tradeoff to strictness, which is coupling. You want to validate and type check as much as necessary—but not more. If you are over-coupled in a distributed system, it can make it difficult to change your system incrementally, as one must also do with distributed systems. A canonical example of this is closed enums, which don’t tolerate unknown values. Good luck adding a new enum value if every part of your system is strictly validating that. Each part of the system should validate what it needs in order to guarantee that that system can operate correctly, and no more.

Re: Tour of our 250k line Clojure codebase

#155

> Detecting an error when creating a record is much better than when using it later on, as during creation you have the context needed to debug the problem. This is a great insight no matter what language or framework you're operating in. Laziness has its virtues, but invariants, validity checks, run-time type checks, etc. should all be performed as early (and often) as possible - it's much easier to debug issues whe…

Yes, and this is where statically typed languages shine (in my opinion). I like programming in a style that makes heavy use of the type system to enforce this. For example when writing an api endpoint to create a task I would typically deserialise the json into a CreateTaskRequest. If the object is created without exceptions I can be sure it is valid. CreateTaskRequest implements the ToTask interface. The service lay…

Isn't this as far from Clojure's "it's just data" as it gets? The author was making the case for not doing this, ie. turning data into Java-style classes and interfaces.

Re: Tour of our 250k line Clojure codebase

#156

Earlier quoted context omitted.

Nathan Marz was the original creator of what became Apache Storm [1], which powered Twitter for some time. Skepticism is healthy, perhaps even warranted here, but I'm not betting against him just yet. [1] https://en.wikipedia.org/wiki/Apache_Storm

He is also the creator of Cascalog (Hadoop query dsl in Clojure) and the Lambda architecture pattern. Not lambda as we know it now popularised by AWS, but an architecture for stream processing where batch views from expensive and slow batch jobs are combined with speed views from stream processors into the final live result. https://en.m.wikipedia.org/wiki/Lambda_architecture

100x increase in productivity is a silly, hyperbolic claim no matter who makes it. I'd even be skeptical of a 2x claim, because in 25 years I have yet to see any of these productivity plays actually pan out. What I have seen are small incremental improvements here and there, but you can't point to anything in the recent past that has improved productivity 10x or 100x (unless your old process was just total crap).

At best I would expect a small niche collection of very specific tasks to be improved, but definitely not applicable to general productivity.

Re: Tour of our 250k line Clojure codebase

#157

Earlier quoted context omitted.

I like this design for making requests, but I don't like it for serving requests. Because if you don't just make every field a String in CreateTaskRequest, and you want to examine it in some way, then you're losing information. E.g. if you make a field an int, and they provide a non-int value, your CreateTaskRequest can't hold that value so its just gonna have a 0. A class full of strings feels like a code smell - bu…

If a field is an int and the incoming data has a string the parsing shouldn't succeed - instead it should give an error like "expected an int but saw a string". Maybe include the string it saw in the error. Add in the JSON path for bonus debugging points. You shouldn't get to the point of having a CreateTaskRequest with a 0 that shouldn't be there.

I understand and have done that before with XyzRequest classes. What I'm saying is the field shouldn't be an int. It's not the proper level of abstraction for what it represents, and any of: tossing out, transforming, or hiding under layers of abstraction the data that comes into your system like this is a bad idea and is the sort of thing that makes people hate OOP. Especially when perfectly fine alternatives exist for this.

Re: Tour of our 250k line Clojure codebase

#158
post #60

we have a clojure codebase that's about 100k lines. Honestly I'm kinda fed up with it. Certain 3rd party libs we've used have been abandoned. We wrote our own libs for a major framework and it is failing behind. Too many "I'm very clever" functions that are hard to understand and also have subtle bugs.

Obligatory evangalism: Considered Kotlin as a JVM-lang-of-choice? We use it on all our backends and we really love it.

I think its a "to each their own" honestly. We use Java, Kotlin, Scala and Clojure, and of all four my favourite by far is Clojure.

We also don't suffer from any of the problems mentioned above, had no issues with libraries and no one abuses macros or makes prod repl changes willy nilly, etc.

Re: Tour of our 250k line Clojure codebase

#159

Earlier quoted context omitted.

He is also the creator of Cascalog (Hadoop query dsl in Clojure) and the Lambda architecture pattern. Not lambda as we know it now popularised by AWS, but an architecture for stream processing where batch views from expensive and slow batch jobs are combined with speed views from stream processors into the final live result. https://en.m.wikipedia.org/wiki/Lambda_architecture

100x increase in productivity is a silly, hyperbolic claim no matter who makes it. I'd even be skeptical of a 2x claim, because in 25 years I have yet to see any of these productivity plays actually pan out. What I have seen are small incremental improvements here and there, but you can't point to anything in the recent past that has improved productivity 10x or 100x (unless your old process was just total crap). At…

you concluded something is impossible based on the fact that you have never seen it before? talking about silly claims :)

Re: Tour of our 250k line Clojure codebase

#160
post #3

Earlier quoted context omitted.

I've found I typically reach for clojure when i need to do something on the jvm and want a better java than java.

Clojure's value prop is: 1) default immutability (same simple data structures used in every library -> ecosystem composes better) 2) portable code across multiple host platforms (jvm, node, browser) 3) metaprogramming IMO, in 2007, immutability on the JVM was a competitive value prop, but in 2021+ it is nothing special. It is the combination of the three things which is a competitive value prop today. Metaprogramming…

I think Clojure is still a great choice when your concurrent solution to a problem is amenable to working with stale data. If it is then the correctness of your design will rely heavily upon immutability, and the bugs that can be introduced by not using an immutable-first language like Clojure just isn't worth the risk.
Post reply on HN