Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tour of our 250k line Clojure codebase

tech.redplanetlabs.com

Post reply on HN
Print view View original

181–190 of 237 posts

Re: Tour of our 250k line Clojure codebase

#181

Post by BiteCode_dev »

bedobi wrote: post #137
vbsteven wrote: post #35

Earlier quoted context omitted.

Yes, and this is where statically typed languages shine (in my opinion). I like programming in a style that makes heavy use of the type system to enforce this. For example when writing an api endpoint to create a task I would typically deserialise the json into a CreateTaskRequest. If the object is created without exceptions I can be sure it is valid. CreateTaskRequest implements the ToTask interface. The service lay…

This is more or less how my teams' back end REST API server code is written as well. It's far, far superior to the idiotic God classes most Java devs tend to write - the ones covered in attributes that are sometimes populated, other times not, and annotations for ten different purposes. Just no. Don't use the same single class to parse incoming requests, persist records at the innermost db layer, serializing the outg…

That's basically how most modern frameworks do it.

Django-Rest-Framework does that with view + serializer + validators.

A more recent example, with a leaner and cooler implementation is FastAPI, where type hints are also use to declare validation on your end point, while a model is used for serializing the response.

Re: Tour of our 250k line Clojure codebase

#182

Post by TeMPOraL »

lmm wrote: post #152
TeMPOraL wrote: post #72

Earlier quoted context omitted.

One thing I wish for is some kind of "type tags". Being able to express concepts like List[Widget], List[Widget, Nonempty], List[Widget, Nonempty, Sorted], Vector[User, Sorted], etc. - or even, more generic, [ , NonEmpty] (where and are parameters, like in C++ templates) - without implementing an explicit new type for each. Logic verification through typing would then involve not just changing "main" types, but also…

When you have higher-kinded types you can build that kind of thing yourself. Dependent types going further in that direction. Take a look at Idris.

I will, thanks. I never had a chance to catch up on the "state of the art" in typing systems.

I'm not even sure if my "types as a set of tags" idea makes much sense - perhaps it decays to what is typically understood as types. Or perhaps it hits computability problems.

I did some mental experiments on a "set of tags" type system last night, and I quickly realized the complexity will be around deciding when to keep a tag (property) on a type, and when to drop it. With a set of tags being open, a function could not possibly know about them. This creates a problem.

Consider a function like: Sort([List[T], ...], [Function([T, T] -> Bool)]) -> [List[T], Sorted, ...]. Takes a list and a comparator, sorts the list, attaches a "Sorted" tag to return type, retains all other tags. Looks like a reasonable definition. Let's look at the use cases below (with P being some applicable predicate):

- Sort([List[Int], P) -> [List[Int], Sorted] -- OK!

- Sort([List[Int], NonEmpty], P) -> [List[Int] Sorted, NonEmpty] -- Awesome, exactly what I want!

- Sort([List[Int], Foobar], P) -> ...? Should it be [List[Int], Sorted, Foobar]? But what if Foobar designates some business rule that depends on the element ordering? How is Sort supposed to know?

Without knowing a tag, we'd sometimes want a function to drop it, and other times want the function to retain it. I have no idea how to solve this at compile time (or even at runtime, without crippling the tagging system).

Re: Tour of our 250k line Clojure codebase

#183

Post by AtNightWeCode »

didibus wrote: post #177
AtNightWeCode wrote: post #42

Earlier quoted context omitted.

What I heard from colleagues that work with Clojure is that it is a horrible language where the default way of writing code is an imperative programming style where contexts are passed around and updated. Far from the concepts of functional programming.

Well it's a bit preposterous of you to say that without actually having tried Clojure in good faith. But, since we all make opinions from others, let me provide you a balance of opinions by giving you mine. I'm a senior engineer and I currently use Clojure professionally. I find it to be a lovely, fun and productive language, my favorite one to date actually. I have prior professional experience with C++, C#, ActionS…

Very good explanation.

Re: Tour of our 250k line Clojure codebase

#184

Post by capableweb »

phist_mcgee wrote: post #131
twobitshifter wrote: post #113

Earlier quoted context omitted.

The team is impressive, I was wondering what all this new internal language, 400 macros, etc., could be put towards, thinking they were stuck in over-engineering. But after seeing that promise for their app, I changed my mind. Something that’s capable of making you 100 times more productive probably does need that level of development.

I don't know even still if efficiency is correlated to lines of code in a product. Sure maybe a weak relationship at best, but the scale of the software does not necessarily mean it will be any more useful than something else.

I don't think twobitshifter's point was that "more code === better product" either but rather that if it's a product with big scope, it probably has more code in it than if it was a product with narrow scope.

Re: Tour of our 250k line Clojure codebase

#185

Post by jwr »

coltnz wrote: post #144

While I've used all of them, some of their listed libraries are a little dated IMO. Of course this true of almost all mature codebases. For the sake of the less experienced, I'd point them to these substitutions in particular: Compojure: Reitit (which they used in the front end too, so migration maybe in progress) would be my preference for backend routing. Component: Integrant takes Component's ideas, but prefers th…

In a lage codebase, the cost of switching is huge, and if older libraries work well, there is often no clear incentive to do so.

Clojure in general being very stable and backwards-compatible makes it even more easier to just continue using older libraries. So what if the library is "dated"? If it works well, why not continue using it?

(I speak from my own experience)

Re: Tour of our 250k line Clojure codebase

#187

Post by [deleted] »

xpe wrote: post #134
chromanoid wrote: post #77

> One of the coolest parts of our codebase is the new general purpose language at its foundation. Though the semantics of the language are substantially different than Clojure, it’s defined entirely within Clojure using macros to express the differing behavior. It compiles directly to bytecode using the ASM library. The rest of our system is built using both this language and vanilla Clojure, interoperating seamlessl…

> Actually this sounds quite horrible. You understand the use case well enough to criticize it?

[deleted]

Re: Tour of our 250k line Clojure codebase

#188

Post by jwr »

I develop and maintain a 60k line Clojure+ClojureScript codebase by myself, so I can definitely confirm that Clojure does allow for smaller teams to maintain larger codebases :-)

I also fully agree with this:

> Detecting an error when creating a record is much better than when using it later on, as during creation you have the context needed to debug the problem.

I made it a rule to perform integrity checks as early as possible: when creating, accepting or transforming data. I use spec (but schema would work just as well here), and have lots of pre/post conditions in my code.

I tend to settle on "simpler" solutions. I use mount, rather than component, because it uses the namespace hierarchy and requires less code and management. I use very few macros, and try to use simpler tools rather than more complex ones.

I noticed that these days roughly 30-40% of the code I write deals with integrity checks, anomalies and anomaly handling.

Re: Tour of our 250k line Clojure codebase

#189

Post by j-pb »

stevenhuang wrote: post #179
cle wrote: post #154

Earlier quoted context omitted.

> invariants, validity checks, run-time type checks, etc. should all be performed as early (and often) as possible Having worked on numerous large-scale distributed systems, I strongly disagree (and I would think the author would too). There is a tradeoff to strictness, which is coupling. You want to validate and type check as much as necessary—but not more. If you are over-coupled in a distributed system, it can mak…

> Each part of the system should validate what it needs Easier said than done. Each part of the system can't know what it needs, since you've just added a new thing it might need to contend with. The point of closed enums is to force these checks, even if all you're doing is adding a new fall through to ignore it, since at least that means you've considered it.

> Each part of the system can't know what it needs,

Yeah it can. It needs what it needed before.

If you have a system that is shipping stuff to customers, it doesn't have to care about the new childhoodPetName Field that you've added for a different part of the system.

This is especially relevant if you handle any kind of description of the real world, e.g. in robotics or medicine, and have multiple distributed systems and codebases all communicating with each other.

The inability to gracefully ignore cases that don't fit the expected data model is a weakness most contemporary programming languages and type systems share.

Re: Tour of our 250k line Clojure codebase

#190

Post by stevenhuang »

j-pb wrote: post #189
stevenhuang wrote: post #179

Earlier quoted context omitted.

> Each part of the system should validate what it needs Easier said than done. Each part of the system can't know what it needs, since you've just added a new thing it might need to contend with. The point of closed enums is to force these checks, even if all you're doing is adding a new fall through to ignore it, since at least that means you've considered it.

> Each part of the system can't know what it needs, Yeah it can. It needs what it needed before. If you have a system that is shipping stuff to customers, it doesn't have to care about the new childhoodPetName Field that you've added for a different part of the system. This is especially relevant if you handle any kind of description of the real world, e.g. in robotics or medicine, and have multiple distributed syste…

If field is childhoodPetName and other case labels are e.g., mothersMaidenName, and the common operation is to perform input sanitization, then this new field would have introduced a security hole if it's just ignored.

It's a matter of correctness enforced at language level, and why languages like Rust's match statement enforces all checks at compile time by default.

Post reply on HN