Live data from Hacker News

Tour of our 250k line Clojure codebase

tech.redplanetlabs.com

91–100 of 237 posts

Re: Tour of our 250k line Clojure codebase

#91

> One of the coolest parts of our codebase is the new general purpose language at its foundation. Though the semantics of the language are substantially different than Clojure, it’s defined entirely within Clojure using macros to express the differing behavior. It compiles directly to bytecode using the ASM library. The rest of our system is built using both this language and vanilla Clojure, interoperating seamlessl…

I have dabbled in Clojure and Julia, and both of them support macro to make it easier to develop DSL looking syntax. I have always wondered how difficult it it to debug macro heavy code - where's other language cite 'no macro' as a feature such as Zig.

> I have always wondered how difficult it it to debug macro heavy code

It isn't much harder than regular core. `macroexpand' is your friend! Particularly when wrapped by your IDE[0] into a "macrostepper" tool, which lets you macroexpand into an overlay[1], step by step.

Ultimately, macros are just functions - if slightly peculiar ones (they receive unevaluated arguments, and their return values are treated as if they were the code at the point of invocation). You can unit-test them just like any other function.

That's not to say there aren't macros that are very tough to debug. But the problem isn't macro-heavy code, as in code containing a lot of macros. The problem are heavy macros - monstrosities with complex internal logic, expanding their inputs into large amount of code generation, hidden state, etc. Complex macros like these take skill to write[2], but they're also rare to see.

--

[0] - By which I mean Emacs, though I guess there are Clojure IDEs now too.

[1] - I.e. a read-only view replacing the code you're macroexpanding on screen.

[2] - First rule: minimize the amount of code within the actual macro definition, move all logic into functions to be called by the macro, and unit test those extensively.

Re: Tour of our 250k line Clojure codebase

#92
post #64

Earlier quoted context omitted.

Most static type systems that can do what clojure.spec can do tend to include runtime assertion and type checks and do not erase type data from runtime (what some static type zealots call "uni-type" approach). For example Ada's type system, which has equivalent of Common Lisp's SATISFIES construct, which implements a runtime type assert that can use all the power of the language.

Sorry, I don't understand >Most static type systems that can do what clojure.spec can do tend to include runtime assertion and type checks and do not erase type data from runtime (what some static type zealots call "uni-type" approach). When you don't erase the type data... you're gonna have more than one type. How is this a "uni-type" approach?

A popular (stupid) talking point in the stupid discussions about static/dynamic while missing that the axes were orthogonal, was for proponents of static types to claim that dynamic languages were "unityped" based on some convoluted logic about tagging data at runtime.

Re: Tour of our 250k line Clojure codebase

#93
post #60

we have a clojure codebase that's about 100k lines. Honestly I'm kinda fed up with it. Certain 3rd party libs we've used have been abandoned. We wrote our own libs for a major framework and it is failing behind. Too many "I'm very clever" functions that are hard to understand and also have subtle bugs.

I wonder if they'd be abandoned if companies using them considered these 3rd party libraries valuable enough to contribute/fund the development.

Re: Tour of our 250k line Clojure codebase

#94

Earlier quoted context omitted.

One thing I wish for is some kind of "type tags". Being able to express concepts like List[Widget], List[Widget, Nonempty], List[Widget, Nonempty, Sorted], Vector[User, Sorted], etc. - or even, more generic, [ , NonEmpty] (where and are parameters, like in C++ templates) - without implementing an explicit new type for each. Logic verification through typing would then involve not just changing "main" types, but also…

A language like Kotlin can do some of these things using delegates, interfaces and extension methods. For example a MutableList can be dropped down to a List which is not mutable. And a generic conversion from Collection to NonEmptyCollection should be trivial to write as an extension method.

Can Kotlin handle multiple "tags" on a type as a set, and not a sequence? I'm not familiar with the language, so I'll use a C++ analogy. If you tried to tag types in C++, you'd end up with something like:

  TaggedType
but such type is strictly not the same as:

  TaggedType
What I mean by "set" instead of a "sequence" is to have the two lines above represent the same type, i.e. the order of tags should not matter.

Re: Tour of our 250k line Clojure codebase

#95
post #60

we have a clojure codebase that's about 100k lines. Honestly I'm kinda fed up with it. Certain 3rd party libs we've used have been abandoned. We wrote our own libs for a major framework and it is failing behind. Too many "I'm very clever" functions that are hard to understand and also have subtle bugs.

Seconded. I work in a clojure codebase that were trying to get out of. There's just dead libraries everywhere and stuff that maintained by one person that gets no updates at all. That or we just end up making functional "wrappers" around Java libraries and at that point we might as well just write straight Java.

Also yea everyone wants to be so damn smart having macros within macros within macros that no one knows what the original intent of the code is anymore.

The repl based development I find also breeds a really bad mentality of forgoing building a deployment process and instead people just repl in and make a bunch of changes and prod rarely matches whats checked into github.

Re: Tour of our 250k line Clojure codebase

#96
post #82

Earlier quoted context omitted.

Yes, and this is where statically typed languages shine (in my opinion). I like programming in a style that makes heavy use of the type system to enforce this. For example when writing an api endpoint to create a task I would typically deserialise the json into a CreateTaskRequest. If the object is created without exceptions I can be sure it is valid. CreateTaskRequest implements the ToTask interface. The service lay…

> If the object is created without exceptions I can be sure it is valid. Without some kind of custom validation system in place (JSON schema, property attributes, etc), that doesn't tell you much. With most serialization libraries I've used the default settings would let you deserialize {} into any class without an exception and all the properties would just have the default value for their type. Using stricter setti…

You implement your own custom validation. The point is to encode the fact that you've validated a value in its type. So you have e.g. DeserializeJSON(String) -> Foo, and then ValidateFoo(Foo) -> ValidatedFoo. And then all the business code works on ValidatedFoo.

Re: Tour of our 250k line Clojure codebase

#97
post #92

Earlier quoted context omitted.

Sorry, I don't understand >Most static type systems that can do what clojure.spec can do tend to include runtime assertion and type checks and do not erase type data from runtime (what some static type zealots call "uni-type" approach). When you don't erase the type data... you're gonna have more than one type. How is this a "uni-type" approach?

A popular (stupid) talking point in the stupid discussions about static/dynamic while missing that the axes were orthogonal, was for proponents of static types to claim that dynamic languages were "unityped" based on some convoluted logic about tagging data at runtime.

As someone who prefers static types, I agree with you that claiming dynamic languages are "unityped" is stupid. Javascript/Python/Clojure all literally have types.

Re: Tour of our 250k line Clojure codebase

#98

> One of the coolest parts of our codebase is the new general purpose language at its foundation. Though the semantics of the language are substantially different than Clojure, it’s defined entirely within Clojure using macros to express the differing behavior. It compiles directly to bytecode using the ASM library. The rest of our system is built using both this language and vanilla Clojure, interoperating seamlessl…

JSX would be an (early) example of this class of language. XML is equivalent to s-expressions. So React.js (drunkly) is just a reactive language embedded in a normal language with seamless ability to hop between them. I elaborate about this equivalence here https://www.reddit.com/r/Clojure/comments/mavg81/the_distinc... . The brilliance is they did this so smoothly that nobody noticed, and with dynamic types, and without the typed people even noticing!

Re: Tour of our 250k line Clojure codebase

#99
post #82

Earlier quoted context omitted.

Yes, and this is where statically typed languages shine (in my opinion). I like programming in a style that makes heavy use of the type system to enforce this. For example when writing an api endpoint to create a task I would typically deserialise the json into a CreateTaskRequest. If the object is created without exceptions I can be sure it is valid. CreateTaskRequest implements the ToTask interface. The service lay…

> If the object is created without exceptions I can be sure it is valid. Without some kind of custom validation system in place (JSON schema, property attributes, etc), that doesn't tell you much. With most serialization libraries I've used the default settings would let you deserialize {} into any class without an exception and all the properties would just have the default value for their type. Using stricter setti…

That depends on the language and deserialisation library used. And that is also the reason why I have multiple classes.

My CreateTaskRequest class which is used for deserialisation only does not have defaults (unless intentional) so it throws when required values are not present in the source json. It also takes care of dateformat/uuid parsing. The output is always fully valid typed or it throws. It’s a 1-to-1 mapping on what is described in the API docs.

The service layer that takes CreateTaskRequest and converts to Task for persistence is where the business logic validation happens (valid foreign keys, date ranges, etc, unique checks). Cleanly separated from deserialization.

For reference: I use Kotlin with Jackson which has great support for this.

Re: Tour of our 250k line Clojure codebase

#100
post #60

we have a clojure codebase that's about 100k lines. Honestly I'm kinda fed up with it. Certain 3rd party libs we've used have been abandoned. We wrote our own libs for a major framework and it is failing behind. Too many "I'm very clever" functions that are hard to understand and also have subtle bugs.

Are those "clever" functions pure?
Post reply on HN