Six Years of Professional Clojure • Hacker News Board

Re: Six Years of Professional Clojure

#231

Post by didibus » Tue, Aug 03, 2021, 9:52 PM UTC

hota_mazi wrote: post #223

didibus wrote: post #210

Earlier quoted context omitted.

I believe you're mistaken, but please explain otherwise? None of those seem to require type information from my reasoning (and are also all available in Emacs for Clojure) For example, moving a function from one namespace to another, you know where this function is being used from the require declarations, and you know where you've been told to move it too and where it currently resides. So you can simply change the…

Sure: https://www.beust.com/weblog/2021/06/20/refactoring-a-dynami... Even Smalltalk's refactoring browser made mistakes which humans had to fix by hand. Which is not surprising, because in the absence of type annotation, the IDE doesn't have enough knowledge to perform safe refactorings.

That blog is talking about refactoring a method, not a function.

In Clojure, I'm talking about renaming a function, which can be done without types.

See the difference is that with a method:

x.f()

You have to know the type of `x` to find the right `f`, but with a function in Clojure:

    (ns foo
      (:require [a :refer [f]]))

    (f x)

The location of `f` is not dependent on the type of `x`, you known statically that this `f` is inside the namespace `a`, because of the require clause that says that in `foo`, `f` refers to the `f` inside of `a`.

And this is unambiguous in Clojure because there cannot be more than one `f` inside `a`.

If you had two `f` this would be the code in Clojure:

    (ns a)
    (defn f [] "I'm in a")

    (ns b)
    (defn f [] "I'm in b")

    (ns foo
      (:require [a :refer [f]]
                [b :refer [f] 
                   :rename {f bf}]))

    (f x)
    (bf x)

You're forced to rename the other f, and now it's clear statically again that `bf` is the `f` from `b` and `f` is the one from `a`, no need to know the type of `x` for it.

Re: Six Years of Professional Clojure

#232

Post by smaudet » Wed, Aug 04, 2021, 2:48 AM UTC

hota_mazi wrote: post #200

didibus wrote: post #196

Earlier quoted context omitted.

In Clojure you know the number of arguments to a function and the name of functions and variables, and the code is all very well structured as an AST (being a Lisp). So you can do a lot of refactorings with that such as: Rename function, rename variable, rename namespace, extract constant, extract function, extract local variable, extract global variable, convert to thread-first, convert to thread-last, auto-import,…

All these refactorings can only be done automatically and safely if you have type annotations (i.e. core). Without them, all these refactorings can break your code (as in, not even compiling, let alone run).

You are pushing fud about not having typing systems at all... they are valuable to automated systems for introspection to some degree.

However you are talking about typing as if all typing is static - static typing has little value beyond warm and fuzzies on the developers part, dynamic typed systems are able to perform just as well. At which point, the dynamic part can allow you to mostly drop the types.

statically typed systems, you will note, tend to come with ecosystems dedicated to using the static bits as little as possible. And they provide no guarantee of correctness.

Yes, new devs might be able to latch onto some specific typing a bit better, but I don't care if you have all the automated refactors and a hundred new employees, if your codebase sucks and is incorrect, your static analysis is worth didly squat.

Re: Six Years of Professional Clojure

#233

Post by tsss » Wed, Aug 04, 2021, 11:01 AM UTC

blacktriangle wrote: post #229

tsss wrote: post #228

Earlier quoted context omitted.

You can very easily write a "safe" version of that function that will not type check and in this case you don't even need dependent types. So: bad example on your part.

So you're saying without dependent types you can express in a type a function that will return double each element in the list thus removing the need to test the function? If you can do that, that's awesome, but I'm not seeing how.

I meant that you can write a version of your function with the same definition that will not type check since `take 2` is illegal for lists of length less than 2.

As for a function that will "double" a list, i.e. turn [1,2] into [1,1,2,2]: That is definitely possible with dependent types as they can express arbitrary statements. I'm not sure if you can do it without dependent types, but I'm inclined to say yes: something like an HList should work. Universal quantification over the HList parameters will ensure that the only way to create new values of the parameter types is to copy the old ones, as long as you disallow any form of `forall a. () -> a` in the type system.

Something like this, which is just the `double` function lifted into the universe of types, _might_ work, though its utility is questionable:

    type family DoubleList xs :: 'HList -> 'HList where
        DoubleCons ('HCons x ': xs) =  'HCons x ': 'HCons x ': Double xs
        DoubleNil 'HNil = 'HNil

Re: Six Years of Professional Clojure

#234

Post by blacktriangle » Wed, Aug 04, 2021, 12:29 PM UTC

tsss wrote: post #233

blacktriangle wrote: post #229

Earlier quoted context omitted.

So you're saying without dependent types you can express in a type a function that will return double each element in the list thus removing the need to test the function? If you can do that, that's awesome, but I'm not seeing how.

I meant that you can write a version of your function with the same definition that will not type check since `take 2` is illegal for lists of length less than 2. As for a function that will "double" a list, i.e. turn [1,2] into [1,1,2,2]: That is definitely possible with dependent types as they can express arbitrary statements. I'm not sure if you can do it without dependent types, but I'm inclined to say yes: somet…

So totally different function, I meant by double to turn [1,2] into [2,4], however that's a really neat example. I hadn't seen the family extension in Haskell before. You're right there is a ton more to type systems than I was aware of. I was following some links in this thread and found this as well: https://www.parsonsmatt.org/2017/10/11/type_safety_back_and_....

I was less than impressed with type systems because like the blog post says, they tend to just kick the can down the road. The blog post uses a technique like you did in your example where rather than emitting more complicated type, they use the type system to protect the inputs of the function thus moving handling with the problem to the edges of the system which seems like a huge win. Between your example and that post I'm starting to see what people mean when they talk about programming in types, as its almost like the type system become a DSL with its own built-in test suite with which to program rather than a full programming language.

Either way very thought provoking, thank you for your responses.

Re: Six Years of Professional Clojure

#235

Post by tsss » Wed, Aug 04, 2021, 2:29 PM UTC

blacktriangle wrote: post #234

tsss wrote: post #233

Earlier quoted context omitted.

I meant that you can write a version of your function with the same definition that will not type check since `take 2` is illegal for lists of length less than 2. As for a function that will "double" a list, i.e. turn [1,2] into [1,1,2,2]: That is definitely possible with dependent types as they can express arbitrary statements. I'm not sure if you can do it without dependent types, but I'm inclined to say yes: somet…

So totally different function, I meant by double to turn [1,2] into [2,4], however that's a really neat example. I hadn't seen the family extension in Haskell before. You're right there is a ton more to type systems than I was aware of. I was following some links in this thread and found this as well: https://www.parsonsmatt.org/2017/10/11/type_safety_back_and_... . I was less than impressed with type systems because…

> I meant by double to turn [1,2] into [2,4]

Hmm, I think you can do that too, but you'd have to assign each int value its own singleton type, which would be ridiculous and not gain you anything since you're just moving the logic up one level in the hierarchy of universes.

> what people mean when they talk about programming in types

If the type system is powerful enough then you can express any function at the type level. Some languages with universal polymorphism make no difference between types and terms. Any function can also be used at the type-level, kind-level and so on. Though usually just defining a simple wrapper type with smart constructor will get you 80% of the way in a business application with 2% of the effort of real type-level programming.

Re: Six Years of Professional Clojure

#236

Post by c-cube » Wed, Aug 04, 2021, 2:59 PM UTC

travisjungroth wrote: post #160

c-cube wrote: post #142

Earlier quoted context omitted.

I'm not saying types always model your problem properly! That's not even well specified. I'm saying that "x has type foo" is never wrong if the program typechecks properly. That's totally different, and it means that you can rely on type annotations as being correct, up-to-date documentation. You can also trust that functions are never applied to the wrong number of arguments, or the wrong types; my point is that thi…

You can statically analyze specs and check them at runtime if you want.

Can you? Is there any tool that actually does that reliably?

Re: Six Years of Professional Clojure

#237

Post by kingdomcome50 » Wed, Aug 04, 2021, 3:13 PM UTC

lkitching wrote: post #230

kingdomcome50 wrote: post #226

Earlier quoted context omitted.

> but avoiding Maybe is more preferable still You can't avoid `Maybe` in this system. It is in the nature of the problem (as it is designed) that the input might not exist (and therefore a list might be empty). The question isn't one of avoidance, rather, integration. How do we deal with problems like the example? "Parse don't validate" is a great way to deal with it! Even more convenient is the existence of a tool t…

Your example does not achieve idential behaviour at all since it 'parses' an [a] to another [a] and therefore throws away the very property you've just checked. The (NonEmpty a) property encodes the non-emptiness of the list in the type which is then known at every point the list is accessed throughout the entire rest of the program. The point is not just to check the non-emptiness in main as you appear to be implyin…

I think you are confusing implementation with behavior. That is, I am achieving that same result through different means. I am mostly uninterested in specifically how the configuration string is parsed. It's not really important.

What is important is that we know we will have to deal with the possibility of something not existing. That is where the complexity lies, and where we want to take care to make our program as sensible as possible. Validating your input to throw an exception or return is one way to satisfy the compiler, another way is to use `Maybe` as intended. The author's "solution" is simply a poor illustration of parsing over validation (read that sentence again).

I suspect, and this applies to you as well, that they are just not comfortable working with the `Maybe` construct. Adding extra ceremony to remove a `Maybe` is simply not worth the trouble, and your idea of "continuously propagating" is severely overblown. Again, we can write every single line of the rest of our program as if `Cache` exists. You don't need to "handle" anything extra (other than the holding the concept of a slightly more complex value in your mind).

Re: Six Years of Professional Clojure

#238

Post by travisjungroth » Wed, Aug 04, 2021, 11:38 PM UTC

c-cube wrote: post #236

travisjungroth wrote: post #160

Earlier quoted context omitted.

You can statically analyze specs and check them at runtime if you want.

Can you? Is there any tool that actually does that reliably?

https://clojure.org/guides/spec#_using_spec_for_validation

Re: Six Years of Professional Clojure

#239

Post by [deleted] » Thu, Aug 05, 2021, 7:00 AM UTC

dharmaturtle wrote: post #225

didibus wrote: post #187

Earlier quoted context omitted.

Multi-threaded code is normally not implemented in an async style, but instead is done where each thread of execution is synchronous. Async style comes into play generally for languages that lack real threads, or as a way to manage callbacks (even if single threaded), or in order to wait for blocking IO without the need for a real thread. So ya, it's idiomatic to use blocking to coordinate between different threads i…

Thanks for the reply - what you say makes a lot of sense. I watched Rich's talk on Async and was like... "cool so `core.async` follows this pattern right?!" ...not quite. I'll check out your other links though, much appreciated. Also hearing that I should just be okay with blocking is well, good to hear explicitly.

[deleted]

Re: Six Years of Professional Clojure

#240

Post by lkitching » Thu, Aug 05, 2021, 10:43 AM UTC

kingdomcome50 wrote: post #237

lkitching wrote: post #230

Earlier quoted context omitted.

Your example does not achieve idential behaviour at all since it 'parses' an [a] to another [a] and therefore throws away the very property you've just checked. The (NonEmpty a) property encodes the non-emptiness of the list in the type which is then known at every point the list is accessed throughout the entire rest of the program. The point is not just to check the non-emptiness in main as you appear to be implyin…

I think you are confusing implementation with behavior. That is, I am achieving that same result through different means. I am mostly uninterested in specifically how the configuration string is parsed. It's not really important. What is important is that we know we will have to deal with the possibility of something not existing. That is where the complexity lies, and where we want to take care to make our program a…

The difference between parsing and validation in the author's formulation is not between returning Maybe and throwing an exception, it's between returning a more precise type and not. Here's the types of the two version of `getConfigurationDirectories`:

    getConfigurationDirectories :: IO [FilePath]
    getConfigurationDirectories :: IO (NonEmpty FilePath)

The second version is preferred because the (NonEmpty FilePath) encodes the property that was checked in the type which means it doesn't have to be handled repeatedly throughout the entire rest of the program.

Yes the second version could have been changed to one of:

    getConfigurationDirectories :: IO (Maybe (NonEmpty FilePath))
    getConfigurationDirectories :: MaybeT IO (NonEmpty FilePath)

but this would only have moved the error reporting up one level to the main function. I would guess the existing version was chosen to simplify the types for a non-Haskell audience.

your attempted 'improvement' of using

     getConfigurationDirectories :: IO (Maybe [FilePath])

is NOT an example of parsing because [FilePath] does not remove the possibility (in the types!) of the list being empty. When you later attempted to use

    maybeCache >>= useCache

this requires the type of useCache to have type

    [FilePath] -> IO a

for some output type a. This function must deal with the possibility of the input list being empty because the type allows it. Every call to `head` returns (Maybe FilePath) and must handle the Nothing case. Neither I nor the author is unaware that there are many combinators that make this more convenient than explicit matching against Just/Nothing but doing so is strictly worse than returning a FilePath directly. Presumably none of the lower-level functions will be able to provide a default FilePath to use so every single one will be forced to return a Maybe somewhere in their return type (or use fromJust which is very ugly). This affects every single one of their callees which will again be forced to propagate Maybe up to their callees etc. To reiterate: the issue is not the possible non-existence of Cache, which can be handled in main. It's that the representation of Cache forces every single operation on it (of which head is just one simple example) to potentially have to represent conditions that should not actually be possible. This is a failure to 'make invalid states unprepresentable', which most proponents of static types aspire to.