Live data from Hacker News

Understanding the Power of Lisp (2020)

joshbradley.me

71–80 of 140 posts

Re: Understanding the Power of Lisp (2020)

#71
post #37
post #24

Earlier quoted context omitted.

I've learned Clojure. Tried and failed to see this unique power of macros. Truly asking for help: can you help explain what can I do with macros that I cannot do with functions? Or, maybe, cannot do with high quality or low complexity using functions?

Here's a simple example you can't write in most languages. (first-working (get-it-from-the-cache) (get-it-from-the-database) (get-it-from-an-external-api) (compute-it-the-slow-way) default-value) Note each of those functions could throw an exception. You want to ignore it (or you could log it) and move on to the next.

In JS/TS with this package[0]

     const firstSuccessfulOptionValue = fnReturnsOption
         .orElse(fn2ReturnsOption())
         .orElse(fn3ReturnsOption())
         .orElse(fn4ReturnsOption())
         .getOrElse("val");
[0]https://github.com/sbernheim4/excoptional/blob/main/src/inde...

Re: Understanding the Power of Lisp (2020)

#72
post #59
post #45

Earlier quoted context omitted.

I know a startup that hired a Marxist-collective group of programmers. I was told this story years ago, and they were acquired, so I’ll just name the firm - “White Ops”. This story is so absurd that I’m naming the firm in hopes someone can verify the accuracy of this, although I trust the person that told me the tale. The programmers were based in Canada and only wrote in Haskell. The CTO of White Ops had apparently…

You can do that in almost any language, though. There was a furry who worked as a web developer for a small firm. He had built their back-end runtime in MUCK scripting language. (A MUCK is like a MUD, but without the statistics and stuff, so it's much less a role-playing game than it is simply role- playing .) Eventually he was fired, probably because he spent more time pretending to be a highly sexualized female fox…

[deleted]

Re: Understanding the Power of Lisp (2020)

#73

I'm programming in Lisp (Chez Scheme) now. I've been a serious Lisp programmer (recreationally and sometimes professionally) for around 10 years and I have programmed in Common Lisp, Scheme, my own weird dialects, Emacs Lisp, etc. At this point I feel like almost everything written about Lisp is silly. Taken as a language family as a whole, there isn't much that separates Lisp from most of the other languages that ar…

> It won't really help you solve hard problems. I have to take issue with this. There are only two languages I consider when faced with a very hard problem: Common Lisp and Haskell. One can solve easy problems in any language. But when I have to solve a new problem from scratch that nobody's ever solved before, I don't reach for Java or Python. I reach for the languages that augment my brain rather than limiting it.

I agree. I have several times solved hard problems in something like Lisp or Haskell, only to have to port the solution to a more common language for integration later. While I would expect the port to take less time than figuring out the solution in the first place, it generally takes longer because the target language is riddled with gotchas and inefficiencies that make the actual programming more difficult.

Re: Understanding the Power of Lisp (2020)

#74
post #24

Earlier quoted context omitted.

I've learned Clojure. Tried and failed to see this unique power of macros. Truly asking for help: can you help explain what can I do with macros that I cannot do with functions? Or, maybe, cannot do with high quality or low complexity using functions?

My impression, and I hope I'm wrong, is that macros and meta programming were powerful ideas 30 years ago, but now most modern languages have generics, template meta programming [1] and reflection. I'm a curious amateur and probably out to lunch, but there you go. [1] Some other languages support similar, if not more powerful, compile-time facilities (such as Lisp macros), but those are outside the scope of this arti…

Macros are not generics even though you can use them as such. Macros define new statements (and definitions for the languages that make a distinction). This allows you to extend the language to the specific domain and it's impossible to do with functions/generics in general case.

Re: Understanding the Power of Lisp (2020)

#75
post #23

Earlier quoted context omitted.

don't count on (eq '(a b) '(a b)) being NIL. In Common Lisp it can be T. From what I read of Racket, it could be true, too.

If that were true I would consider it a very weird Common Lisp implementation. The main point is that you should never depend on it being true even though the two forms look the same. A couple more examples for the Common Lispers in the audience that highlight the difference between the reader and the evaluator: (setf foo (list #1=(list 3 4 5) #1#)) (eq (car foo) (cadr foo)) ; --> ?? What should we expect the second…

> If that were true I would consider it a very weird Common Lisp implementation.

One of those is called SBCL.

'The main point' is that you can't depend on the value of (EQ '(a b) '(a b)) being NIL.

Reason: a compiler compiles a Lisp file. The compiler sees literal data like '(a b) ' and '(a b). The compiler determines that these are 'similar' objects and 'coalesces' them into one object. It might even do it for '(a b) and '(1 a b), seeing that one is a sublist of the other and coalesces the first list into the sublist of the other.

Common Lisp compilers are explicitly allowed by the language standard to do these things with literal data in a source file. It defines a concept of 'similarity', where this is allowed.

Re: Understanding the Power of Lisp (2020)

#76
post #38

Earlier quoted context omitted.

> So that's how I view Lisp macros - creating code on the fly and executing it. This is an incorrect assumption. In compiled Lisps[0], macros are a compile-time construct, which manipulate the data structures that represent your code[1]. All manipulation occurs at compile-time. In interpreted Lisps, macro evaluation is temporally intertwined with program execution, but the two phases are logically distinct. [0]: Comp…

So Clojure programs are data structures, and you can pass a structure to a macro to rewrite it into something the compiler can evaluate. In this way you can extend the language. You can build your own DSL that solves your particular problem in an organic way that grows as you go along. If I understand that correctly I don't get it, because I would still have to write the macro, which in C# I would write as a function…

A canonical example of macro capabilities that methods/functions cannot recreate is a short-circuiting conditional.

Clojure's only built-in conditional operator is 'if'. Despite this, we have nice short-circuiting or, and, when, unless, and other conditional operations in Clojure, defined as macros.

Clojure (and C# and most languages) eagerly evaluate their function arguments. You cannot write a function that short circuits. Let's consider the following Clojure:

    (when false (infinite-loop))
If when is a function, it's compiled to the appropriate instructions to evaluate both arguments at call-time. This will cause our program to hang on (infinite-loop).

But, when is a macro, which means that during compilation time, the compiler defers to the when macro. The when macro is near-trivial.

    (defmacro when
      "Evaluates test. If logical true, evaluates body in an implicit do."
      {:added "1.0"}
      [test & body]
      (list 'if test (cons 'do body)))
So the compiler has encountered when and sends the data structure of the form to this macro to evaluate. The result of this macro evaluation is passed back to the compiler.

So, when receives a list, '(false (infinite-loop)). when returns a list to the compiler, '(if false (do (infinite-loop)). The compiler sees that there is no more macro expansion to be done, so it emits the appropriate code to execute that operation. When we get to run-time, the if happily short-circuits and this ends up being a no-op.

This sort of conditional macro tends to be trivial in implementation, but highlights the difference between an eagerly evaluated argument to a function and a syntactic form passed to a macro for transformation at compile time. And if Clojure lacked a conditional construct, you could happily add it.

Continuing with the comparison between Clojure and C#. C# gained async as a new keyword which required compiler modifications to implement. Clojure got core.async as a library. Anyone could have implemented core.async on their own. Now, Clojure's core.async is CSP-style (similar to Go) and C#'s rewrites your code into a state machine on your behalf, which basically pretties up callback handlers for you. If you prefer that C# async style to CSP, you can introduce that construct in Clojure yourself with something that looks as "native" as core.async. If you want to use CSP in C#, you cannot create any syntax for this and so will never be able to make something feel native as async does.

Or another example would be something like C#'s using construct for IDisposables. If you wanted to implement using in a C# without it, you couldn't. You cannot create new control flow constructs. Something similar with Java AutoClosables is implemented as a macro in Clojure as well (again, if it didn't exist in the language, you can add it just like below). You can implement arbitrary control flow in Lisp macros in a way that would require syntax and compiler modifications in other languages.

    (defmacro with-open
      "bindings => [name init ...]
      Evaluates body in a try expression with names bound to the values
      of the inits, and a finally clause that calls (.close name) on each
      name in reverse order."
      {:added "1.0"}
      [bindings & body]
      (assert-args
         (vector? bindings) "a vector for its binding"
         (even? (count bindings)) "an even number of forms in binding vector")
      (cond
        (= (count bindings) 0) `(do ~@body)
        (symbol? (bindings 0)) `(let ~(subvec bindings 0 2)
                                  (try
                                    (with-open ~(subvec bindings 2) ~@body)
                                    (finally
                                      (. ~(bindings 0) close))))
        :else (throw (IllegalArgumentException.
                       "with-open only allows Symbols in bindings"))))

I hope these examples help to illustrate the differences. If you're looking for more examples, you can take a look at the Clojure source to see what functionality is implemented by macros. https://github.com/clojure/clojure/search?q=defmacro&type=co...

Re: Understanding the Power of Lisp (2020)

#77
post #38

Earlier quoted context omitted.

> So that's how I view Lisp macros - creating code on the fly and executing it. This is an incorrect assumption. In compiled Lisps[0], macros are a compile-time construct, which manipulate the data structures that represent your code[1]. All manipulation occurs at compile-time. In interpreted Lisps, macro evaluation is temporally intertwined with program execution, but the two phases are logically distinct. [0]: Comp…

So Clojure programs are data structures, and you can pass a structure to a macro to rewrite it into something the compiler can evaluate. In this way you can extend the language. You can build your own DSL that solves your particular problem in an organic way that grows as you go along. If I understand that correctly I don't get it, because I would still have to write the macro, which in C# I would write as a function…

As for REPL driven development, it's not about typing code at a REPL. This video provides a pretty good example of working at a REPL in Clojure. Note that with the exception of one doc reference at the beginning, all code is typed directly into a file buffer. The editor provides shortcuts to send various forms to the REPL for evaluation.

https://vimeo.com/230220635

Re: Understanding the Power of Lisp (2020)

#78
post #45

Earlier quoted context omitted.

I follow that up with knowledge of a large project that was started in Lisp (actually Scheme) that had to be converted to C# because the difficulty of finding experienced Scheme developers for the years of maintenance that would be expected was far more than the cost of converting to a language that's more "usable." Still remember all the meetings that generally always included someone complaining "what the $#&^%! we…

I know a startup that hired a Marxist-collective group of programmers. I was told this story years ago, and they were acquired, so I’ll just name the firm - “White Ops”. This story is so absurd that I’m naming the firm in hopes someone can verify the accuracy of this, although I trust the person that told me the tale. The programmers were based in Canada and only wrote in Haskell. The CTO of White Ops had apparently…

You described every enterprise Java project developed by contractors except that it can’t be rewritten because Java is THE enterprise language.

Re: Understanding the Power of Lisp (2020)

#79
post #40
post #37

Earlier quoted context omitted.

Here's a simple example you can't write in most languages. (first-working (get-it-from-the-cache) (get-it-from-the-database) (get-it-from-an-external-api) (compute-it-the-slow-way) default-value) Note each of those functions could throw an exception. You want to ignore it (or you could log it) and move on to the next.

Maybe I am missing something. Here's how I might write something equivalent in Python: for f in GetFromCache, GetFromDatabase, GetFromAPI, Compute: try: return f() except Exception as ex: log(ex) return default_value (You'd probably define a custom exception class for your application that represents the kinds of errors you can tolerate, so you don't end up swallowing programming errors, but you get the idea.)

In ruby your lower level API should support methods with ! syntax that throw and without it that return nil instead:

    def get_from_cache
       get_from_cache!
    rescue NoRecord
       nil
    end
Then you just do this to not only do it all but only do it once lazily on access:

    def record
      @record ||= get_from_cache || get_from_database || get_from_api || compute || default_value
    end
Be slightly better if ruby supported a proper null coalescing operator like C# ?? and ??= or Perl // and //= but as long as you stay way from "false" as a valid value it works fine.
Post reply on HN