Live data from Hacker News

Ooh Ooh My Turn Why Lisp? (2008)

smuglispweeny.blogspot.com

151–160 of 160 posts

Re: Ooh Ooh My Turn Why Lisp? (2008)

#151
post #147

Earlier quoted context omitted.

The reason typing is not a solved problem is this: there are valid programs which can be expressed in an untyped language that cannot be (directly) expressed in a typed one at the moment. For example: (define (foo p?) (if p? 42 "forty-two")) What is the type of `foo`? It could be `bool -> int` or `bool -> string`, depending on the result of `p?`! In an untyped language, this is a perfectly valid program. In a typed l…

The type of FOO in SBCL is: (FUNCTION (T) (VALUES (OR (INTEGER 42 42) (SIMPLE-ARRAY CHARACTER (9))) &OPTIONAL)) SBCL is based on Kaplan & Ullman flow-graph analysis (inherited from CMUCL), not Hindley-Milner. See also http://home.pipeline.com/~hbaker1/TInference.html (Nimble Type Inference, H. Baker). Apparently, Typed Racket's papers have other references about related work, albeit not explicitly those. In OCaml, a…

> The type of FOO in SBCL is:

Is that the type of FOO or the return type of FOO? How are the types or number of arguments specified for function types?

> SBCL is based on Kaplan & Ullman flow-graph analysis (inherited from CMUCL), not Hindley-Milner. See also http://home.pipeline.com/~hbaker1/TInference.html (Nimble Type Inference, H. Baker). Apparently, Typed Racket's papers have other references about related work, albeit not explicitly those.

I've read some of Henry Baker's other work, and the man is brilliant. I wish he were more well-known.

This is no exception, however it seems that this inferencer is useful primarily for enabling compiler optimizations rather than for performing type checking. Is that correct?

What I mean is, say you want to pass the result from FOO above into another function BAR which expects a numeric argument. It seems to me that BAR would need to have an argument of type (OR (INTEGER 42 42) (SIMPLE-ARRAY CHARACTER (9))) in order to guarantee type safety. Otherwise, the type checker couldn't reject a program that passes a string result of FOO as a numeric argument of BAR. Am I understanding this correctly?

> In OCaml, a function which throws an exception has not a different type than one which doesn't. So you can get runtime errors even when you typecheck.

Right, but I'm talking specifically about run-time type errors, i.e., errors that occur because some value was passed to an operation that does not handle values of such type. What I meant by my statement is that even with type checking, the programmer must handle all cases of a sum type when dealing with it in order for the program to be type safe (in the sense that not only does it not "go wrong" in the Milner sense, but also that it does not suffer a type error at run-time).

Re: Ooh Ooh My Turn Why Lisp? (2008)

#152

Earlier quoted context omitted.

The reason typing is not a solved problem is this: there are valid programs which can be expressed in an untyped language that cannot be (directly) expressed in a typed one at the moment. For example: (define (foo p?) (if p? 42 "forty-two")) What is the type of `foo`? It could be `bool -> int` or `bool -> string`, depending on the result of `p?`! In an untyped language, this is a perfectly valid program. In a typed l…

This is quite a nice write-up, thank you. Regarding this point: >because what we'd really like is a way to specify that the type of `foo` depends on the result of `p?` Is this situation not resolved by allowing p? to not be a bool necessarily but also a sum type that informs foo which case of a different sum type to return? Anyway, I like your footnote [1] which, for me, is what allows a typed program to actually be…

> Is this situation not resolved by allowing p? to not be a bool necessarily but also a sum type that informs foo which case of a different sum type to return?

In the example I provided, the `bool -> (int * string) either` function is essentially a map from a `bool` to an `(int * string) either`.

From a type-theoretic perspective, bool (which has two nullary type constructors, `true` and `false`) can be expressed as the sum of two units: `1 + 1` (or equivalently, `() + ()` or `unit + unit`) = `2`. If I understand your question correctly, then you're asking if we could just use some other sum type to express the branch to take, like: `type if-branch = consequent | alternative;`, and the answer is that such a type is precisely isomorphic to a Boolean -- because it consists of two nullary constructors, it can also be expressed as the sum of two units, `1 + 1` = `2`.

To answer your question more directly, `p?` already is a sum type that informs `foo` which case to return: `type bool = true | false;`.

> Anyway, I like your footnote [1] which, for me, is what allows a typed program to actually be more expressive, in the sense that you can literally express the types in your head as you code, which is difficult or simply not provided in a dynamic language, despite that most developers will be "thinking in types" regardless of typing of the language.

"Expressive" can mean any of quite a few different things, so I wanted to be specific which I meant. In this case, I mean that untyped languages are more expressive because they're capable of expressing a greater number of programs than their typed counterparts. The ultimate goal of type theory is to eliminate that gap, but short of some miracle revelation, we've resigned to gradually filling it in.

Note that this isn't the same as saying you can express it more elegantly or less verbosely in an untyped language -- there are some programs that are strictly not expressible with current type systems. That's on purpose! The point of the type system is to reject programs which have bugs caused by type errors, so we make those programs inexpressible on purpose, so it's usually a good thing to be less expressive in that way. The problem is that our type systems sometimes reject bug-free programs that actually would run totally fine simply because it wasn't able to prove that such was the case. Again, type theorists are working to improve the situation.

What it comes down to is a question of what is being expressed. Taking programs as descriptions of processes, at the lowest level a language allows us to express a run-time process. Going up one level to the level of types, we can express information about the program itself. Thus, in a sense, by using a typed language we trade in the expressiveness of some processes for the expressiveness of some metaprocesses.

Re: Ooh Ooh My Turn Why Lisp? (2008)

#154
post #147

Earlier quoted context omitted.

The type of FOO in SBCL is: (FUNCTION (T) (VALUES (OR (INTEGER 42 42) (SIMPLE-ARRAY CHARACTER (9))) &OPTIONAL)) SBCL is based on Kaplan & Ullman flow-graph analysis (inherited from CMUCL), not Hindley-Milner. See also http://home.pipeline.com/~hbaker1/TInference.html (Nimble Type Inference, H. Baker). Apparently, Typed Racket's papers have other references about related work, albeit not explicitly those. In OCaml, a…

> The type of FOO in SBCL is: Is that the type of FOO or the return type of FOO? How are the types or number of arguments specified for function types? > SBCL is based on Kaplan & Ullman flow-graph analysis (inherited from CMUCL), not Hindley-Milner. See also http://home.pipeline.com/~hbaker1/TInference.html (Nimble Type Inference, H. Baker). Apparently, Typed Racket's papers have other references about related work,…

> Is that the type of FOO or the return type of FOO?

It is the type of FOO, it reads as: a function which takes one argument of any type (T) and returns exactly one value, which is either 42 or a string of length 9.

Another example:

    (defun xyz (x y z)
      (declare (type fixnum x)
               (type float y)
               (type (vector (unsigned-byte 8) 1024) z))
      (aref z (+ x (round y))))

    (FUNCTION
     (FIXNUM FLOAT (VECTOR (UNSIGNED-BYTE 8) 1024))
     (VALUES (UNSIGNED-BYTE 8) &OPTIONAL))
Here there are three arguments, the third one being a vector of bytes of length 1024. Note that the return values was inferred from the inputs.

> This is no exception, however it seems that this inferencer is useful primarily for enabling compiler optimizations rather than for performing type checking. Is that correct?

It is a mix of both, really.

CL is primarily designed to be dynamic. Static analyses are used to optimize code and prevent classes of errors if they can be detected in advance. If you define detecting a type error as a positive test, then SBCL allows to have false negatives. That happens in cases where the expected and actual types overlap: there might be an error, or not, so the actual check is delegated at runtime.

Another thing is that with global functions (defun), it seems that there is some widening happening, for the return type in particular, so that (OR INTEGER STRING) is treated as T. This does not happen with inline or local functions. Note also that global functions can be called from anywhere, be redefined (except standard ones) and they are generally responsible for checking their arguments, except when you explicitely turn the safety knob down and add type declarations.

So let's say that XYZ above is declared to be inlined, and we use it as follows:

    (defun use-xyz-1 (x y z)
      (declare (type positive-float y)
               (type (integer 0 3000) x))
      (xyz x y z))
The above is compiled without problems, even though you could give values which would make an out-of-bounds access. However, you will surely agree that there are theoretical limits to static type checking, so it is expected that not all expressions can be typed in CL as precisely as you could wish (of course, going up the lattice, functions accept type T arguments). However, when you change type declarations so that the intersection of expected/actual types is empty:

    (defun use-xyz-2 (x y z)
      (declare (type positive-float y)
               (type (integer 2000 3000) x))
      (xyz x y z))
... the compiler warns you that:

    ;; Derived type (INTEGER 2000 4611686018427387900) is not a suitable
    ;; index for (VECTOR (UNSIGNED-BYTE 8) 1024)


The way SBCL treats declaration is that they are used as assertions (except for return types in global declarations, see manual). So what does it mean to treat declaration as assertions? Here is FOO:

    (defun foo (float)
      (make-string (abs (ceiling float)) :initial-element #\#))
It makes a new string made of N times character "#", where N is computed from the float input. Then, we call FOO from BAR:

    (defun bar (x)
      (foo x)
      (typecase x
        (float   0)
        (integer 1)
        (t       2)))
The unique value returned by BAR is of type `(integer 0 0)`, because knowing that `(foo x)` succeeds allows us to conclude that X was indeed a FLOAT, and thus the TYPECASE expression necessarily returns zero. Declarations, assertions, etc. can be used by the compiler. Note that the equivalent (w.r.t. return value) function below has a different type:

    (defun bar (x)
      (prog1
          (typecase x
            (float   0)
            (integer 1)
            (t       2))
        (foo x)))
This time, the return type is (MOD 3), i.e. the set {0,1,2}, even though propagation could be applied backward. However, backward propagation seems to pose problem w.r.t. the CL standard, at least that's what is said here (a good reference, by the way):

https://www.pvk.ca/Blog/2013/04/13/starting-to-hack-on-sbcl/

Static typing in SBCL gives something I did not yet witness in other languages. I defined a state machine with local functions, roughly as follows:

    (defun sm ()
      (let ((state))
        (labels ((a () (setf state #'b))
                 (b () (setf state #'c))
                 (c () (if (plusp (random 2))
                           (setf state #'d)
                           (setf state #'b)))
                 (d () (setf state #'a))
                 (e () (return-from sm)))
          (loop (funcall state)))))
So the local variable "state" holds current function. And so, this compiles (and the actual, longer code did so as well) with a note saying "deleting unused function (LABELS E :IN SM)", because "state" is known to never reach E. By the way, function SM never returns normally, as explained by the NIL return type. That was a useful and unexpected thing to notice.

Re: Ooh Ooh My Turn Why Lisp? (2008)

#155
post #154

Earlier quoted context omitted.

> The type of FOO in SBCL is: Is that the type of FOO or the return type of FOO? How are the types or number of arguments specified for function types? > SBCL is based on Kaplan & Ullman flow-graph analysis (inherited from CMUCL), not Hindley-Milner. See also http://home.pipeline.com/~hbaker1/TInference.html (Nimble Type Inference, H. Baker). Apparently, Typed Racket's papers have other references about related work,…

> Is that the type of FOO or the return type of FOO? It is the type of FOO, it reads as: a function which takes one argument of any type (T) and returns exactly one value, which is either 42 or a string of length 9 . Another example: (defun xyz (x y z) (declare (type fixnum x) (type float y) (type (vector (unsigned-byte 8) 1024) z)) (aref z (+ x (round y)))) (FUNCTION (FIXNUM FLOAT (VECTOR (UNSIGNED-BYTE 8) 1024)) (V…

Wow! That's a much more detailed response than I'd expected. Thank you! I wish I could upvote you twice! :)

I've used Common Lisp and even SBCL quite a bit, and had no idea SBCL could do some of this stuff. That last example is particularly impressive, given how hairy control-flow analysis can get when higher-order functions are involved.

Re: Ooh Ooh My Turn Why Lisp? (2008)

#156
post #142

Earlier quoted context omitted.

Naughty Dog used a custom Scheme system for their early games. That one was written in Allegro Common Lisp. When they were bought by Sony, they abandoned their tools and moved on to C++, hoping to share code with the rest of Sony game developers. Didn't work out as wished. They brought Scheme back into their game development, but differently. For example: http://www.slideshare.net/naughty_dog/statebased-scripting-i..…

i have seen that presentation, but i am fairly certain i recall from an uncharted 4 chat with a naughty dog developer that they no longer use that system once they moved to the ps4. this is in large part, i believe, due to their being a technology developer for multiple sony studios.

http://n4g.com/news/1268353/naughty-dog-will-use-existing-un...

Re: Ooh Ooh My Turn Why Lisp? (2008)

#157
post #122
post #62

Earlier quoted context omitted.

Well, I'm still using Common Lisp. If there was anything better, I'd have probably switched.

Curious, what are you using it for? Would you say it is well suited for writing operating system helper tools (working with files, pipes, kernel api, calling other programs)? I like bash expressiveness a lot but sometimes I feel I could use a more powerful scripting language.

SCSH, a scheme library/derivitive built on scheme48, works well here. It's got functions and macros for doing all those things. It works quite well, but is sadly seemingly barely maintained. However, its features have been ported to just about every scheme out there, to some extent.

Re: Ooh Ooh My Turn Why Lisp? (2008)

#158

Okay, I've spent a lot of time writing Common Lisp and even more time reading about Common Lisp, and to be honest, I'm just a little tired of the cult around it. People who know it well gloat about how great Common Lisp is, which just so happens to make them look great too. And people who don't know Common Lisp all talk about the little Common Lisp they know, because they don't want to seem like they aren't in on it,…

> So the very first thing you do with your macro powers, and pretty much the only useful thing you can do, is break s-expressions. Once the first macro is written you can no longer assume that inputs to macros will be s-expressions with the function at the beginning and arguments following. Every future macro must account for every previous macro. The more you use the capability to manipulate code as data gracefully,…

>You should check out Racket's macro system. It's a lot more sophisticated than Common Lisp's. Common Lisp macros are to C (e.g., gensym is a macro-level malloc, you need to manually destructure S-expressions, etc.) as Racket macros are to ML and Haskell (syntax objects are aware of which variables are in scope, so automatic fresh name generation is possible; user-defined syntax classes and patterns let you process arbitrarily complicated structures in a sane way, etc.). If you like the idea of metaprogramming, but `defmacro` left you with a bad taste in the mouth, Racket is totally the language for you.

Better yet, check out some other schemes. ir, er, and sc macros has the raw procedurual power of defmacro, but with the hygene and safety of syntax-case/syntax-rules, without the declarative syntax of syntax-rules, and the disadvantages of syntax-case (stupidly complex, breaking the standard macro abstraction with syntax/datum distinctions, etc.).

Given, syntax-case has some advantages, but I don't think it carries its own weight from a programmer's perspective.

Re: Ooh Ooh My Turn Why Lisp? (2008)

#159

Okay, I've spent a lot of time writing Common Lisp and even more time reading about Common Lisp, and to be honest, I'm just a little tired of the cult around it. People who know it well gloat about how great Common Lisp is, which just so happens to make them look great too. And people who don't know Common Lisp all talk about the little Common Lisp they know, because they don't want to seem like they aren't in on it,…

>I'm just a little tired of the cult around it

Me too, but you seem to not be entirely clued in to how macros work. So there's that.

>So the very first thing you do with your macro powers, and pretty much the only useful thing you can do, is break s-expressions. Once the first macro is written you can no longer assume that inputs to macros will be s-expressions with the function at the beginning and arguments following. Every future macro must account for every previous macro. The more you use the capability to manipulate code as data gracefully, the less graceful it becomes.

While perhaps more true in Common Lisp than in scheme, that's still fairly untrue. IIRC, macro expansion is innermost-first, so macros don't have to worry about tripping over each other, but even if that's not true, you're still wrong. Macros are sexprs, just like anything else: they break the semantic rules of lisp, NOT the syntactic rules of lisp. Since macros operate primarily on a syntactic level (their job is to sugar over Lisp), and when they operate on a semantic level, they're being used to write DSLs, which aren't lisp, the problem you describe is rare to nonexistant.

>macros make your program harder to reason about.

Yes, but not for the reasons you think. Macros make your program harder to reason about for the same reason functions do: they're an abstraction. The higher level the abstraction you're using is, the harder it is to reason about your code, because by definition, you can't see what it's doing.

>Common Lisp programmers spend a ton of time talking about how to write macros so that they’re not going to come back and bite you in the butt when they get used in an unexpected situation. And the reason is, nobody really knows how to do it.

However, us schemers (NOT racketeers, Racket using syntax-case instead, which while similar seeming, is an entirely different ball game, although it also fixes this) have had hygenic defmacro-style lowlevel macros for years now, and that fixes the worst of it.

>Python, for example has all those things.

...Not actually true. Python's functions aren't first-class. They try very hard to be, but they're not. Which is why idiomatic Python doesn't use a lot of higher-order functions. Ruby does better, but of the trinity of popular high-level scripting languages, only JS has true first-class functions. In fact, for a while, JS had no distinct syntax for function declaration.

>People on this thread are claiming, “Learning Common Lisp turns you into a better programmer”. But I tend to think that learning functional programming is the part that people are referring to

It's not what I'd refer to. FP makes you a better programmer to, but learn than in Haskell or ML, or even Scheme, though it's not as good for that purpose. Learning lisp exposes you to code-as-data in a very viceral way, and can help you understand a variety of programming concepts, but more importantly, it exposes you to new, different ways of programming, which always makes you a better programmer.

>And sure, there are other features that only Common Lisp has, but nobody is talking about those. Restarts? I’d love to see more people experimenting with those. Then again, Erlang has a way better threading model than anything else and much more sophisticated pattern matching. Scheme has call/cc. Standard ML has a powerful type system. Haskell has functional purity. Prolog has unification. A great many of these are more interesting than restarts.

I agree, but restarts are the one I want most. It might seem like we schemers got a better deal with call/cc, and I love call/cc, but sometimes I look at restarts, and wish we had gone the other way (call/cc and functioning restarts are effectively mututally exclusive). Trust me, call/cc looks cool, but so does self-rewriting asm. Both come in handy from time to time, but you rarely want either in production (and especially not if your scheme doesn't have cheney on the mta compilation, because then call/cc is criminally slow)

>I just don’t really think it’s the be-all and end-all of programming languages any more, and I’m kind of tired of the cult that has formed around it.

That's true. No language is, or ever can be the be-all, end-all. Rust and FP taught me that: some paradigms and ideas require resrictions, others require freedom. There's always something new to learn. And the cult of common lisp is not one you need to join. In fact, if you're a good lisper, you can't really be part of it.

Re: Ooh Ooh My Turn Why Lisp? (2008)

#160
post #103

Earlier quoted context omitted.

With respect to Clojure, you don't write macros, but many of the really interesting language features within Clojure are built upon macros. (core.async and core.match come to mind). It would be difficult to add them without macros. I do actually think of my code as a tree/graph structure. Not just in lisps but in other languages as well, I'm always manipulating a stack of trees and thinking about how the data flows t…

> many of the really interesting language features within Clojure are built upon macros That's true with any lisp. But that doesn't mean the the ability to write macros is a requirement for most development work on a daily basis, and homoiconicity is often touted as amazing specifically because of its help in writing macros. Therefore, since writing macros is not super common or necessary most of the time, I would sa…

code as data, however, does become an important concept in becoming really good at Perl or Python (and I would assume Ruby etc), and understanding common pitfalls here also helps one spot, for example, LSP problems and provide better solutions (because most LSP problems can be solved by replacing a mutator with something that returns a new object).

In Javascript, also, effectively you have to work with code as data.

So this isn't just about writing macros. There are a lot of areas where this does touch things.

Post reply on HN