Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Schema for Clojure(Script) Data Shape Declaration and Validation

blog.getprismatic.com

Post reply on HN
Print view View original

41–50 of 58 posts

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#41

Post by jamii »

adamgravitis wrote: post #28
jamii wrote: post #27

I've been working on a similar idea for replacing types. If you can specify how to take apart a type, name the parts and put them back together you can then get validators, pattern matching, recursive-descent parsing with backtracking, generic traversals, lenses and generators more or less for free. By using simple data-structures to describe the types and compiling them to clojure at the call site you can have first…

Very cool. Also check out the implementation of encapsulation in John Shutt's Kernel Lisp: http://web.cs.wpi.edu/~jshutt/kernel.html

Interesting. I've seen his fexpr work before on LTU but never really looked at it closely. I suppose the only way to go is to read the whole thesis?

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#42

Post by ambrosebs »

core.typed author here.

It might be easy to consider Schema as "competitor" to core.typed. In fact, it's the opposite: once they play nicely together they will form a formidable bug-fighting, finely-documenting team. :)

core.typed has accurate compile time checking, and Schema gives an expressive contracts interface for runtime checking.

Once they understand each other, you can start pushing and pulling between static and dynamic checking by using both libraries to their strengths.

Currently, core.typed requires all vars have top level annotations. This is partly because there is no way to recover type information once inside a function body. However, if we have an entire namespace using Schema liberally, we can use schemas to recover information!

This means we can lean on schemas for most annotations, and rely on core.typed to catch arity mismatches, bad arguments to clojure.core functions, potential null-pointer exceptions and many more nasties at compile time.

Then you might start adding static annotations or removing schemas, depending on the kind of code your dealing with. You might do some "static" debugging to ask whether a schema is needed to prevent a type error. core.typed would also let you know when your contracts are insufficient to rule out type errors. Really, you're free to use both tools as you'd like.

Schema looks very nice, thanks for open sourcing it Prismatic folks!

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#43

Post by john2x »

danneu wrote: post #23
lkrubner wrote: post #21

Earlier quoted context omitted.

I do this a lot: "Often I stoop to the level of caching my findings in a docstring/comment." I am not ashamed of this. When my co-workers have to read what I wrote, this kind of information makes clear what the function does. If I don't do this, then they will eventually figure it out by running it at the REPL and examining the input and the output, but if I document that in the docstring, I have saved them a lot of…

Yeah, you're right. I would love if every Clojure function came with an example input and output. What I meant to suggest is that my docstring example is something that could be expressed in code.

You would think that being most functions being pure, that these types of docsctrings would be more common (or standard, even).

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#44

Post by JackMorgan »

What timing! I have been making this exact thing myself.

Would you be interested in pull requests making your api simpler? Maybe allow parameters to not have to have shapes? Perhaps allowing a syntax that allows a simple way to shapes in the meta alongside the ability to change the signature for people who don't want to couple too tightly to the library? When I showed mine to my local user group last month, that was one of their biggest requests (that I was in the middle of on mine but would be happy to attempt in yours).

Check my (simpler and more immature) library here https://github.com/steveshogren/deft

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#45

Post by JackMorgan »

JackMorgan wrote: post #44

What timing! I have been making this exact thing myself. Would you be interested in pull requests making your api simpler? Maybe allow parameters to not have to have shapes? Perhaps allowing a syntax that allows a simple way to shapes in the meta alongside the ability to change the signature for people who don't want to couple too tightly to the library? When I showed mine to my local user group last month, that was…

Also, what about having your defn macro generate a second function like foo-t that just calls with-fn-validation on foo, to clean up that extra call? Would you be interested in such a pull request?

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#46

Post by JackMorgan »

JackMorgan wrote: post #44

What timing! I have been making this exact thing myself. Would you be interested in pull requests making your api simpler? Maybe allow parameters to not have to have shapes? Perhaps allowing a syntax that allows a simple way to shapes in the meta alongside the ability to change the signature for people who don't want to couple too tightly to the library? When I showed mine to my local user group last month, that was…

I really like that your defn macro generates a real function, good call on that!

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#47

Post by ambrosebs »

danneu wrote: post #19

The opening paragraphs really resonated with me including that code example that can be replaced with any function I wrote yesterday. A lot of my time in Clojure is spent re-reading and repl-evaluating my function implementations just to remind myself what its symbols look like. Even code I wrote an hour ago. Often I stoop to the level of caching my findings in a docstring/comment. (defn alpha-map-sum-combining-thing…

May I suggest stealing some of core.typed's syntax for your docstrings, especially for polymorphic functions?

It scales very well, from simple types like clojure.core/symbol:

https://github.com/clojure/core.typed/blob/57da1175037dfd61c...

to polymorphic types like clojure.core/repeat:

https://github.com/clojure/core.typed/blob/57da1175037dfd61c...

to succinct representations of complicated polymorphic types like clojure.core/fnil

https://github.com/clojure/core.typed/blob/57da1175037dfd61c...

and I'll spare you the really insane annotations.

Of course you can pick and choose what's to your liking.

Here's a brief tutorial on type syntax: https://github.com/clojure/core.typed/wiki/Types

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#49

Post by w01fe »

ambrosebs wrote: post #38
w01fe wrote: post #36

Earlier quoted context omitted.

I think there are a few benefits. First and foremost, schemas are simple , minimal, easy to read and write, and gracefully extend Clojure's existing type hints. This means that (in my biased opinion) they are significantly better for documentation, which was the primary motivation for developing them. Second, schemas are data, so it's easy to do more with them beyond documentation. Runtime data validation is one such…

Author of core.typed here. I will challenge that schemas are "significantly" better for documentation than core.typed types. With unions, intersections, heterogeneous maps, parameterised classes, recursive types we can be very expressive. Here's some examples of the syntax for types: https://github.com/clojure/core.typed/wiki/Types And some declarative types in action: https://github.com/frenchy64/core.typed-example/…

Sorry, that didn't come across the way I wanted -- it certainly wasn't meant as a dig on the expressiveness or power of core.typed, which is a project I'm really excited about.

The main driver for Schema was to make annotating function inputs and outputs as simple and readable as possible. Personally, I find annotations directly on the function arguments easier to parse than separate function type declarations, but I suppose that's a matter of preference.

Re: Schema for Clojure(Script) Data Shape Declaration and Validation

#50

Post by urlwolf »

danneu wrote: post #23
lkrubner wrote: post #21

Earlier quoted context omitted.

I do this a lot: "Often I stoop to the level of caching my findings in a docstring/comment." I am not ashamed of this. When my co-workers have to read what I wrote, this kind of information makes clear what the function does. If I don't do this, then they will eventually figure it out by running it at the REPL and examining the input and the output, but if I document that in the docstring, I have saved them a lot of…

Yeah, you're right. I would love if every Clojure function came with an example input and output. What I meant to suggest is that my docstring example is something that could be expressed in code.

This is exactly what R does. It's a huge timesaver. Best. docs. ever.
Post reply on HN