Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transducers are coming to Clojure

blog.cognitect.com

Post reply on HN
Print view View original

71–80 of 103 posts

Re: Transducers are coming to Clojure

#71

Post by tel »

tel wrote: post #70
pron wrote: post #57

I think that a good way to understand transducers is to look at their implementation (shortened a bit). Here it is for map: ([f] (fn [f1] (fn ([result input] (f1 result (f input))) ([result input & inputs] (f1 result (apply f input inputs)))))) filter: ([pred] (fn [f1] (fn ([result input] (if (pred input) (f1 result input) result))))) And it gets more interesting with take: ([n] (fn [f1] (let [na (atom n)] (fn ([resu…

Interesting. Continuing to try to analyze these in Haskell, I think this is a direct translation: -- z is just there to not clobber some standard prelude names type Red r a = r -> a -> r zmap :: (b -> a) -> Red r a -> Red r b zmap f f1 result input = f1 result (f input) zfilt :: (a -> Bool) -> Red r a -> Red r a zfilt p f1 result input = if p input then f1 result input else result ztake :: Int -> Red r a -> (r -> a -…

Also, you can get rid of the sentinel value by packing along a termination continuation value as well:

    type Red r a = (r -> a -> r, r)

    zmap :: (b -> a) -> Red r a -> Red r b
    zmap f (f1, z1) = (\result input -> f1 result (f input), z1)

    zfilt :: (a -> Bool) -> Red r a -> Red r a
    zfilt p (f1, z1) = (\result input -> if p input then f1 result input else result, z1)

    ztake :: Int -> Red r a -> Red r a
    ztake n (f1, z1) = (run n, z1) where
      run n result input =
        let n'     = n - 1
            result = if n >= 0 then f1 result input else result
        in if n' == 0 then z1 else result
This has the theoretical niceness of having `Red r a` just be the signature functor for linked lists.

Re: Transducers are coming to Clojure

#72

Post by tel »

tel wrote: post #71
tel wrote: post #70

Earlier quoted context omitted.

Interesting. Continuing to try to analyze these in Haskell, I think this is a direct translation: -- z is just there to not clobber some standard prelude names type Red r a = r -> a -> r zmap :: (b -> a) -> Red r a -> Red r b zmap f f1 result input = f1 result (f input) zfilt :: (a -> Bool) -> Red r a -> Red r a zfilt p f1 result input = if p input then f1 result input else result ztake :: Int -> Red r a -> (r -> a -…

Also, you can get rid of the sentinel value by packing along a termination continuation value as well: type Red r a = (r -> a -> r, r) zmap :: (b -> a) -> Red r a -> Red r b zmap f (f1, z1) = (\result input -> f1 result (f input), z1) zfilt :: (a -> Bool) -> Red r a -> Red r a zfilt p (f1, z1) = (\result input -> if p input then f1 result input else result, z1) ztake :: Int -> Red r a -> Red r a ztake n (f1, z1) = (r…

I'm also assuming transduce and sequence are something like this

    type RT r a b = Red r a -> Red r b

    sequence :: RT [a] a b -> [b] -> [a]
    sequence xform bs = transduce xform (flip (:)) [] bs

    transduce :: RT r a b -> (r -> a -> r) -> r -> [b] -> r
    transduce xform cons nil =
      let (cons', nil') = xform (cons, nil)
      in foldr (flip cons') nil'
but I can't find the actual source yet. Again, there's a weird flipping going on induced by the contravariance. It makes me want to define RT-composition backwards.

Re: Transducers are coming to Clojure

#73

Post by ww520 »

defen wrote: post #68
ww520 wrote: post #66

Earlier quoted context omitted.

Sorry I was editing before I saw your comment. Note: There was a myDoublingTransducer before. On a separate note, a reducer returns nothing doesn't make sense. A reducer might not add the current value to the memo (filter) but it always returns some memo for the next step. Also a reducer would not return multiple values. It returns just one value, the memo. If there are multiple values produced at a step, they would…

> A reducer might not add the current value to the memo (filter) but it always returns some memo for the next step. I think the idea is that your reducer function shouldn't have to care about steps or memos. e.g. I should be able to do filter(function(x) { return x and get back some thing that I can then pass in to map, or to reduce, or compose with other things .

Hmm, isn't that just the filter predicate function? Not the reducer itself? Looks like the filter() function constructs a reducer using the predicate. The new reducer adds the filtering functionality with the predicate.

Looking at the code for filter (copied from above)

filter:

    ([pred]
    (fn [f1]
The reducer still behaves as reducer, returning one result.

A Javascript version would be:

    function filterTransducer(innerReducer, pred) {
        return function(memo, value) {
            if (pred(value))  
                return innerReducer(memo, value)
            return memo;
        }
    }

Re: Transducers are coming to Clojure

#75

Post by oafitupa »

As someone who tried Clojure and failed, serious question: Does anyone actually use all these crazy features/patterns that keep getting added/discovered and talked about?

I ask because even though I can imagine someone smart mastering these things and programming faster, I can't imagine a second person being able to understand his code, maintain it, and generally be productive. I imagine the second person losing a lot of time trying to understand what is going on, or even thinking he understood but in reality he didn't and messing things up.

So how do you even form a Clojure team?

Re: Transducers are coming to Clojure

#76

Post by calibraxis »

oafitupa wrote: post #75

As someone who tried Clojure and failed, serious question: Does anyone actually use all these crazy features/patterns that keep getting added/discovered and talked about? I ask because even though I can imagine someone smart mastering these things and programming faster, I can't imagine a second person being able to understand his code, maintain it, and generally be productive. I imagine the second person losing a lo…

This sentence is important: "There will be more explanations, tutorials and derivations to follow, here and elsewhere." If a team ever adopts transducers, it'll generally be after a lot of explanations exist and has been made easy to understand.

Clojure teams can defend themselves against exotic features like any other team: code review, conventions, etc. If I pull in some weird new feature, others had better be ok with it.

(Obviously, what I'm saying doesn't apply to every team in the world; presumably there's a spectrum of ways to deal, or fail to deal, with this problem.)

In my view, Clojure teams are formed via a political process, like any other programming language adoption.

Re: Transducers are coming to Clojure

#77

Post by e12e »

nohat00 wrote: post #39
prospero wrote: post #36

Earlier quoted context omitted.

It means that reducers were some macro sugar atop the underlying mechanism described in the post. See https://github.com/clojure/clojure/blob/master/src/clj/cloju...

... so 'macrology' means "use of macros" rather than "long and tedious talk without much substance"? Is this usage specific to Clojure, or to all languages that have macros? It seems like a pretty bizarre repurposing of a word that already has a very different meaning, and I wonder how widespread it is. Are we too late to stop it?

Well, it's the use of macros to avoid having to write code that has the characteristics of macrology. If nothing else, it's sort of strangely recursive, like perhaps many "complex macros" are...

Re: Transducers are coming to Clojure

#78

Post by JackMorgan »

oafitupa wrote: post #75

As someone who tried Clojure and failed, serious question: Does anyone actually use all these crazy features/patterns that keep getting added/discovered and talked about? I ask because even though I can imagine someone smart mastering these things and programming faster, I can't imagine a second person being able to understand his code, maintain it, and generally be productive. I imagine the second person losing a lo…

It's a mindset change.

Other than macros, which fundamentally alter the flow of code in a very non-standard way, the rest of these "crazy patterns and features" in Clojure are just like the crazy libraries used by typical Java/Ruby/C# developers, only thousands of times simpler. If I came to you and said, "does anyone even use these tens of thousands of libraries in Maven? How do other developers work on this afterwards, they'd have to learn all these new APIs!" I'd likely get a response like, "they'd just be expected to", with a chuckle. The mindset I've seen a lot is that language features are "too hard" to learn and should be avoided, but library complexity is beyond reproach and is rarely questioned.

Clojure the language takes the approach that it's better to provide a dozen simple but incredibly composable tools that can easily replace a lot of duplicated boilerplate in other languages. Like these transducers, in Java/C# they'd likely be one of the design patterns that needs a whole set of classes to make what he shows in a few characters. Would you rather learn to write and read maintain a handful of classes, or just learn what those few characters mean? I don't get paid by the line, and any abstraction built into the language like that is a few more classes I don't have to maintain and possibly implement incorrectly in some subtle way.

Like I said, it's just a mindset change. I know a company that only uses Clojure, and they hire a lot of entry level developers who haven't yet gotten stuck in a mindset that "languages are too hard; libraries are easy". They have great success with that, and their entry level guys are up to speed much faster than those in my shop, using .NET, where they have to learn a new language AND a mountain of boiler plate code using dozens of libraries and frameworks.

Re: Transducers are coming to Clojure

#79

Post by oafitupa »

JackMorgan wrote: post #78
oafitupa wrote: post #75

As someone who tried Clojure and failed, serious question: Does anyone actually use all these crazy features/patterns that keep getting added/discovered and talked about? I ask because even though I can imagine someone smart mastering these things and programming faster, I can't imagine a second person being able to understand his code, maintain it, and generally be productive. I imagine the second person losing a lo…

It's a mindset change. Other than macros, which fundamentally alter the flow of code in a very non-standard way, the rest of these "crazy patterns and features" in Clojure are just like the crazy libraries used by typical Java/Ruby/C# developers, only thousands of times simpler. If I came to you and said, "does anyone even use these tens of thousands of libraries in Maven? How do other developers work on this afterwa…

> Would you rather learn to write and read maintain a handful of classes

I appreciate your answer, but that's not what I do with libraries. I don't have to read their internals, let alone write them.

Re: Transducers are coming to Clojure

#80

Post by JackMorgan »

oafitupa wrote: post #79
JackMorgan wrote: post #78

Earlier quoted context omitted.

It's a mindset change. Other than macros, which fundamentally alter the flow of code in a very non-standard way, the rest of these "crazy patterns and features" in Clojure are just like the crazy libraries used by typical Java/Ruby/C# developers, only thousands of times simpler. If I came to you and said, "does anyone even use these tens of thousands of libraries in Maven? How do other developers work on this afterwa…

> Would you rather learn to write and read maintain a handful of classes I appreciate your answer, but that's not what I do with libraries. I don't have to read their internals, let alone write them.

I may have communicated that badly. A pattern like transducers is one that everyone needs. Everyone either uses it built into the language or writes it out long form. Design patterns are a way of giving you instructions for writing it out long form. So, you either write it yourself or use it built in.

Some people do not want to learn language tools that will reduce such boilerplate, but those people often are (ironically) happy to learn lots of libraries. Libraries by definition can't remove any more boilerplate than you can, they are just as limited by the language, so often times just save the user some writing, rather than reducing (haha) it across a whole codebase like a built in feature can.

Post reply on HN