Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple apply/filter/reduce package in Go

github.com

Post reply on HN
Print view View original

61–68 of 68 posts

Re: Simple apply/filter/reduce package in Go

#61

Post by adrusi »

drcode wrote: post #40
adrusi wrote: post #22

Earlier quoted context omitted.

That's not why clojure has transducers. You can use a mapping transducer to express map, but it's actually going to have slightly worse performance than regular map. There's nothing inherently slow about higher order collection functions, rust has them and they compile to the same machine code as the equivalent loop construct. Its just that most languages implement them on the wrong data structures. Functional langua…

> You can use a mapping transducer to express map, but it's actually going to have slightly worse performance than regular map. That's a strawman argument- Neither OP nor my comment made any mention of "map". This is a discussion about reduction and filtering. Yes, using transducers in a way that is silly like you suggest will produce disappointing results.

Bad example, but my point holds.

Transducers aren't performance improvers, they're an abstraction over the process of iteration. They actually degrade performance in the absence of a particularly smart inlining compiler.

Re: Simple apply/filter/reduce package in Go

#62

Post by drcode »

adrusi wrote: post #61
drcode wrote: post #40

Earlier quoted context omitted.

> You can use a mapping transducer to express map, but it's actually going to have slightly worse performance than regular map. That's a strawman argument- Neither OP nor my comment made any mention of "map". This is a discussion about reduction and filtering. Yes, using transducers in a way that is silly like you suggest will produce disappointing results.

Bad example, but my point holds. Transducers aren't performance improvers, they're an abstraction over the process of iteration. They actually degrade performance in the absence of a particularly smart inlining compiler.

> They actually degrade performance in the absence of a particularly smart inlining compiler.

So it sounds like you're saying that iterators in Rust are faster than transducer-based filter/reduce.

I think it's clear that my original point was that transducer-based filter/reduce is faster than lazy-list based filter/reduce, and comparable in performance to a for loop. (in that lazy-list operations are MUCH MUCH slower in most cases for filter reduce operations than the latter two approaches)

Re: Simple apply/filter/reduce package in Go

#63

Post by pcwalton »

bunderbunder wrote: post #60
pcwalton wrote: post #49

Earlier quoted context omitted.

> .NET iterators are objects that implement a specific interface, and iterating over them involves two function calls per element - one to move to the next item, and one to retrieve the current item. Any optimizing compiler will inline those functions. "Avoid function calls for performance" hasn't been relevant optimization advice for at least a decade.

A good optimizer can inline those functions, but that doesn't necessarily mean it will. .NET JIT is conservative in what it decides to inline. There's even a special hint you can use to ask it to be more aggressive about inlining a method. You can't necessarily apply that hint to every function, though. Anonymous ones, for example.

> A good optimizer can inline those functions, but that doesn't necessarily mean it will.

If the optimizer isn't choosing to inline where it is a performance win, it's not a good optimizer.

Especially for trivial functions like Current() and MoveNext(), which are pretty much the textbook example of functions where inlining really helps (not so much because of the function call overhead, but because it enables so many intraprocedural loop optimizations).

Re: Simple apply/filter/reduce package in Go

#64

Post by NateDad »

inglor wrote: post #51
NateDad wrote: post #50

Earlier quoted context omitted.

var admins = users.Where(user => user.isAdmin()).Select(user => user.Name) vs var admins []string for _, user := range users { if user.IsAdmin() { admins = append(admins, user.Name) } } So, at the end of the day, these two pieces of code do the exact same thing, and have mostly the exact same meaning. I read them almost identically... Make a list of admins. iterate through the users, if the user is an admin, append i…

I humbly disagree - in the C# code I can pass an IQueryable and reuse the same code to make an SQL query, or a RavenDB query, or an XML query and it works just the same - there is a unified way to access data in the language. As for your query: ``` var admins = users.Where(u => u.IsAdmin()).Select(u => u.Name); var stars = admins.Select(name => name + "*"); ``` Like you said, you don't get paid per line feed - it's o…

Well, so, that's actually not doing the right thing. You just changed the names in the list of strings that holds admin names, but not the names on the user objects themselves.

Even if the first half were returning the users themselves to the admins list, you're still iterating over the list twice.

This is yet another reason why the for loop is good - it makes it more clear what is actually going on. This is your code in for loops:

    var admins []string
    for _, u := range users {
        if u.IsAdmin() {
            admins = append(admins, u.Name)
        }
    }
    
    for i := range admins {
        admins[i] = admins[i] + "*"
    }
The problem with the fancy syntax is that it makes it too easy to write suboptimal code that you'd never write if you were writing plain old for loops, like the above.

Re: Simple apply/filter/reduce package in Go

#65

Post by seanmcdirmid »

eropple wrote: post #32
seanmcdirmid wrote: post #21

Earlier quoted context omitted.

I saw that in June, and I'm interested in how effective it is, and what the usability characteristics are (it looked complex at the time, but it was an early demo). But going from 2010 to 2013 was a painful, and I don't want to throw this in a VM, so I'll just wait for RTM.

2013 to 2015 is a much smaller jump. Unless you are doing weirder things than me (and I mean, that's kind of hard!), I'd bet you'd be party. YMMV, of course.

I use a plugin that makes curly braces very small, so that my C# code looks more like Python. it seems to break on every version upgrade, and I can't stand normal sized C# curly braces anymore.

Re: Simple apply/filter/reduce package in Go

#66

Post by eropple »

seanmcdirmid wrote: post #65
eropple wrote: post #32

Earlier quoted context omitted.

2013 to 2015 is a much smaller jump. Unless you are doing weirder things than me (and I mean, that's kind of hard!), I'd bet you'd be party. YMMV, of course.

I use a plugin that makes curly braces very small, so that my C# code looks more like Python. it seems to break on every version upgrade, and I can't stand normal sized C# curly braces anymore.

I'd honestly be surprised if that broke this time around. Every extension I've tried in VS2015 worked fine. Some I had to fight with (VSIX manifests with too-restrictive versions), but it's mostly just drop-in-and-go.

They do run side-by-side just fine, too, FWIW.

Re: Simple apply/filter/reduce package in Go

#67

Post by frowaway001 »

NateDad wrote: post #64
inglor wrote: post #51

Earlier quoted context omitted.

I humbly disagree - in the C# code I can pass an IQueryable and reuse the same code to make an SQL query, or a RavenDB query, or an XML query and it works just the same - there is a unified way to access data in the language. As for your query: ``` var admins = users.Where(u => u.IsAdmin()).Select(u => u.Name); var stars = admins.Select(name => name + "*"); ``` Like you said, you don't get paid per line feed - it's o…

Well, so, that's actually not doing the right thing. You just changed the names in the list of strings that holds admin names, but not the names on the user objects themselves. Even if the first half were returning the users themselves to the admins list, you're still iterating over the list twice. This is yet another reason why the for loop is good - it makes it more clear what is actually going on. This is your cod…

> you're still iterating over the list twice

There is nothing about higher-order operations that would require developers to implement xs.map(...).map(...).map(...) or xs.Select(...).Select(...).Select(...) or whatever as multiple traversals over the data structure.

> The problem with the fancy syntax is that it makes it too easy to write suboptimal code that you'd never write if you were writing plain old for loops, like the above.

Maybe Google should just invest in hiring better people? I know it's getting increasingly harder to find people who still haven't evolved from the 1960ies mindset, but still ...

Re: Simple apply/filter/reduce package in Go

#68

Post by Veedrac »

pcwalton wrote: post #48
bunderbunder wrote: post #6

Earlier quoted context omitted.

In most languages, the for loop ends up being faster. Sometimes noticeably so. Me, I generally start with declarative syntax, but the profiler frequently tells me to go back and change it. Being a systems programmer, wonder if it's easier for him to just use the for loop as a default. Performance demands are always high in systems programming (because there'll be a whole stack of additional software standing on top o…

> In most languages, the for loop ends up being faster. Sometimes noticeably so. Me, I generally start with declarative syntax, but the profiler frequently tells me to go back and change it. The optimization techniques to make higher-order functions compile down into the same code as a for loop are well-known. All you have to do is inline, constant propagate, and maybe SROA. Every optimizing compiler I know of, even…

FWIW, PyPy has a hard time with this since Python's iterators are really heavyweight.
Post reply on HN