Live data from Hacker News

Outperforming LAPACK with C++ metaprogramming

wordsandbuttons.online

61–70 of 82 posts

Re: Outperforming LAPACK with C++ metaprogramming

#61

In his first factorial example, he could've avoided templates altogether with constexpr: https://godbolt.org/g/ZaoMNa Or just use a template specialisation: https://godbolt.org/g/TVkHMd

Thanks for sharing the godbolt site, can see myself using it for debugging.

the author of the site (Matt Godbolt) is in general pretty great.

Re: Outperforming LAPACK with C++ metaprogramming

#62
post #26
post #16

D's Mir does the same sort of thing, and metaprogramming in D is much nicer, because it was designed from the ground up instead of the bug elevated to feature that metaprogramming is in C++. http://docs.mir.dlang.io/latest/index.html http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...

>and metaprogramming in D is much nicer, because it was designed from the ground up instead of the bug elevated to feature that metaprogramming is in C++ Yet neither of them are as elegant as LISP macros.

I'm not sure how you're measuring elegance, but writing Lisp macros and getting them to work correctly has never been something I'd call elegant. You have the Common Lisp approach, where you work in a complicated sublanguage like this example from PCL:

  (defmacro when (condition &rest body)
    `(if ,condition (progn ,@body)))
and then you think hard about where the leaks are. Or else you take the Scheme approach of hygienic macros which are even less elegant. Macros in Lisp are powerful, but I'm not sure metaprogramming in Lisp is better than in a language like D.

Re: Outperforming LAPACK with C++ metaprogramming

#63
post #59

Others have already mentioned Cramer's rule, but even if this code were switched to Gaussian elimination it might not beat lapack on big problems. Lapack implementations do extra tricks to optimize cache usage on big matrices. Would be interesting to include eigen in the benchmark too.

I've read your comment several times and can't follow it. I think you're saying that even if the author continued to use Cramer's rule, he would lose to LAPACK on big problems.

Re: Outperforming LAPACK with C++ metaprogramming

#64
post #59

Others have already mentioned Cramer's rule, but even if this code were switched to Gaussian elimination it might not beat lapack on big problems. Lapack implementations do extra tricks to optimize cache usage on big matrices. Would be interesting to include eigen in the benchmark too.

I've read your comment several times and can't follow it. I think you're saying that even if the author continued to use Cramer's rule, he would lose to LAPACK on big problems.

No. If he continued to use Cramer's rule, he would definitely lose to LAPACK on big problems, purely because of using Cramer's rule. But even if he didn't continue to use Cramer's rule (if he switched to Gaussian elimination), he still might lose to LAPACK, because LAPACK is pretty well optimized.

At least, that's how I interpreted the comment...

Re: Outperforming LAPACK with C++ metaprogramming

#65

Earlier quoted context omitted.

I've read your comment several times and can't follow it. I think you're saying that even if the author continued to use Cramer's rule, he would lose to LAPACK on big problems.

No. If he continued to use Cramer's rule, he would definitely lose to LAPACK on big problems, purely because of using Cramer's rule . But even if he didn't continue to use Cramer's rule (if he switched to Gaussian elimination), he still might lose to LAPACK, because LAPACK is pretty well optimized. At least, that's how I interpreted the comment...

Correct interpretation

Re: Outperforming LAPACK with C++ metaprogramming

#66
post #7

Earlier quoted context omitted.

As an anecdote of just how powerful this can be, I once wrote a prototype of a simulation (written in C) that used a lot of integer divisions and needed to be run many times with different parameters. As a quick and dirty hack that took about half hour to implement, I decided to just have the program as a template string and the parameters be substituted in by a python script that called GCC. To the amazement of a ve…

That's pretty much what you get with Julia, except it will also do interprocedural optimization (i.e. inline constants and functions across calls) on v0.7. All without playing with templates. I'll gladly pay a 1 second JiT lag for a few days off my computation. At the same time, I don't have screw around with templates.

Your comment has taken me from having mixed feelings about Julia to making me convinced I want to try Julia for something numerically intensive that I would be inclined to turn to C for.

Re: Outperforming LAPACK with C++ metaprogramming

#67
post #66

Earlier quoted context omitted.

That's pretty much what you get with Julia, except it will also do interprocedural optimization (i.e. inline constants and functions across calls) on v0.7. All without playing with templates. I'll gladly pay a 1 second JiT lag for a few days off my computation. At the same time, I don't have screw around with templates.

Your comment has taken me from having mixed feelings about Julia to making me convinced I want to try Julia for something numerically intensive that I would be inclined to turn to C for.

Even better, Julia code is super high-level and readable. And you can just drop down to C using the builtin "ccall" facility. Julia's type system is designed for excellent interop with the C ABI.

Re: Outperforming LAPACK with C++ metaprogramming

#68
post #26

Earlier quoted context omitted.

>and metaprogramming in D is much nicer, because it was designed from the ground up instead of the bug elevated to feature that metaprogramming is in C++ Yet neither of them are as elegant as LISP macros.

I'm not sure how you're measuring elegance, but writing Lisp macros and getting them to work correctly has never been something I'd call elegant. You have the Common Lisp approach, where you work in a complicated sublanguage like this example from PCL: (defmacro when (condition &rest body) `(if ,condition (progn ,@body))) and then you think hard about where the leaks are. Or else you take the Scheme approach of hygie…

> You have the Common Lisp approach, where you work in a complicated sublanguage like this example from PCL:

I don't think you actually understand anything about the example code you posted. There is no special "complicated sublanguage" - the backquote[1] is an extension of the quote shorthand notation[2] for list literals. It is for building data structures and has nothing to do with "sublanguages" or macros. There is no special syntax for macros in Common Lisp. Common Lisp macros do not produce source code - they produce any type of Common Lisp objects directly. That is why they are simpler and strictly more powerful than D templates.

When it comes to templates, I wouldn't call anything from this list of D examples[3] elegant.

[1] http://www.lispworks.com/documentation/HyperSpec/Body/02_df.... [2] http://www.lispworks.com/documentation/HyperSpec/Body/s_quot... [3] https://dlang.org/templates-revisited.html

Re: Outperforming LAPACK with C++ metaprogramming

#69
post #68

Earlier quoted context omitted.

I'm not sure how you're measuring elegance, but writing Lisp macros and getting them to work correctly has never been something I'd call elegant. You have the Common Lisp approach, where you work in a complicated sublanguage like this example from PCL: (defmacro when (condition &rest body) `(if ,condition (progn ,@body))) and then you think hard about where the leaks are. Or else you take the Scheme approach of hygie…

> You have the Common Lisp approach, where you work in a complicated sublanguage like this example from PCL: I don't think you actually understand anything about the example code you posted. There is no special "complicated sublanguage" - the backquote[1] is an extension of the quote shorthand notation[2] for list literals. It is for building data structures and has nothing to do with "sublanguages" or macros. There…

> I don't think you actually understand anything about the example code you posted.

This doesn't add anything to the conversation. If you want to discuss my post, I'm happy to discuss. You're clearly upset that I didn't praise Common Lisp, but things like that don't lead to useful discussion.

> There is no special "complicated sublanguage" - the backquote[1] is an extension of the quote shorthand notation[2] for list literals. It is for building data structures and has nothing to do with "sublanguages" or macros. There is no special syntax for macros in Common Lisp.

I've never seen anyone write macros using only s-expressions. True, you don't need to learn any special syntax to do it, but that's not how anyone writes macros.

> When it comes to templates, I wouldn't call anything from this list of D examples[3] elegant.

How is that is relevant to my comment?

Re: Outperforming LAPACK with C++ metaprogramming

#70

This article makes no sense at all. It compares a numerically unstable O(n!) algorithm (Cramer's rule) to a numerically stable O(n^3) algorithm (Gaussian elimination).

What bothers me more is the fact that the article seems to compare LAPACKs function for solving any-sized systems of equations with a specialized function for solving 5x5 systems.

I mean, it's neat trick and all, but the comparison with LAPACK does not really stand on equal ground.

Post reply on HN