R, OpenMP, MKL, Disaster

In this excellent article Patrick Li, the author of a new optionally type language Stanza and co-founder of JITX (YC S18), provided a compelling reason to design a new language [1]. TL;DR, a powerful language like Ruby enabled the creation of powerful RoR library and framework that help spawned unicorn size startups like Github and Twitter, that's otherwise not feasible.

I want to add another dimension to this argument, what if we can maintain an existing language eco-system (library, community, etc) but modernize the engine that's running and compilation of the R language. This new engine can avoid the dreaded global locking limitation, provides native multi-thread applications and seamless interface with non-native R libraries in C/C++. Interestingly someone has tried this, with a sponsorship from Oracle no less, and presented this futile effort in the last year's R flagship conference keynote [2].

IMHO he will be more successful in his endeavour if using D language in his previous endeavors. What so special about D language you may ask? I would refer to the fact most of the languages do not provide Ruby on Rails (RoR) like tool except D but that for another story (see ref [1]). There's also the fact that D has a working alternative library to OpenBLAS and MKL, and it's even faster than both them five years back [3]! D also supports open method as an alternative for multiple dispatches that is much touted by Julia language community. D is also bringing native support for borrow checker feature that's always mentioned in the same sentence as Rust language. In addition D also has second to none FFI support for C and C++ language. Heck the latest D compiler has standard C compiler built-in. I can go on furthermore but I think you've probably already got the pictures.

My not so humble proposal to R and D language community is to compile R on top of D language. Essentially you a have dynamic language of R that is compiled at runtime (CTFE) on top of static D language. This approach is becoming more popular now as posted recently for the new Val and Valet language combination [4]. Just think of CTFE as the new JVM, but provides truly static and native compilation for R.

[1] What makes a programming language productive? “Stop designing languages. Write libraries instead.”:

https://jaxenter.com/stop-designing-languages-write-librarie...

[2] Why R? 2020 Keynote - Jan Vitek - How I Learned to Love Failing at Compiling R:

https://www.youtube.com/watch?v=VdD0nHbcyk4

[3] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen:

http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...

[4] Show HN: Val - A powerful static and dynamic programming language(val-lang.org):

https://news.ycombinator.com/item?id=28683171

> Lisps are incredibly flexible, for instance, but do not generally encourage bad habits

People often say "LISP is so powerful but nobody can understand anyone else's code". That's the dominant explanation for why LISP isn't more popular (along with the "oatmeal with toenail clippings mixed in" syntax, which most people don't find readable, regardless of the fervent beliefs of the LISP community to the contrary). The community stays small because of the unappealing syntax, and even within the community people find it hard to work together, because everyone has their own style, so the kind of coding and collaboration that produces generally useful libraries doesn't tend to happen. I would argue there's no meaningful distinction between "bad habits" and "habits that inhibit the development of generally useful software". In fact, that's the most useful definition of "bad habits" I can imagine.

Flexibility is a root cause of bad habits thus defined, because flexibility is what enables people to make bad choices. Language designers have long recognized this. It's why certain languages impose heavy restrictions on how you can structure your code, from Java's everything-is-a-class to Python's indentation-based scoping. They have a certain vision for what constitutes effective code and they know it won't happen unless they force everyone to follow it. In other words, they choose to reduce flexibility to prevent what they consider to be bad habits.

> R encourages bad habits for the following reasons:

Your reasons are just common complaints about R issues, not actual arguments that these issues encourage bad habits.

> - R is made "for statisticians, by statisticians" so a lot of the example code out there is very poorly written

At best this is an argument that widely publicized badly written R code encourages bad habits, not R itself.

> There's also the 1 indexing

There's a big difference between "language feature I don't like" and "language feature that encourages bad habits". A language that has different conventions than your favorite language is just different.