Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The state of low-level system programming

erik-poupaert.blogspot.com

Post reply on HN
Print view View original

41–50 of 62 posts

Re: The state of low-level system programming

#41

Post by Symmetry »

TheCondor wrote: post #22
sparkie wrote: post #20

Earlier quoted context omitted.

The need for C isn't so much restricted by the operating system, but by the hardware manufacturers. The simple fact is that C is the defacto choice of language for hardware vendors to ship a compiler for, and that doesn't look to change any time. It's a standard that can't be avoided, unless we can work cooperatively with enough major vendors to change this.

There is C specific hardware?!? This is a myth and part of the vicious circle here. I'm not a C hater by any means but the reason hardware vendors have historically included it is because there are free compilers that are nearly C enough that they can include and they have to include something. C is just a portable assembly, you can almost write a simplistic compiler in pure yacc and more than a couple of these C com…

Well, what sort of compiler optimizations you can make with C do have an effect on how people create microprocessors. For instance if you could be sure you could always disambiguate stack and heap accesses you could use two separate and somewhat memory pipes and get a big power or performance win, since there are a lot of N^2 structures involved in keeping everything coherent.

Also, most computers have a GPU which can't really be programmed in straight C.

Re: The state of low-level system programming

#42

Post by pcwalton »

Freestanding, ABI-level interoperability with C is planned for Rust, but is not yet implemented. There are two main reasons why being able to be called with dlsym() was less important for us than the current setup:

1. M:N scheduling. We wanted high scalability with hundreds of thousands or millions of tasks, which means we need the capability to have our own scheduler. OS threads aren't lightweight enough for this, especially on Windows, where over 80% of our Firefox users are. Having our own scheduler means we need our own ABI. We would like to make the scheduler optional, but getting it working was a higher priority than making it optional.

2. Safety. On many platforms (basically, anything but MSVC on Windows), the C stack is not bounds-checked; either there is no guard page or the guard page is unreliable because you can overshoot it. This is not acceptable for our Rust code, since stack overflows are a frequent source of exploitable security vulnerabilities for us. So we do our own stack management with a custom ABI that performs stack checks at every function entry, freeing us from this source of pain on all operating systems. Again, this is planned to be optional, but it was more important at this early stage for us to demonstrate advantages over C than for us to duplicate its ABI exactly.

Many libraries, however, would prefer direct interoperability with C to M:N scheduling or stack overflow safety. So we plan to provide a mechanism for Rust code to be called from C without having to set up a scheduler or perform stack checks. It's not implemented yet, but it's very much part of the current design, and I would be thrilled to see it happen.

Re: The state of low-level system programming

#43

Post by qznc »

MatthewPhillips wrote: post #32
groovy2shoes wrote: post #28

Clay[1] doesn't get as much attention as it deserves. The authors seem well-versed in programming language theory, but they also understand the practical needs of a systems language. What you end up with is a language that's sufficiently low-level to be a C replacement, but it offers a plethora of mechanisms for abstraction. Of all the new languages billed as "systems languages", Clay is the only one that excites me.…

I can't find any kind of getting started guide, and most of the libs have no documentation at all.

I found a "Clay for C++ programmers" tutorial

https://github.com/jckarter/clay/wiki/Clay-for-C---programme...

Re: The state of low-level system programming

#44

Post by slurry »

damian2000 wrote: post #19

If programmers wanted a statically-typed, compiled system language for their scripting efforts, they would be using the existing system languages for that already. I think Go has been fairly successful at Google in terms of improved code maintainability, and as an alternative to not just C but C++.

It seems to me that they are, in fact, using "statically-typed, compiled system language[s] for their scripting efforts" - that is in fact what people are doing with server-side Java, Scala and C#, not to mention Java/Dalvik and Objective-C on mobile platforms.

Perhaps this article misses the point a bit. For several decades we have been using scripting languages to glue together sharp pointy C[++] tools (sh/Perl/Python/Ruby approach) or to puppetize monolithic C[++] hulks from within (VB/elisp approach).

More recent practice, on both mobile and server platforms, has been to abstract away from C[++] with a middle layer of memory-managed, compiled C-ish goop - JVM, .Net, Objective-C, whatever. This is the level where people are doing everything from amazing high-availability big data things to amazing little bleep-bloop handheld things. I think the answer to "what's the state of low-level systems programming?" is "not much is going on, but damn, there's a lot of fresh stuff happening in mid-level systems programming."

Go and Rust are late entrants to this party, and probably exist more as a hedge against Oracle litigating Java/JVM into the ground than anything else.

Re: The state of low-level system programming

#45

Post by ponce »

groovy2shoes wrote: post #28

Clay[1] doesn't get as much attention as it deserves. The authors seem well-versed in programming language theory, but they also understand the practical needs of a systems language. What you end up with is a language that's sufficiently low-level to be a C replacement, but it offers a plethora of mechanisms for abstraction. Of all the new languages billed as "systems languages", Clay is the only one that excites me.…

May I mention Nimrod [1]? It has a lot going for it when it comes to meta-programming.

[1] http://nimrod-code.org/

Re: The state of low-level system programming

#46

Post by angersock »

justincormack wrote: post #14

The C ABI is not great for interacting with scripting languages though, even if it is the best we have right now. For a start there is no type information obtained with dlsym and you basically have to parse headers the interpretation of which is not clearly defined as the C language really defines an API and the ABI is defined by the compiler. Many libraries for this reason do not define stable ABIs and are a pain to…

You know, it's strange, because the Lua folks seem to be getting along just fine.

Re: The state of low-level system programming

#47

Post by erichocean »

doublec wrote: post #7
zem wrote: post #4

there's also ats [ http://www.ats-lang.org/ ]. somewhere on my long-term todo list is to write a library of efficient data strucures in ats, to take advantage of some of its safety guarantees.

It's a pity ATS doesn't get more attention in the "Systems programming language" field. It can call into C, be called from C, provides a powerful type system, and can be used to write very low level code. I've written a few posts about how ATS can interact with C on my weblog here: http://www.bluishcoder.co.nz/tags/ats/

I like reading your blog posts on ATS, but I don't think I could use it effectively.

I've had much better luck with exhaustive testing with Klee and related libraries (e.g. for testing multi-threaded programs).

For whatever reason, they map better to how I think about problems, and I've found it easier to develop new code that is testable within those constraints.

Re: The state of low-level system programming

#48

Post by nnq »

frujka wrote: post #36

it's probably an interesting read except that being hosted on google's blogspot means it only display blank pages due to content being embedded in some javascript shenanigans. Even a screenshot would be better than this. Please move on to some actual html website that just works or rehost the content elsewhere before posting it here. Thanks.

> rehost the content elsewhere before posting it here

Google's blogspot allows you to do normal html blogs (maybe now they've made this their "default", but still, it's OP's choice to have it this way)

Re: The state of low-level system programming

#49

Post by amirouche »

pjscott wrote: post #40
dschiptsov wrote: post #38

This is called FFI.) And yes, a language with which you cannot dload and call any shared library you wish is, how to say, handicapped language. NodeJS, Java to name a few. Look what they do due to inability to call, say, libpq.so.) But of course, re-monkey-patching everything from scratch by amateurs coders is much better idea. And, you know, you could create an entire "standard" with hundreds of pages of specificati…

I believe his thesis was actually that most languages are handicapped because they can not easily be called via the usual "load a .so file and call a function" ABI. Java, for instance, would be a more useful language to me if I could easily create Python bindings for useful Java libraries like Lucene, without needing to use something like Jython that runs on the JVM.

tl;dr: use JNI through pyjnius of jpype.

You can run Python 2.5 on lastest Jython, then you have access to JVM libs and the JVM libs to Python libs 2.5 compatible.

Calling JVM from CPython is actually possible: - with pyjnius from kivy fame built with cython it is still in development, AFAIK it is missing the possibility to call Python code from Java code aka. submitting a Python "callback" to a Java method, I think it is called IoC pattern and Hollywood principle «Don't call me, I call you». jnius is used in Kivy to make it possible to call Android libs. As an example, I created Python bindings of Blueprints with it. Blueprints is an abstraction library for graphdb like Neo4j, OrientDB, Titan, available here https://github.com/Printemps/python-blueprints/. Tutorial: «JavaClass = autoclass('path.to.JavaClass'); JavaClass.JavaMethod(python_object)». Those bindings are merely Pythonification but they are gotchas in types conversions.

- There is also jpype which looks like an established solution used by Neo4J fame to make it possible to embed Neo4J embedded into CPython.

I choosed jnius because it's faster; Those are funny benchmarks in log scale: https://docs.google.com/spreadsheet/ccc?key=0Ajc7MnpSjCokdGt...

Post reply on HN