Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go channels, goroutines and GC available in Nim

forum.nim-lang.org

Post reply on HN
Print view View original

61–70 of 87 posts

Re: Go channels, goroutines and GC available in Nim

#61

Post by pcwalton »

vvanders wrote: post #47
pcwalton wrote: post #23

Earlier quoted context omitted.

> If you're crossing threads in games stuff you're doing it wrong. There's a good chance you're running into false-sharing and other pitfalls. Of course, you shouldn't use shared memory unless you need it. But often you need it. Look at how game developers have demanded shared memory in JavaScript, for example. Modern multicore CPUs do a lot of work to make shared memory work, and work well. > Most games use worker-q…

> Look at how game developers have demanded shared memory in JavaScript Have a source for that? I find it pretty dubious. If you're looking to multicore for performance with javascript then you're using the wrong language. Correct memory layout and access patterns will give you a real-world 50-100x win.

Source: I work directly with people who interact with game developers who are asking for it.

Look for asm.js threads on HN: virtually every time it shows up someone brings up shared memory multithreading.

SharedArrayBuffer is the direct result of this popular demand: https://blog.mozilla.org/javascript/2015/02/26/the-path-to-p...

Re: Go channels, goroutines and GC available in Nim

#62

Post by MichaelGG »

Pharohbot wrote: post #57
steveklabnik wrote: post #55

Earlier quoted context omitted.

Cool. When you said 'any other language,' I thought you were speaking more broadly than C or C++.

I probably should have been more clear, but I think it's safe to say that unlike C/C++, Nim can handle these types of issues like other languages that deal with pointers (Java, Go etc) with control from the programmer. The only memory safe language I know is Rust, but I am probably wrong on that part, so that's why I singled out Rust on how It's safer than Nim.

That's not at all the impression I get from reading the Nim manual. It sounds rather clear when it says unsafe over many different features. Can you declare a function pointer and point it at anything? It allows unchecked array access - what's stopping traditional overflows? I've not used Nim, but compiling to C and exposing a lot of C-like functionality seems to indicate that code will still be subject to the same types of errors. Why do you say this isn't the case? Why does the manual not mention such things? (Another example: doesn't Nim need most objects to be GC allocated to be safe? So if you're not using GC (which I imagine lots of perf sensitive code will want to avoid), what's preventing errors there?)

Maybe I've got the wrong impression and their docs are terribly misleading and there are safety checks all over. But I found the dics easy to understand last time I read them and the safety issues seemed clearly marked and more or less where'd you expect.

Re: Go channels, goroutines and GC available in Nim

#63

Post by nulltype »

stefantalpalaru wrote: post #59
jeremyjh wrote: post #58

Earlier quoted context omitted.

Yes, but when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? If it doesn't, then nothing prevents foreign code from blocking other goroutines. How does the Nim code yield the thread for other goroutines, does it have to register a callback?

> when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? No. > How does the Nim code yield the thread for other goroutines, does it have to register a callback? There are no callbacks. Yielding happens automatically when launching another goroutine, when sending, receiving or selecting on a channel. You can also yield explicitly with go_yield()…

How does it handle making a blocking socket call in a go goroutine?

Re: Go channels, goroutines and GC available in Nim

#64

Post by jeremyjh »

stefantalpalaru wrote: post #59
jeremyjh wrote: post #58

Earlier quoted context omitted.

Yes, but when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? If it doesn't, then nothing prevents foreign code from blocking other goroutines. How does the Nim code yield the thread for other goroutines, does it have to register a callback?

> when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? No. > How does the Nim code yield the thread for other goroutines, does it have to register a callback? There are no callbacks. Yielding happens automatically when launching another goroutine, when sending, receiving or selecting on a channel. You can also yield explicitly with go_yield()…

> > when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it?

>No.

Are you sure? Once a thread enters cgo it is considered blocked and is removed from the thread pool according to these sources [1][2]. I previously found a thread where Ian Lance Taylor explained it more explicitly but I couldn't find that now. Is that not what is happening though when __go_go invokes your function pointer?

I do not understand how the Nim code can live in the segmented stack of a goroutine, nor how the Go runtime could know it is time to grow that stack.

[1] https://groups.google.com/forum/#!searchin/golang-nuts/gorou...

[2]http://stackoverflow.com/questions/27600587/why-my-program-o...

Re: Go channels, goroutines and GC available in Nim

#65

Post by rbehrends »

MichaelGG wrote: post #39
Pharohbot wrote: post #38

Earlier quoted context omitted.

Why not? I'm interested to know because in my opinion I don't see it any less safe than languages that don't have automatic memory management and/or languages like Rust.

Because it is flatly untrue? Memory safety is rather a binary thing. C# without /unsafe is safe. Same for Java and Rust. Not true for Nim or C/C++. Rust is unique in doing this without any GC or other runtime overhead, AFAIK, which makes it a bit special.

Nim does not have a separate unsafe keyword, because all unsafe features are already characterized by keywords; that's a result of its Pascal heritage. To check whether a piece of Nim code is safe, you check for the presence or absence of these keywords; e.g., you can grep for "ptr" in Nim, while grepping for "*" in C# isn't particularly helpful. Every unsafe feature in Nim has an associated keyword/pragma. Having a special "unsafe" keyword that says, essentially, "this procedure can contain other unsafe keywords" is sort of superfluous.

Note: these unsafe features have two purposes. One is to interface with C/C++ code. The other is to be able to write close-to-the-metal code in Nim rather than in C (where you wouldn't gain any safety by using C, but lose the expressiveness of Nim). This is, for example, how Nim's GC is itself written in Nim.

None of the unsafe features are necessary for high-level programming, i.e. unless you actually want to operate that close to the metal.

Re: Go channels, goroutines and GC available in Nim

#66

Post by stefantalpalaru »

nulltype wrote: post #63
stefantalpalaru wrote: post #59

Earlier quoted context omitted.

> when the Go runtime calls that foreign function pointer, it is going to schedule an entire OS thread for its duration isn't it? No. > How does the Nim code yield the thread for other goroutines, does it have to register a callback? There are no callbacks. Yielding happens automatically when launching another goroutine, when sending, receiving or selecting on a channel. You can also yield explicitly with go_yield()…

How does it handle making a blocking socket call in a go goroutine?

It doesn't, yet.

Re: Go channels, goroutines and GC available in Nim

#67

Post by dbaupp »

rbehrends wrote: post #65
MichaelGG wrote: post #39

Earlier quoted context omitted.

Because it is flatly untrue? Memory safety is rather a binary thing. C# without /unsafe is safe. Same for Java and Rust. Not true for Nim or C/C++. Rust is unique in doing this without any GC or other runtime overhead, AFAIK, which makes it a bit special.

Nim does not have a separate unsafe keyword, because all unsafe features are already characterized by keywords; that's a result of its Pascal heritage. To check whether a piece of Nim code is safe, you check for the presence or absence of these keywords; e.g., you can grep for "ptr" in Nim, while grepping for "*" in C# isn't particularly helpful. Every unsafe feature in Nim has an associated keyword/pragma. Having a…

There's undefined behaviour in the "main" language, e.g. https://news.ycombinator.com/item?id=9050999

Re: Go channels, goroutines and GC available in Nim

#68

Post by rbehrends »

dbaupp wrote: post #67
rbehrends wrote: post #65

Earlier quoted context omitted.

Nim does not have a separate unsafe keyword, because all unsafe features are already characterized by keywords; that's a result of its Pascal heritage. To check whether a piece of Nim code is safe, you check for the presence or absence of these keywords; e.g., you can grep for "ptr" in Nim, while grepping for "*" in C# isn't particularly helpful. Every unsafe feature in Nim has an associated keyword/pragma. Having a…

There's undefined behaviour in the "main" language, e.g. https://news.ycombinator.com/item?id=9050999

That's a bit misleading and mostly due to the fact that (1) Nim hasn't reached 1.0 yet and (2) in practice these issues are relatively uncommon for the C code that Nim generates, so this hasn't been a particularly high priority.

First of all, Nim's backend does not target the C standard; it targets a number of "approved" C compilers, which makes it (1) a bit easier to avoid undefined behavior, because these C compilers know that they may be used as backends by high-level languages and provide efficient means to disable some of the undefined behavior and (2) allows Nim to emit optimizations specifically for them. For example, Nim knows that gcc understands case ranges in its switch statement and can optimize for that. See compiler/extccomp.nim for some more examples. Nim also makes some additional assumptions about how data is represented that are not defined in the C standard, but are true for all target architectures (or can be made true with the appropriate compiler configurations).

Second, regarding the specific cases of undefined behavior:

1. That shift widths aren't subject to overflow checks is an oversight; most shifts are by a constant factor, anyway, so they can be checked at compile time with no additional overhead. Nim does not do signed shifts (unless you escape to C), so they are not an issue.

2. Integer overflow is actually checked, but expensive; there's an existing pull request for the compiler to generate code that leverages the existing clang/gcc builtins to avoid the overhead, but that hasn't been merged yet; -fno-strict-overflow/-ftrapv/-fwrapv can also be used for clang/gcc to suppress the undefined behavior (depending on what you want) and one of them may be enabled by default in the absence of checks.

3. Nils are not currently being checked, but they will be. There's already a nilcheck pragma, but that isn't fully implemented and also not available through a command line option. This will be fixed. Until then, you can use gcc (where -O2 implies -fisolate-erroneous-paths-dereference) or use --passC:-fsanitize=null for clang to fix the issue.

Re: Go channels, goroutines and GC available in Nim

#69

Post by pcwalton »

rbehrends wrote: post #65
MichaelGG wrote: post #39

Earlier quoted context omitted.

Because it is flatly untrue? Memory safety is rather a binary thing. C# without /unsafe is safe. Same for Java and Rust. Not true for Nim or C/C++. Rust is unique in doing this without any GC or other runtime overhead, AFAIK, which makes it a bit special.

Nim does not have a separate unsafe keyword, because all unsafe features are already characterized by keywords; that's a result of its Pascal heritage. To check whether a piece of Nim code is safe, you check for the presence or absence of these keywords; e.g., you can grep for "ptr" in Nim, while grepping for "*" in C# isn't particularly helpful. Every unsafe feature in Nim has an associated keyword/pragma. Having a…

Presumably there are plans to disallow accidental sending of thread-local GC'd pointers to other threads as well?

Re: Go channels, goroutines and GC available in Nim

#70

Post by rbehrends »

pcwalton wrote: post #69
rbehrends wrote: post #65

Earlier quoted context omitted.

Nim does not have a separate unsafe keyword, because all unsafe features are already characterized by keywords; that's a result of its Pascal heritage. To check whether a piece of Nim code is safe, you check for the presence or absence of these keywords; e.g., you can grep for "ptr" in Nim, while grepping for "*" in C# isn't particularly helpful. Every unsafe feature in Nim has an associated keyword/pragma. Having a…

Presumably there are plans to disallow accidental sending of thread-local GC'd pointers to other threads as well?

See: http://nim-lang.org/docs/manual.html#threads-gc-safety

(work in progress)

Post reply on HN