Rust means never having to close a socket • Hacker News Board

Re: Rust means never having to close a socket

#101

Post by pcwalton » Tue, Oct 07, 2014, 9:25 PM UTC

spullara wrote: post #96

wycats wrote: post #72

Earlier quoted context omitted.

In Rust, you often use stack allocation, which, as you pointed out, is much more efficient. There's some heap allocation in Rust, but in my experience it's actually a minority of all allocations. Rust also has an Arena library ( http://doc.rust-lang.org/0.11.0/arena/index.html ), which can be used for cases where you know that you have enough memory for a given operation and are willing to throw out the entire thing…

Arena allocation has similar efficiencies as GC.

Except for not having the mark phase (also not having to copy). The mark phase can take a lot of time.

Re: Rust means never having to close a socket

#102

Post by pcwalton » Tue, Oct 07, 2014, 9:28 PM UTC

spullara wrote: post #95

masklinn wrote: post #66

Earlier quoted context omitted.

> It is far more predictable than GC but I don't believe that it isn't more expensive. A GC has to determine liveness of objects, so it has to traverse the object graph (or at least the most recent generation if the GC is generational) to know which objects are still alive and which aren't. That's what "GC scaling" refers to. Then it still has to deallocate dead objects, and pay the same deallocation cost as scope-bo…

It doesn't have to deallocate dead objects. That is sort of the point of most collectors.

All GCs have to deallocate objects. It may be very cheap to do so, for example when you have a semispace collector and no finalizers, but switching the spaces is still a form of deallocation. Production-quality GCs don't use semispace for everything because of the huge memory use overhead, and all practical GC'd languages I know of need finalizers in some form.

Re: Rust means never having to close a socket

#103

Post by pcwalton » Tue, Oct 07, 2014, 9:30 PM UTC

lbrandy wrote: post #98

masklinn wrote: post #73

Earlier quoted context omitted.

> What part of that is undefined? When you move the pointer, the original becomes null.. Nope, the argument is "valid but unspecified" not necessarily null. And http://en.cppreference.com/w/cpp/utility/move specifically has this as an example of UB: std::vector v; std::string str = "example"; v.push_back(std::move(str)); // str is now valid but unspecified str[0]; // undefined behavior: operator[](size_t n) has a pre…

OP is correct. You've quoted the general language around move semantics but unique_ptr itself has a stronger gaurantee. From 20.7.1 Additionally, u can, upon request, transfer ownership to another unique pointer u2. Upon completion of such a transfer, the following postconditions hold: — u.p is equal to nullptr, unique_ptr isn't what's unsafe here. Null pointers are. The difference, admittedly, is mostly semantic. Th…

> There is nothing undefined about using a unique_ptr after moving unless you use the internal pointer in undefined ways.

If you dereference (i.e. use) a moved unique pointer you get undefined behavior.

When talking about "using a pointer" most people mean dereferencing it, not for example comparing against null. It may be somewhat imprecise language, but it's what security people mean when they talk about "use after free", for example.

Re: Rust means never having to close a socket

#104

Post by steveklabnik » Tue, Oct 07, 2014, 9:34 PM UTC

jaekwon wrote: post #100

This works well for system resources, but what about custom objects destructors?

You implement the Drop trait yourself, and it Just Works.

Re: Rust means never having to close a socket

#105

Post by malisper » Tue, Oct 07, 2014, 9:42 PM UTC

agumonkey wrote: post #79

ben0x539 wrote: post #78

Earlier quoted context omitted.

That only works if the lifetime of a resource is a lexical region, right?

Good question. I wonder if there was some Lisp with linear logic that would allow lexical and ~borrowed (sorry if I misuse the term) resources management.

It shouldn't be very hard to create a basic ownership system. Here is a version I just wrote, which works by defining a new version of "defun" that uses dynamic variables to keep track of ownership:

  (defvar *transferring* '() "Objects which are going to be transferred from one owner to another.")
  (defvar *resources* '() "The resources which need to be freed when the current procedure is done.")

  (defmacro new-defun (name args &body body)
    "Define a procedure which will have ownership over the resources it creates using any of the 'new' procedures.
    `(defun ,name ,args
       (let ((*resources* *transferring*))
         (setf *transferring* '())
         (unwind-protect (progn ,@body)
           (mapcar #'free *resources*)
           (mapcar #'free *transferred*))))) ; These are resources which were supposed to be transferred by weren't due to an error.

  (defun transfer (resource)
    "Transfer ownership of a resource from the current one to the next procedure called which can accept ownership."
    (push resource *transferring*)
    (setf *resources* (remove resource *resources*))
    resource)

  (defun new-open (&rest args)
    "Open a file whose owner is the first procedure up the stack which can accept ownership."
    (let ((file (apply #'open args)))
      (push file *resources*)
      file))

All that is needed now is a method "free" which will free any given resource. With this, any resource allocated with a "new" procedure (the only one I wrote was "new-open") will by put under ownership of the last procedure on the stack which was defined with "new-defun". Transfer between owners can take place by using "transfer". The best part is that more facilities are easy to build on top of this. The "drop" procedure would be trivial to add. The only problem I can see here is that there may be some issues if "free" throws an error, since "unwind-protect" doesn't protect the cleanup-forms. This should be fixable by just wrapping every call to "free" with "ignore-errors".

Re: Rust means never having to close a socket

#106

Post by apendleton » Tue, Oct 07, 2014, 9:51 PM UTC

illumen wrote: post #33

wycats wrote: post #27

Earlier quoted context omitted.

What happens if there's a cycle? It will get closed when the cycle is finally collected. You can rely on a GC (and refcounting + a cycle collector is absolutely a GC) to finalize your resources for you, but it is quite difficult to reason about when the file will eventually be closed. Also, the reference counting semantics of CPython are implementation-specific. From the PEP that introduced `with`: > Note that we're…

CPython is the implementation that 98% of the market uses, but you are partially correct for that other 2%. You are also wrong for CPython which does NOT require the garbage collector to be on, and DOES release the file in a deterministic way. Simply, the article is misleading with regards to python.

If the garbage collector were turned off and a cycle were to be present, the file would simply never be closed. Though I suppose that would, indeed, be deterministic.

Re: Rust means never having to close a socket

#107

Post by mithrilskeleton » Tue, Oct 07, 2014, 10:06 PM UTC

steveklabnik wrote: post #91

mwcampbell wrote: post #81

Earlier quoted context omitted.

I know Rust has RC pointers. But I'm guessing a Rust programmer would still have to understand ownership and borrowing, since the standard library and third-party libraries use those concepts. I think what I'd really like is something like C# and the .NET base class library, but AOT compiled to native code and using automatic reference counting, with some way to indicate weak references. I know that neither GC nor re…

That's true, you can't totally avoid knowing about these things. But if you don't want to think about them in your code, and are okay with the price, you can. It sounds like Swift may be close to what you want, yeah?

Unfortunately Swift is not open source or cross platform

Re: Rust means never having to close a socket

#108

Post by steveklabnik » Tue, Oct 07, 2014, 10:13 PM UTC

mithrilskeleton wrote: post #107

steveklabnik wrote: post #91

Earlier quoted context omitted.

That's true, you can't totally avoid knowing about these things. But if you don't want to think about them in your code, and are okay with the price, you can. It sounds like Swift may be close to what you want, yeah?

Unfortunately Swift is not open source or cross platform

Right, I meant solely on paradigm.

Re: Rust means never having to close a socket

#109

Post by viraptor » Tue, Oct 07, 2014, 10:18 PM UTC

wycats wrote: post #65

spullara wrote: post #62

"Rust achieves both of these features without runtime costs (garbage collection or reference counting), and without sacrificing safety." If it is freeing everything at the scope boundary it is incurring a huge runtime cost relative to garbage collected languages. GC scales with the number of live objects during GC while this scheme scales with the number of allocated objects. It is far more predictable than GC but I…

I'm a little bit confused. Every object that gets allocated has to be freed at some point. That cost is the same no matter how the memory is managed. Are you talking about an additional cost beyond "freeing memory that was allocated"?

Not necessarily the same. For example for arena allocation, you can drop lots of related objects at the same time. If you don't have to run destructors on them, the cost will be much lower than any other way of managing them.

Re: Rust means never having to close a socket

#110

Post by spullara » Tue, Oct 07, 2014, 11:20 PM UTC

pcwalton wrote: post #102

spullara wrote: post #95

Earlier quoted context omitted.

It doesn't have to deallocate dead objects. That is sort of the point of most collectors.

All GCs have to deallocate objects. It may be very cheap to do so, for example when you have a semispace collector and no finalizers, but switching the spaces is still a form of deallocation. Production-quality GCs don't use semispace for everything because of the huge memory use overhead, and all practical GC'd languages I know of need finalizers in some form.

Finalizers are certainly the exception rather than the rule for objects. Deallocation to me represents some action taken for a particular object and scales with the number of objects deallocated. Semi-space copying collectors do not have this property. I think we only disagree on the semantics of "deallocation" — which is why I specified my argument in terms of how things scale. Similarly compacting collectors like Java's CMS also have this scaling property without semi-spaces.