How is the byte counting reader supposed to work in user space without putting the buffer in user space? The article claims there is a way but I want to see what is meant by counting bytes in that case.
Zero-copy in Go: sendfile, splice, and the cost of io.Copy
11–20 of 24 posts
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#12How is the byte counting reader supposed to work in user space without putting the buffer in user space? The article claims there is a way but I want to see what is meant by counting bytes in that case.
sendfile(2) and io.ReaderFrom both return the number of bytes transmitted. The issue is that users are unaware of (or forget about) the optional interface upgrades and fail to define all the methods required for interface upgrades on their wrapper structs. You can definitely make a counting reader with a minimal performance loss, but the proper solution is less obvious than it ideally should be.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#13How is the byte counting reader supposed to work in user space without putting the buffer in user space? The article claims there is a way but I want to see what is meant by counting bytes in that case.
sendfile(2) and io.ReaderFrom both return the number of bytes transmitted. The issue is that users are unaware of (or forget about) the optional interface upgrades and fail to define all the methods required for interface upgrades on their wrapper structs. You can definitely make a counting reader with a minimal performance loss, but the proper solution is less obvious than it ideally should be.
Isn’t this a symptom of structural/duck typing where interfaces are not declared? In Java for all its faults this wouldn’t happen because you’d be forced to implement all the interfaces.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#14Beware, there are versions of go where sendfile is broken and only sends the first 4k of a file on macos.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#15Earlier quoted context omitted.
sendfile(2) and io.ReaderFrom both return the number of bytes transmitted. The issue is that users are unaware of (or forget about) the optional interface upgrades and fail to define all the methods required for interface upgrades on their wrapper structs. You can definitely make a counting reader with a minimal performance loss, but the proper solution is less obvious than it ideally should be.
Isn’t this a symptom of structural/duck typing where interfaces are not declared? In Java for all its faults this wouldn’t happen because you’d be forced to implement all the interfaces.
> Isn’t this a symptom of structural/duck typing where interfaces are not declared
Yes, essentially duck typing. See https://github.com/golang/go/blob/65504872cbca64d77f45828409...
The logic uses a type assertion to safely verify if the value backing the provided io.Reader interface also implements the io.ReaderFrom interface. If it matches, then it will use the more efficient implementation
if rf, ok := dst.(ReaderFrom); ok {
return rf.ReadFrom(src)
}
I don't think I would go that far. In Java, many libraries make heavy use of the `instanceof` keyword, which is more or less the same as Go type assertions.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#16Earlier quoted context omitted.
Isn’t this a symptom of structural/duck typing where interfaces are not declared? In Java for all its faults this wouldn’t happen because you’d be forced to implement all the interfaces.
> Isn’t this a symptom of structural/duck typing where interfaces are not declared Yes, essentially duck typing. See https://github.com/golang/go/blob/65504872cbca64d77f45828409... The logic uses a type assertion to safely verify if the value backing the provided io.Reader interface also implements the io.ReaderFrom interface. If it matches, then it will use the more efficient implementation if rf, ok := dst.(ReaderF…
Yes, but contrary to Go, if you change an interface it will be a compiler error if additional methods are missing, unless they have default implementations.
In Java type assertions are mostly used when writing code pre-generics style, like downcasting from a common subclasse into the actual implementation, not to see if an interface is supported, as it is a given from the type system.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#17Earlier quoted context omitted.
> Isn’t this a symptom of structural/duck typing where interfaces are not declared Yes, essentially duck typing. See https://github.com/golang/go/blob/65504872cbca64d77f45828409... The logic uses a type assertion to safely verify if the value backing the provided io.Reader interface also implements the io.ReaderFrom interface. If it matches, then it will use the more efficient implementation if rf, ok := dst.(ReaderF…
Yes, but contrary to Go, if you change an interface it will be a compiler error if additional methods are missing, unless they have default implementations. In Java type assertions are mostly used when writing code pre-generics style, like downcasting from a common subclasse into the actual implementation, not to see if an interface is supported, as it is a given from the type system.
It’s pretty common to type check interface implementations in Go using package-level casts.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#18Earlier quoted context omitted.
Yes, but contrary to Go, if you change an interface it will be a compiler error if additional methods are missing, unless they have default implementations. In Java type assertions are mostly used when writing code pre-generics style, like downcasting from a common subclasse into the actual implementation, not to see if an interface is supported, as it is a given from the type system.
It’s pretty common to type check interface implementations in Go using package-level casts.
I know, it is yet another language hack, just like the iota/const dance, instead of proper language constructs.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#19Earlier quoted context omitted.
It’s pretty common to type check interface implementations in Go using package-level casts.
I know, it is yet another language hack, just like the iota/const dance, instead of proper language constructs.
How is a package level interface check a “language hack”? They are straight forward and provide a user the same compile time guarantees as the Java implements keyword.
Re: Zero-copy in Go: sendfile, splice, and the cost of io.Copy
#20Earlier quoted context omitted.
I know, it is yet another language hack, just like the iota/const dance, instead of proper language constructs.
How is a package level interface check a “language hack”? They are straight forward and provide a user the same compile time guarantees as the Java implements keyword.
Because they are a workaround for what should be a language feature in first place, just like on ML languages to type check structural types.