Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Buffer: Composable Buffers for Go

github.com

Post reply on HN
Print view View original

1–10 of 15 posts

Re: Buffer: Composable Buffers for Go

#2

Post by stinos » 

   // Buffer 32KB to Memory, after that buffer to 100MB chunked files
  buf := buffer.NewUnboundedBuffer(32*1024, 100*1024*1024)

  pool := NewFilePool(100*1024*1024, "") // "" -- use temp dir
I seem to have a twofold feeling about this code: on one hand I really like the brevity of these statements, but on the other hand it does come at the cost of being rather cryptic: without the comments one can only guess what exactly is going on. To the point that should I want to use this in my own code and make it understandable I'd almost would be forced to either copy the comments as well or else wrap it in a method with another name or. In such cases it probably would have been better if the original API already had done this.

Re: Buffer: Composable Buffers for Go

#3

Post by gouggoug »

stinos wrote: post #2

// Buffer 32KB to Memory, after that buffer to 100MB chunked files buf := buffer.NewUnboundedBuffer(32*1024, 100*1024*1024) pool := NewFilePool(100*1024*1024, "") // "" -- use temp dir I seem to have a twofold feeling about this code: on one hand I really like the brevity of these statements, but on the other hand it does come at the cost of being rather cryptic: without the comments one can only guess what exactly i…

I feel like we could say that about any code. It doesn't sound cryptic to me. It does require to read the doc (http://godoc.org/github.com/djherbis/buffer#NewUnboundedBuff...) but I'd argue that's the case for everything.

Re: Buffer: Composable Buffers for Go

#5

Post by nkatsaros »

stinos wrote: post #2

// Buffer 32KB to Memory, after that buffer to 100MB chunked files buf := buffer.NewUnboundedBuffer(32*1024, 100*1024*1024) pool := NewFilePool(100*1024*1024, "") // "" -- use temp dir I seem to have a twofold feeling about this code: on one hand I really like the brevity of these statements, but on the other hand it does come at the cost of being rather cryptic: without the comments one can only guess what exactly i…

If anything this is just a nomenclature problem. It's common for Go packages to have New* functions for common use cases and this is one example. After looking at the source it is very clear what it does.

Re: Buffer: Composable Buffers for Go

#6

Post by fishnchips »

kenferry wrote: post #4

As a relative go newb, I do have a question about this: I thought channels/go routines were already essentially composable buffers. No?

Yes and no. Goroutines are lightweight threads - what other languages might call fibers. Channels are synchronisation primitives designed to work with goroutines but they do indeed provide some buffering. What OP is doing though is creating customisable buffers that allow you to control for memory usage and that are likely designed to replace things like bytes.Buffer rather then buffered channels. Of course I may be totally wrong ;)

Re: Buffer: Composable Buffers for Go

#8

Post by djherbis »

fishnchips wrote: post #6
kenferry wrote: post #4

As a relative go newb, I do have a question about this: I thought channels/go routines were already essentially composable buffers. No?

Yes and no. Goroutines are lightweight threads - what other languages might call fibers. Channels are synchronisation primitives designed to work with goroutines but they do indeed provide some buffering. What OP is doing though is creating customisable buffers that allow you to control for memory usage and that are likely designed to replace things like bytes.Buffer rather then buffered channels. Of course I may be…

You've got the right idea. bytes.Buffer is great (I even use it under the hood) but this repo is intended to give you more control over how your buffer behaves as the amount of data stored in it changes.

Re: Buffer: Composable Buffers for Go

#9

Post by djherbis »

stinos wrote: post #2

// Buffer 32KB to Memory, after that buffer to 100MB chunked files buf := buffer.NewUnboundedBuffer(32*1024, 100*1024*1024) pool := NewFilePool(100*1024*1024, "") // "" -- use temp dir I seem to have a twofold feeling about this code: on one hand I really like the brevity of these statements, but on the other hand it does come at the cost of being rather cryptic: without the comments one can only guess what exactly i…

Just to add to the other comments made here, the NewUnboundedBuffer is meant to be a convenience method. It's actually just a (hopefully useful) example of how to compose the other buffer types to create a more complicated buffer. Thanks for the comment, hope it helps clear that up.

Re: Buffer: Composable Buffers for Go

#10

Post by TillE »

stinos wrote: post #2

// Buffer 32KB to Memory, after that buffer to 100MB chunked files buf := buffer.NewUnboundedBuffer(32*1024, 100*1024*1024) pool := NewFilePool(100*1024*1024, "") // "" -- use temp dir I seem to have a twofold feeling about this code: on one hand I really like the brevity of these statements, but on the other hand it does come at the cost of being rather cryptic: without the comments one can only guess what exactly i…

That's one problem that named arguments solve pretty nicely. Unfortunately, I don't think Go supports them.
Post reply on HN