Gonix – Unix tools written in Go • Hacker News Board

Re: Gonix – Unix tools written in Go

#71

Post by laumars » Sat, May 09, 2015, 11:59 AM UTC

misframer wrote: post #34

polegone wrote: post #31

Earlier quoted context omitted.

I'm working on that as well. I just fixed cat to crash and log on error, and I'll be fixing the other commands soon. By the way, I'm wondering how I should go through the file line by line with a reader. I think the most efficient way may be to scan byte by byte from the start (head) or the end (tail), and count until reaching n amount of newlines (or stop if at the end of the file), then print the bytes between the…

Good question. I'm not sure. You might want to seek to the end and move back. You probably shouldn't do it byte-by-byte directly from the file since that's very inefficient. As you can tell, this is already starting to get complicated! Maybe you could try mmaping the file so you could treat it as a []byte.

Last year I was trying to write a Go routine that read a file backwards. I was amazed how unexpectedly difficult that proved to be.

In the end I settled for reading it from the start which worked 99.999% of the time and enabled me to finish the project to the tight deadline I had. But I've always meant to go back and "fix" that code at some point.

Re: Gonix – Unix tools written in Go

#72

Post by lttlrck » Sat, May 09, 2015, 1:03 PM UTC

The line mentioning using gccgo to make the binaries small intrigued me... it worked! I've only written a couple of small tools in go but the size of the binary always bugged me. It's just a shame that setting up cross-compilation with gccgo looks a lot more involved vs. gc.

Re: Gonix – Unix tools written in Go

#73

Post by zserge » Sat, May 09, 2015, 1:04 PM UTC

SSLy wrote: post #62

zserge wrote: post #57

It would be cool to have a "busybox" alternative targeted at Plan9 commands. I personally find Plan9 utils much more logical (and easier to implement!). Something like 9base from suckless, but as a single binary and hopefully in a more modern language (most of the code like sam or rc is not so easy to understand).

> I personally find Plan9 utils much more logical Could you elaborate on that?

Plan9 utils usually have less options and smaller functionality, but you can easily compose them.

"rc": I like it much more than sh for scripting. It's easier to learn and has less pitfalls.

"cat" has no options at all, it just concatenates.

"du": I like plan's "du" utility, which is easier than "find" ("du" simply lists file names recursively, while "find"... Can anyone list all the options of the find utility from memory?).

"rm" is a combination of rm and rmdir (directory is removed only if empty, unless you do "rm -r" to delete it recursively. Only two options: "-r" and "-f"

"tar" is a simple way to do recursive copying instead of cp, scp etc

"sleep" takes seconds only as an argument, but it can be a floating point (no suffixes like in coreutils, e.g. "sleep 1h" or "sleep 3d")

"who" takes no options at all

I'm not really good at plan9 utils, and I still use coreutils much more, but for embedded systems I would prefer to have a tiny number of simple building bricks like "du", "cp", "rm", "tar" etc, and a few smarter commands like awk/sed/grep/rc. I like toybox a lot, but if only that had an rc shell..

Re: Gonix – Unix tools written in Go

#74

Post by cloverich » Sat, May 09, 2015, 2:35 PM UTC

caipre wrote: post #18

Not meaning to bring up a Rust vs Go debate, but since there are a few comments claiming that this is a waste of time, I figured it's worth mentioning the Rust based re-implementation of coreutils: https://github.com/uutils/coreutils/ And another by suckless in plain C: http://git.suckless.org/sbase/tree/README

That re-write lists its license as MIT, not GPLv2. Is that legitimate? e.g. in general, can you re-implement a library with one license and change it to a non-compatible license?

Re: Gonix – Unix tools written in Go

#75

Post by sagichmal » Sat, May 09, 2015, 2:38 PM UTC

Animats wrote: post #46

barakm wrote: post #13

Forget the haters, this is awesome. GNU started as GNU's Not Unix, reimplementing Unix userland for free. The people who are adamant about calling it GNU/Linux are, on some level, remembering that the userspace is historically a reimplementation. Give me a busybox that I can 'go build' and that becomes really quite interesting.

We really need a busybox in Rust or Go. The existing one has licensing problems and security problems, and is built into too many embedded devices.

There is a busybox in Go. https://github.com/surma/gobox

Re: Gonix – Unix tools written in Go

#76

Post by luckydude » Sat, May 09, 2015, 2:49 PM UTC

ezequiel-garzon wrote: post #60

luckydude wrote: post #22

Heh, this brings me back. I was a young guy at Sun, perl 4 was a thing, I actually argued that we should redo /usr/bin in perl. In the days of a 20mhz SPARC. Silly me. Maybe it makes sense now.

I'm curious: what was your main motivation? I can understand it as a worthy challenge, but it would probably have led to a worse performance than the C-based utilities, no?

Simplicity mainly, memory management.

while () print;

makes for a pretty simple core of cat.

At the time I had written a source management system (nselite, morphed into teamware) almost entirely in perl except for one C program that was performance sensitive.

And I just really liked perl4 at the time.

Re: Gonix – Unix tools written in Go

#77

Post by lorddoig » Sat, May 09, 2015, 3:40 PM UTC

cloverich wrote: post #74

caipre wrote: post #18

Not meaning to bring up a Rust vs Go debate, but since there are a few comments claiming that this is a waste of time, I figured it's worth mentioning the Rust based re-implementation of coreutils: https://github.com/uutils/coreutils/ And another by suckless in plain C: http://git.suckless.org/sbase/tree/README

That re-write lists its license as MIT, not GPLv2. Is that legitimate? e.g. in general, can you re-implement a library with one license and change it to a non-compatible license?

From everything I've read I'm pretty sure that if it's a complete rewrite in another language then it's legally "divorced" from the original project. You can own (and ergo license) an implementation of an idea, but you can't own or license the idea itself. Same with patents - you own the how, not the why.

Re: Gonix – Unix tools written in Go

#78

Post by hzhou321 » Sat, May 09, 2015, 4:28 PM UTC

mpdehaan2 wrote: post #12

microcolonel wrote: post #4

https://github.com/uiri/coreutils .>

While I'm not sure if he posts here, I used to work in the same group with Jim Meyering, who maintained/maintains coreutils. Great guy. Anyway, he told some great stories of the complexities of POSIX, and what happens in Solaris when you have directories 20,000 lines deep (and how to do it efficently, and the fun of teaching various coreutils commands about SELinux). Lots of it gets surprisingly low level quick. Core…

People are lazy and we took legacy complexity as granted. Time evolves and rarely any problem looks the same. If for no other values, having a second (and third and beyond) look at legacy complexity will worth all the attention.

Re: Gonix – Unix tools written in Go

#79

Post by polegone » Sat, May 09, 2015, 4:37 PM UTC

barsonme wrote: post #64

microtonal wrote: post #42

Earlier quoted context omitted.

Work together! It would be great if complete coreutils was implemented!

I'd love some help. I'm juggling 3 side projects right now, and it's hard making time for all of them. :)

I linked it in the readme so all those people visiting my project will see yours.

I can't help much myself (you are way ahead of me), but more people should visit your project now.

Re: Gonix – Unix tools written in Go

#80

Post by iagooar » Sat, May 09, 2015, 6:36 PM UTC

polegone wrote: post #21

misframer wrote: post #6

Earlier quoted context omitted.

From a glance at the code [0], it looks like it will read it all into memory first. A better way to do this would be to utilize the io.Reader interface. [0] https://github.com/polegone/gonix/blob/0b65cd4fb9c6c44357d0a...

Thank you. I will look into this.

The strategy that is used by the original GNU coreutils written in C, and the one I used to implement tail with Rust, is to jump to the end of the file, than rewind AVERAGE_CHARS_PER_LINE * NUMBER_OF_LINES_TO_BE_READ, check if enough lines have been read, and repeat until enough lines have been found.

I found the optimal value of AVERAGE_CHARS_PER_LINE to be around 40 characters, but of course it hugely depends on the file being read.