Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hexyl: A command-line hex viewer

github.com

Post reply on HN
Print view View original

81–90 of 119 posts

Re: Hexyl: A command-line hex viewer

#81

Post by sharkdp »

ByronBates wrote: post #79
pantalaimon wrote: post #34

Earlier quoted context omitted.

It's comparably slow though :p xxd `which hexyl` > /dev/null 0.12s user 0.09s system 99% cpu 0.219 total hexdump `which hexyl` > /dev/null 0.19s user 0.05s system 91% cpu 0.256 total hexyl `which hexyl` > /dev/null 1.69s user 0.39s system 95% cpu 2.175 total

The same author also wrote hyperfine, a tool to compare performance of various program runs. hyperfine './target/release/hexyl ./target/release/hexyl' 'xxd ./target/release/hexyl' 'hexdump ./target/release/hexyl' Benchmark #1: ./target/release/hexyl ./target/release/hexyl Time (mean ± σ): 1.529 s ± 0.028 s [User: 1.476 s, System: 0.050 s] Range (min … max): 1.491 s … 1.581 s 10 runs Benchmark #2: xxd ./target/release…

... and I have already used hyperfine to benchmark hexyl as well :-)

Yes, it's a shame. But I don't think there is too much we can do about it. We have to print much more to the console due to the ANSI escape codes and we also have to do some conditional checks ON EACH BYTE in order to colorize them correctly. Surely there are some ways to speed everything up a little bit, but in the end I don't think its a real issue. Nobody is going to look at 1MB dumps in a console hex viewer (that's 60,000 lines of output!) without restricting it to some region. And if somebody really wants to, he can probably spare 1.5 seconds to wait for the output :-)

Re: Hexyl: A command-line hex viewer

#83

Post by ksherlock »

Sean1708 wrote: post #41
msla wrote: post #39

Earlier quoted context omitted.

Cross-platform terminal coloring was solved in the 1980s with termcap and then terminfo, for every terminal that's even come close to seeing mainstream use and which had colors in the first place. The libraries are part of the ncurses library and are shipped with every OS this stuff would build on to begin with. This also solves the problem of moving the cursor and drawing a screen in general.

Aren't termcap and terminfo linux (or maybe unix) only?

termcap was written by Bill Joy in 1978. terminfo dates back to 1982.

Re: Hexyl: A command-line hex viewer

#84

Post by ksherlock »

msla wrote: post #39
Sean1708 wrote: post #29

Earlier quoted context omitted.

I'm struggling to even find the canonical source code for xxd, let alone a way to provide a patch. You could probably email the address in the man page, but that's from 1997 so I wouldn't be too optimistic about getting a response. And even then it's quite likely that the original author considers xxd finished and would be reluctant to accept patches for it anyway. On top of all that the hard part of this work (cross…

Cross-platform terminal coloring was solved in the 1980s with termcap and then terminfo, for every terminal that's even come close to seeing mainstream use and which had colors in the first place. The libraries are part of the ncurses library and are shipped with every OS this stuff would build on to begin with. This also solves the problem of moving the cursor and drawing a screen in general.

termcap/terminfo styling is limited to features like underline, bold and reverse. No coloring, per se.

Re: Hexyl: A command-line hex viewer

#85

Post by userbinator »

cygned wrote: post #7
Twisol wrote: post #3

I admit, my first thought on seeing the title was "What's wrong with xxd?" This is nice! I like the colorization a lot -- it's like with source code, you don't realize how much you rely on the colors until you get a new computer and need to download syntax definitions for your editor again. Only feature I really think is missing is line numbers. Nicely done!

I have to admit, my first question in mind was how to disable colors because I do not use syntax highlighting at all.

I'm also not a fan of the colouring, especially in this application, because when you're looking at a hexdump chances are you don't care about ASCII or whitespace --- and it's not as if you couldn't tell whether something is in the ASCII range, given the character representation in the right column anyway.

The border lines don't help much either and seem more like decoration; addresses to the left and an indexing header on the top, like the traditional "canonical" hexdump format, would be far more useful.

Re: Hexyl: A command-line hex viewer

#86

Post by userbinator »

sharkdp wrote: post #81
ByronBates wrote: post #79

Earlier quoted context omitted.

The same author also wrote hyperfine, a tool to compare performance of various program runs. hyperfine './target/release/hexyl ./target/release/hexyl' 'xxd ./target/release/hexyl' 'hexdump ./target/release/hexyl' Benchmark #1: ./target/release/hexyl ./target/release/hexyl Time (mean ± σ): 1.529 s ± 0.028 s [User: 1.476 s, System: 0.050 s] Range (min … max): 1.491 s … 1.581 s 10 runs Benchmark #2: xxd ./target/release…

... and I have already used hyperfine to benchmark hexyl as well :-) Yes, it's a shame. But I don't think there is too much we can do about it. We have to print much more to the console due to the ANSI escape codes and we also have to do some conditional checks ON EACH BYTE in order to colorize them correctly. Surely there are some ways to speed everything up a little bit, but in the end I don't think its a real issu…

We have to print much more to the console due to the ANSI escape codes and we also have to do some conditional checks ON EACH BYTE in order to colorize them correctly.

A few extra comparisons and output for each byte shouldn't be that much slower; fortunately the function of this program is extremely well-defined, so we can calculate some estimates. Assuming a billion instructions per second, taking ~1.5s to hexdump ~1 million bytes means each byte is consuming ~1500 instructions to process. In reality the time above is probably on a faster CPU, so that number maybe 2-3x more. That is a shockingly high number just to split a byte into two nybbles (expected to be 1-3 instructions), convert the nybbles into ASCII (~3 instructions), and decide on the colour (let's be very generous and say ~100 instructions.)

The fact that the binary itself is >1MB is also rather surprising, especially given that the source (not familiar with Rust, but still understandable) seems quite small and straightforward.

Re: Hexyl: A command-line hex viewer

#87

Post by teacpde »

sharkdp wrote: post #62
twic wrote: post #59

One ludicrous nitpick: the logo doesn't show anything hexyl, that's 1,2,3,5-tetramethylbenzene. If it was a functional group, it might be 3,4,5-trimethylbenzyl or something like that. Hexyl would look like: R \/\/\/

The logo shows a small part of this molecule: https://en.wikipedia.org/wiki/Hexanitrodiphenylamine .. which is apparently also called "Hexyl". Granted, I was just looking for a short word that starts with "Hex" and I was never good at organic chemistry :-)

Many of your projects have cool logos, how do you make them?

Re: Hexyl: A command-line hex viewer

#88

Post by steveklabnik »

userbinator wrote: post #86
sharkdp wrote: post #81

Earlier quoted context omitted.

... and I have already used hyperfine to benchmark hexyl as well :-) Yes, it's a shame. But I don't think there is too much we can do about it. We have to print much more to the console due to the ANSI escape codes and we also have to do some conditional checks ON EACH BYTE in order to colorize them correctly. Surely there are some ways to speed everything up a little bit, but in the end I don't think its a real issu…

We have to print much more to the console due to the ANSI escape codes and we also have to do some conditional checks ON EACH BYTE in order to colorize them correctly. A few extra comparisons and output for each byte shouldn't be that much slower; fortunately the function of this program is extremely well-defined, so we can calculate some estimates. Assuming a billion instructions per second, taking ~1.5s to hexdump…

Rust binaries can be large because unlike C, the standard library is statically linked, as well as jemalloc. Jemalloc will no longer be the default as of the next release, so that will shave off ~300k...

Re: Hexyl: A command-line hex viewer

#89

Post by [deleted] »

steveklabnik wrote: post #88
userbinator wrote: post #86

Earlier quoted context omitted.

We have to print much more to the console due to the ANSI escape codes and we also have to do some conditional checks ON EACH BYTE in order to colorize them correctly. A few extra comparisons and output for each byte shouldn't be that much slower; fortunately the function of this program is extremely well-defined, so we can calculate some estimates. Assuming a billion instructions per second, taking ~1.5s to hexdump…

Rust binaries can be large because unlike C, the standard library is statically linked, as well as jemalloc. Jemalloc will no longer be the default as of the next release, so that will shave off ~300k...

[deleted]

Re: Hexyl: A command-line hex viewer

#90

Post by fxfan »

If I can make a suggestion - I don't mind the program being slow because of all the logic - how would it be if you didn't give special status to ascii but instead tried to intelligently assume a string encoding for strings and accordingly colored "binary" vs "text" (utf-8, utf-16)
Post reply on HN