Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux utils that you might not know

shiroyasha.io

Post reply on HN
Print view View original

141–150 of 159 posts

Re: Linux utils that you might not know

#141

Post by sigjuice »

kefka wrote: post #102
angry_octet wrote: post #90

Earlier quoted context omitted.

Why assume python? It's huge compared to bash or perl. Lots of minimal configuration operational systems won't have python. More likely than go/ruby/node though.

If you're on RH derived systems, you're going to have a python 2.6 installed for rpm and yum to work. This bit us in our ass with centos 6.8 , when another package required python 2.7... If you install 2.7 directly, your system fails with a thousand cuts. A chroot is needed for that, unfortunately.

chroot is unnecessary. You just install python 2.7 in its own directory and run it from there. The system-provided python 2.6 and your desired version of python should co-exist.

As a general rule, never try to replace any program or library supplied by your distribution.

Re: Linux utils that you might not know

#142

Post by majewsky »

Since the article mentions numfmt(1), I'm surprised that no one mentioned units(1) yet. I use it all the time to convert units.

  $ units '4123412312312 bytes' 'tebibytes'
          4123412312312 bytes = 3.7502217 tebibytes
          4123412312312 bytes = (1 / 0.26665091) tebibytes
  $ units "50 miles per gallon" "liters per 100 kilometers"
          reciprocal conversion
          1 / 50 miles per gallon = 4.7042917 liters per 100  kilometers
          1 / 50 miles per gallon = (1 / 0.21257185) liters per 100 kilometers

Re: Linux utils that you might not know

#143

Post by stevenspasbo »

philjr wrote: post #101
brettproctor wrote: post #20

Anyone here will probably enjoy checking out commandlinefu: http://commandlinefu.com Especially looking down the list of all time greats: http://www.commandlinefu.com/commands/browse/sort-by-votes

Bringing this on a small tangent, but I've never been comfortable with 'sudo !!' and I can't really articulate why aside from wanting to be as explicit as possible. , , type sudo and enter is nearly as quick and much more explicit for me.

Both bash and zsh have the option histverify that will expand the !! like so:

> echo "Test"

> sudo !!

> sudo echo "Test"

You can then modify the command if you want to.

Re: Linux utils that you might not know

#144

Post by Pete_D »

kefka wrote: post #102
angry_octet wrote: post #90

Earlier quoted context omitted.

Why assume python? It's huge compared to bash or perl. Lots of minimal configuration operational systems won't have python. More likely than go/ruby/node though.

If you're on RH derived systems, you're going to have a python 2.6 installed for rpm and yum to work. This bit us in our ass with centos 6.8 , when another package required python 2.7... If you install 2.7 directly, your system fails with a thousand cuts. A chroot is needed for that, unfortunately.

I had this problem recently. The solution was to install Python 2.7 as a software collection: https://www.softwarecollections.org/en/scls/rhscl/python27/

This installs it to /opt/rh, and you can run commands that need them with "scl enable python27 mything" or by sourcing /opt/rh/python27/enable in a script to set up PATH, etc.

Re: Linux utils that you might not know

#145

Post by hprotagonist »

marcosdumay wrote: post #134
DanBC wrote: post #132

Earlier quoted context omitted.

>The reason shred writes "multiple times with garbage" is due to magnetic disks where even when overwritten, old bit values could still be extracted with advanced tools. People thought that you needed multiple overwrites for traditional drives but in reality that data was gone after a single overwrite of 0. There were no "advanced techniques" that could recover the data. Some specs suggested multiple passes, but this…

Hum, no. Some time ago you needed multiple uncorrelated overwrites to hide your data. Then as HDDs shrank you strted needing less and less passes up to today that you just need to clear it once. It's not about people thinking unrealistic things. The command was created for defending against a demonstrated attack.

Yeah. You'd be amazed what you can do with a scanning electron microscope. (ie, read individual bit states off after a rewrite.)

Re: Linux utils that you might not know

#146

Post by vog »

chrisper wrote: post #113
angry_octet wrote: post #90

Earlier quoted context omitted.

Why assume python? It's huge compared to bash or perl. Lots of minimal configuration operational systems won't have python. More likely than go/ruby/node though.

You don't need to install go. It has the runtime thing built into the compiled executable.

Same with C. However, the binaries are then architecture dependent, so not really portable either.

Nevertheless, statically linked binaries are indeed a good alternative, and provide good-enough portability in a wide range of cases.

Re: Linux utils that you might not know

#147

Post by adrianmonk »

angry_octet wrote: post #83
dom0 wrote: post #61

Earlier quoted context omitted.

In data=journal mode, data to be written is first written into the journal. Only after the journal is flushed it will be written out to the correct location. Therefore, a crash at any time is fixed by replaying the journal forwards. Note that the ext3/4 journal is a redo log, not an undo log. Old file contents are not copied into the journal on a write. Thus, I don't see why shred should be less effective in data=jou…

There is no reason for blocks that have been fully rewritten to be written back to the same location. In fact, it is faster to write them somewhere convenient near the write head and update the indirect block. So even though only the meta data goes through the log, block locations can change.

I don't know that I'd say there's no reason at all.

For one thing, if you have a contiguous file and you update some (but not all) bytes, putting them back in the original location allows the file to stay contiguous.

Also, if you write the data back into the original location, you don't have to update metadata such as inodes. Now, you may say that's less data, but on a spinning disc, there is some threshold below which the amount of data written doesn't matter much at all and it's the number of seeks that matters more. That is, if it's a choice between a single 50k continuous write or two separate 1k writes in different locations, the single write is probably quicker. (But this falls apart eventually of course.)

Whether these reasons are enough to prefer updating in place is another question, of course. But it's not like there isn't any benefit at all.

Re: Linux utils that you might not know

#148

Post by angry_octet »

chrisper wrote: post #113
angry_octet wrote: post #90

Earlier quoted context omitted.

Why assume python? It's huge compared to bash or perl. Lots of minimal configuration operational systems won't have python. More likely than go/ruby/node though.

You don't need to install go. It has the runtime thing built into the compiled executable.

You do if you want to modify the code... Which is kind of the point.

Re: Linux utils that you might not know

#149

Post by DanBC »

hprotagonist wrote: post #145
marcosdumay wrote: post #134

Earlier quoted context omitted.

Hum, no. Some time ago you needed multiple uncorrelated overwrites to hide your data. Then as HDDs shrank you strted needing less and less passes up to today that you just need to clear it once. It's not about people thinking unrealistic things. The command was created for defending against a demonstrated attack.

Yeah. You'd be amazed what you can do with a scanning electron microscope. (ie, read individual bit states off after a rewrite.)

Someone would have done it by now if it was possible, so where are the published papers?

It's not possible, and it's never been possible unless we're talking about 1970s 24" platters, and we're not talking about those.

Re: Linux utils that you might not know

#150

Post by DanBC »

marcosdumay wrote: post #134
DanBC wrote: post #132

Earlier quoted context omitted.

>The reason shred writes "multiple times with garbage" is due to magnetic disks where even when overwritten, old bit values could still be extracted with advanced tools. People thought that you needed multiple overwrites for traditional drives but in reality that data was gone after a single overwrite of 0. There were no "advanced techniques" that could recover the data. Some specs suggested multiple passes, but this…

Hum, no. Some time ago you needed multiple uncorrelated overwrites to hide your data. Then as HDDs shrank you strted needing less and less passes up to today that you just need to clear it once. It's not about people thinking unrealistic things. The command was created for defending against a demonstrated attack.

> Some time ago you needed multiple uncorrelated overwrites to hide your data.

You really didn't. It's a really persistent myth that there's some secret technique to recover overwritten data. For PC hard drives "blobby bits" has never been a thing. It might have been a thing on 1970s style 24" disc platters, but we're not talking about those.

No one has ever recovered data from a drive that's had a single overwrite of zeros - no software claims to be able to do this, no recovery service claims to be able to do this, there are no published papers that claim to be able to do this (there's one that gets an accuracy of 50%-55% per bit, ie useless).

Since the early 1980s a single overwrite of 0 has been sufficient to destroy data.

And if you're worried about a well funded government agency that has the money to spend on exotic techniques you can do a single pass with random data, or NIST 7 passes, or you destroy the drive.

Post reply on HN