How and Why to Log Your Bash History • Hacker News Board

Re: How and Why to Log Your Bash History

#31

Post by stormbrew » Tue, May 31, 2016, 2:59 PM UTC

tomswartz07 wrote: post #18

ZeroGravitas wrote: post #15

Possibly a good opportunity to ask: On Mac OS X and iTerm2 I've noticed a thing where pasting in commands means they don't get added to the history and using up arrow or search to find them doesn't work. Anyone seen that? It's one of those things that annoys me every time it happens, but I'm always in the middle of a task when it occurs so I never get a chance to figure it out.

Could it be that you're accidentally adding/copying a space at the beginning of the command? I know Linux BASH will ignore the command from history if that's the case, and I think that OSX is similar.

This is controlled by an environment variable, HISTCONTROL:

              A colon-separated list of values controlling how commands are saved on the history list.  If the list of values includes ignorespace, lines which begin with a space character are not
              saved  in  the  history  list.   A  value  of  ignoredups  causes  lines  matching the previous history entry to not be saved.  A value of ignoreboth is shorthand for ignorespace and
              ignoredups.  A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved.  Any value not in the above  list
              is  ignored.   If  HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the value of HISTIGNORE.  The
              second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTCONTROL.

Re: How and Why to Log Your Bash History

#32

Post by yomly » Tue, May 31, 2016, 3:04 PM UTC

I'm a big fan of saving history - trying to remember all the shell based commands we use is a nightmare!

If you're using zsh, a tip I picked up from [0] was to alias all your common commands like

    'cd', 'ls' 'fg'...

to:

    ' cd' ' ls' ' fg' ...

Then add the following line to your zshrc to ignore lines prepended with a space:

    setopt HIST_IGNORE_SPACE

This keeps your history cleaner from any ls, cd, fg inputs you use.

[0] http://chneukirchen.org/blog/archive/2012/02/10-new-zsh-tric...

EDIT: formatting

Re: How and Why to Log Your Bash History

#33

Post by tetraodonpuffer » Tue, May 31, 2016, 3:05 PM UTC

teamhappy wrote: post #23

I used to do the same thing to figure out which commands I should create short aliases for. Sounded like a good idea at the time but then I realized that I'm creating a file with an awful lot of interesting information in it and I not getting a lot in return. So I set my HISTSIZE to 1000 which is more than enough for interactive shell use and I don't have to worry about having stuff like "youtube-dl fuckmesilly.com/$…

not sure about "not getting a lot in return", there have been plenty of times where I have had to hand build something with strange defaults or additional commands, then six months later I am downloading the new version of the source and having the history of what I did back then is a lifesaver. Not to mention the times you remember you edited something somewhere and partially remember where/when but not exactly so you can just fuzzy search vim dirname and filter.

It does depend on what one does on their computer, obviously, but in my opinion for developers having a more or less permanent history with a fuzzy matcher (fzf is what I use) is extremely important.

I personally have bash set up so it saves a new history file each month, and a custom fzf alias to search on all of them, and I wouldn't want to have it any other way, I also treat these history files as confidential and make sure they are not in git etc. and if I have to input any passwords on the command line I just prepend the command with a space so it's not saved

Re: How and Why to Log Your Bash History

#34

Post by gdulli » Tue, May 31, 2016, 3:07 PM UTC

I go one big step further than this and log everything that comes across the screen.

One time it saved me from a crontab -r that wiped out a 100+ line crontab. I had viewed it recently so I just copied it out of my history.

On a day to day basis it's more about looking up old queries I typed out, the results of those queries at that time, bash commands and their results, the state of a file I edited at a certain time, a stack trace, the output of an ls -l command, etc. Anything I ever do in a session.

Some of those have their own logs or ways to capture history, but a way to capture everything at once is much more comprehensive and less sensitive to forgetting to set the size of the bash history on a given machine, archiving the bash/psql history files when a machine goes away, etc.

And besides not having to worry about the availability of distributed history/log data, being able to grep everything at once is invaluable.

Re: How and Why to Log Your Bash History

#35

Post by dahart » Tue, May 31, 2016, 3:10 PM UTC

Damnit, this is so much smarter and easier than what I did!

I've been trying to leverage bash's history, and wanted to avoid putting a file write into my prompt hook. But I have a big kludgy system for exporting bash history separately for each day, de-duping the overlapping history, and trying to make sure bash always logs the command I typed.

I'm still fighting with bash sometimes not saving my history commands. I've done everything recommended in all corners of the Internet, but certain keyboard combos will still cause history lines to not get saved. It's maddening, and the solution here avoids it completely.

Re: How and Why to Log Your Bash History

#36

Post by matt-attack » Tue, May 31, 2016, 3:11 PM UTC

Just a point on security, many advocate that logging commands is a major security weakness. Similar to why SSH now hashes entries in ~/.ssh/known_hosts by default. The idea is you don't want to provide hints on which remote systems you connect to, as these can offer a springboard to the intruder.

Re: How and Why to Log Your Bash History

#37

Post by rbc » Tue, May 31, 2016, 3:11 PM UTC

I frequently review my shell history to pickup past command invocations, but I think it has limits.

At some point, you have to clean things up and create a script for repetitive tasks, or write wrappers to tame complicated command line syntax. I think curl and groff are my favorite examples of commands that have too many options.

Re: How and Why to Log Your Bash History

#38

Post by wolfhumble » Tue, May 31, 2016, 3:13 PM UTC

saurik wrote: post #29

https://news.ycombinator.com/item?id=10695305 ^ Here is my comment on a previous post for how I am logging all of my bash history in a logical manner (keeping track of terminals and timestamps, etc.) to an sqlite database.

Where/how do you call this script?

Re: How and Why to Log Your Bash History

#39

Post by tymm » Tue, May 31, 2016, 3:13 PM UTC

For the zsh users out there: I wrote a zsh plugin [0] a while ago which does a similar thing (writes time, directory and command to a file) + gives you a better (directory sensitive) history search.

[0] https://github.com/tymm/zsh-directory-history

Re: How and Why to Log Your Bash History

#40

Post by 0942v8653 » Tue, May 31, 2016, 3:14 PM UTC

Isn't this going to be pretty slow? It starts 5 subshells for every prompt. You could at least change $(pwd) to $PWD and switch the square brackets to double square brackets.