Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How and Why to Log Your Bash History

spin.atomicobject.com

Post reply on HN
Print view View original

51–60 of 139 posts

Re: How and Why to Log Your Bash History

#51

Post by brbsix »

saurik wrote: post #29

https://news.ycombinator.com/item?id=10695305 ^ Here is my comment on a previous post for how I am logging all of my bash history in a logical manner (keeping track of terminals and timestamps, etc.) to an sqlite database.

I've been tempted to do something similar. The context information (e.g. environment, working directory) would be especially helpful, since that's what I'm currently missing. Have you tried https://github.com/umang1912/advanced-shell-history? That was where I planned to start.

Here are a few other projects:

https://github.com/thenewwazoo/bash-history-sqlite # simple shell script to store Bash history into SQLite3 database

https://gist.github.com/pklaus/1e381be8592426568df9 # simple Python script to store Bash history into Pandas dataframe

https://gist.github.com/pklaus/26925cfa1fc6b370e043 # simple Python script to store Bash history in SQLite3 database

https://github.com/fredsmith/history-db # store Bash history in PHP/MySQL web server

https://github.com/joshuacronemeyer/shellsink # store Bash/Zsh history in Python web server

Re: How and Why to Log Your Bash History

#53

Post by developer2 »

jakozaur wrote: post #2

Well given that we copy and paste a lot, by mistake there can be credentials and sensitive data in the bash history... Moreover, bash history doesn't work nicely across different terminal tabs or computers. Does anybody know a tool providing better command line history?

1. Regarding commands with credentials or sensitive info, bash allows you to prefix the command with a space character to omit saving the command to the history. This requires remembering this fact, and of course you get nothing in your history to refer to later.

2. I remember a HN submission for a piece of software (or collection of softwares?) that provided a centralized log of your command history across multiple machines. In this way, if you are working on multiple machines in a cluster, you can later search back through your histories from those machines in a single location. My Google Fu is failing me now though.

Re: How and Why to Log Your Bash History

#55

Post by ams6110 »

teamhappy wrote: post #23

I used to do the same thing to figure out which commands I should create short aliases for. Sounded like a good idea at the time but then I realized that I'm creating a file with an awful lot of interesting information in it and I not getting a lot in return. So I set my HISTSIZE to 1000 which is more than enough for interactive shell use and I don't have to worry about having stuff like "youtube-dl fuckmesilly.com/$…

Servers should have zero history saved on the disk. It gives any intruder an easy place to look for passwords, private keys, etc that may have been accidentally recorded and gives clues about related systems.

If you have administrative stuff you need to do more than once, write a little script or alias for it. Depending on history for this is just lazy.

Re: How and Why to Log Your Bash History

#56

Post by thro1237 »

gdulli wrote: post #34

I go one big step further than this and log everything that comes across the screen. One time it saved me from a crontab -r that wiped out a 100+ line crontab. I had viewed it recently so I just copied it out of my history. On a day to day basis it's more about looking up old queries I typed out, the results of those queries at that time, bash commands and their results, the state of a file I edited at a certain time…

How do you do this?

Re: How and Why to Log Your Bash History

#57

Post by jrochkind1 »

jerf wrote: post #44
efyx wrote: post #8

Why logging to a file when you could just set the HITSIZE variable in your .bashrc ? (plus this will give you ctrl+r search which is a must)

I have many shells open at once, sometimes dozens. Only one of them "wins" when saving history. If Bash has out-of-the-box support for merging multiple shell history, it's not obvious in the man page. And I have them open precisely because they're different contexts and I don't really want them sharing history. If you want a log of everything you run, you need to make it some other way. So, basically, in combination…

I took the bash-history related stuff from https://github.com/mrzool/bash-sensible and have been VERY happy with it. Among other things, it makes sure all your histories from various open terminals get merged, not overwrite each other -- I think maybe just `shopt -s histappend` is enough for that? But I was messing around with my settings regarding history settings for a while tweaking and tweaking, until I found bash-sensible, tried it out cut and paste, and found it was perfect.

## SANE HISTORY DEFAULTS ##

# Append to the history file, don't overwrite it

shopt -s histappend

# Save multi-line commands as one command

shopt -s cmdhist

# Record each line as it gets issued

PROMPT_COMMAND='history -a'

# Huge history. Doesn't appear to slow things down, so why not?

HISTSIZE=500000

HISTFILESIZE=100000

# Avoid duplicate entries

HISTCONTROL="erasedups:ignoreboth"

# Don't record some commands

export HISTIGNORE="&:[ ]*:exit:ls:bg:fg:history"

# Useful timestamp format

HISTTIMEFORMAT='%F %T '

Re: How and Why to Log Your Bash History

#58

Post by nailer »

jrochkind1 wrote: post #54

I don't get it, isn't your bash history already logged to `.bash_history`? What's the point of using PROMPT_COMMAND to send it in addition to another file?

Excellent question! Multiple tabs will squash each other, losing history from Tab B between when tab A was opened and when tab A was closed.

It's actually an insecure default - 'insecure' as in, it's a minor form of data loss.

Then again, so is bash launching scripts without pipefail, variable expansion fail, etc.

It'd be worth fixing all this stuff in the next major bash version.

Re: How and Why to Log Your Bash History

#59

Post by dsfyu404ed »

rietta wrote: post #42
matt-attack wrote: post #36

Just a point on security, many advocate that logging commands is a major security weakness. Similar to why SSH now hashes entries in ~/.ssh/known_hosts by default. The idea is you don't want to provide hints on which remote systems you connect to, as these can offer a springboard to the intruder.

And one has to be mindful of that time you ran: export AWS_CREDENTIAL=xpXfLVsY/77Nr+m1mKmys719h0m2z2BCYSv9d5r That is then an increased risk of breach because it is kept around for a long time. YMMV. Defense in depth, don't use production secrets in development, etc, etc.

I agree.

Too bad the author didn't Grep "password" plus a few lines on either side. Even if you sudo and type outside the prompt once every few hundred attempts it's still gonna turn up a lot after a year or more.

Even the most novice of adversaries would have a field day with the bash history of a lower level IT admin.

Re: How and Why to Log Your Bash History

#60

Post by TeMPOraL »

ams6110 wrote: post #55
teamhappy wrote: post #23

I used to do the same thing to figure out which commands I should create short aliases for. Sounded like a good idea at the time but then I realized that I'm creating a file with an awful lot of interesting information in it and I not getting a lot in return. So I set my HISTSIZE to 1000 which is more than enough for interactive shell use and I don't have to worry about having stuff like "youtube-dl fuckmesilly.com/$…

Servers should have zero history saved on the disk. It gives any intruder an easy place to look for passwords, private keys, etc that may have been accidentally recorded and gives clues about related systems. If you have administrative stuff you need to do more than once, write a little script or alias for it. Depending on history for this is just lazy.

OTOH what about audit trail? Are there any standard solutions for saving commands input at servers without giving person inputting those commands access to the logs?

Also, silly idea for a DOS attack vector: script-spam enough commands to have the audit history consume all available space on server.

Post reply on HN