Live data from Hacker News

Atuin replaces your existing shell history with a SQLite database

github.com

191–198 of 198 posts

Re: Atuin replaces your existing shell history with a SQLite database

#191

Is there a Windows/Powershell port coming anytime soon? Honestly this is the one tool missing from my macOS workflow that I can't cross plat yet.

I think it actually works on Windows/powershell for now. We can't guarantee it because we can't test it, but we have a windows user who is always submitting fixes for windows

I tried it out and I couldn't really figure a way to `atuin init` on Powershell, as it requires one of the supported shells as a parameter.

I've also seen a couple of PRs such as [1] that consider Windows support a dead-end as of 2021, so I assumed it to be a dead-end as well. Maybe this user is on WSL? That'd seemingly work as it can get bash/zsh or anything running there.

[1] https://github.com/ellie/atuin/issues/135

Re: Atuin replaces your existing shell history with a SQLite database

#192
post #7

But at what cost?

What kind of cost? Obviously, backups on the author's server are not something I'd ever do, but the "offline only mode" is there, apparently. TBH I was thinking about doing this for a while now. History of my shell, which now is 3.8MB in size, is one of my competitive advantages as a developer (and a very nice thing to have as a power user). It accumulated steadily since ~2005, and with fzf searching the history is m…

This is interesting, thanks I'm checking it out. I was thinking this could be a flash in the pan thing that would then be annoying to maintain, but obviously global history of everything you've done is definitely a super boon to productivity. How do you handle maintaining this as you transition through jobs, machines, etc? 3.8MB is obviouslly trivially small in size so you could store it on a potato, but like what is your workflow around maintaining these one off ad hoc "developer boost" type tools?

Re: Atuin replaces your existing shell history with a SQLite database

#193
post #15

Earlier quoted context omitted.

I think the advantage of the sqlite database is that you retain more context for any given command (e.g. what the current working directory was, ...) in a structured way (it is a database after all). That stored context can then be used to query the database (e.g. filter the history to only show commands that were executed in the cwd). These queries are the point of using sqlite, not anything security as far as I can…

I just don't see how sqlite is up to this; the problem clearly requires PostgreSQL.

sqlite is fine for this usage.

Re: Atuin replaces your existing shell history with a SQLite database

#195
post #7

Earlier quoted context omitted.

What kind of cost? Obviously, backups on the author's server are not something I'd ever do, but the "offline only mode" is there, apparently. TBH I was thinking about doing this for a while now. History of my shell, which now is 3.8MB in size, is one of my competitive advantages as a developer (and a very nice thing to have as a power user). It accumulated steadily since ~2005, and with fzf searching the history is m…

This is interesting, thanks I'm checking it out. I was thinking this could be a flash in the pan thing that would then be annoying to maintain, but obviously global history of everything you've done is definitely a super boon to productivity. How do you handle maintaining this as you transition through jobs, machines, etc? 3.8MB is obviouslly trivially small in size so you could store it on a potato, but like what is…

> How do you handle maintaining this as you transition through jobs, machines, etc?

Currently, the tool reads ~/.mergerc, which is a JSON file with a list of SSH hosts to SCP history to and from. As long as the history file is in the same place (it tends to be on hosts that I setup, and otherwise I check in default locations) and the host has an entry in ~/.ssh/config, the tool will work. It's really just a wrapper for a few SCP invocations plus a history file (extended) format parser.

Changing servers is just a change in the config file, but it's also helpful for changing jobs, because I can quickly add a bit of filtering before the merging happens. I had to erase some API keys and such a few times, adding `filter` call here: https://github.com/piotrklibert/zsh-merge-hist/blob/master/s... took care of it.

> what is your workflow around maintaining these one off ad hoc "developer boost" type tools?

Good question. I don't have such workflow, at all. When I commit to write something like this, I try to make sure that it has a scope limited enough so that it can be "completed" or "done". In this case, the tool builds on SSH/SCP and a file format that hasn't changed in the last 20 years (at least). So, once I had it working, there was nothing much to do with it after that. The only change I had to do recently was changing `+` to `*` in the parser, because somehow (not sure how, actually) an empty command made it into the file. But that's all I had to do in 5 years time.

I'm not as extreme, but suckless.org philosophy appears to work well here. Here's another example: https://github.com/piotrklibert/nimlock - it's a port, done because I wanted to do something in Nim, but it worked for me for years and I suspect it still works now (after going full remote I stopped needing it). There's nothing much that could break (well, Wayland would break it, but I don't use it), and so there's not much you need to do in terms of maintenance.

As for language choices - these are basically random. I made the zsh-merge-hist in Scala simply because I was interested in Scala back then. I have little tools written in Nim, OCaml, Racket, Elisp, Raku - and even AWK (pretty nice language actually) and shell. That's another reason why making the tools any more complex than what's absolutely necessary would be a problem: the churn in the ecosystems tends to be too high for me to keep track of, especially since I'd need to track 10 of them.

EDIT: I forgot, but obviously the most important "trick" is not giving a shit if these things work for anyone else but me :D

> I'm checking it out

If you have Java installed, `./gradlew installDist` should give you `./build/install/bin/zsh-merge-hist` executable to run. The ~/.mergerc (on the host the tool runs) should look like this:

    {
      "hosts": ["host1"],
      "tempDir": "/tmp/zsh-merge-hist",
      "sourcePath": "mgmnt/zsh_history"
    }
where `sourcePath` is a path to history file relative to the home directory.

Re: Atuin replaces your existing shell history with a SQLite database

#196

I decided to implement this using bash. I have a working prototype running on one machine. If anyone wants to help me refine it, let me know and I'll post what I've done on GitHub.

Please do post. I'd love to have a smaller bash alternative to this tool.

Re: Atuin replaces your existing shell history with a SQLite database

#197

I decided to implement this using bash. I have a working prototype running on one machine. If anyone wants to help me refine it, let me know and I'll post what I've done on GitHub.

Please do post. I'd love to have a smaller bash alternative to this tool.

- Save a copy of your current .bash_history file. You can import its contents into the sqlite "commands" table later.

- Add both files in this gist to $BASH_HOME.

https://gist.github.com/chmaynard/dbcaed11534dc54bdc90856d18...

- Install .bash-preexec.sh in $HOME (see https://github.com/rcaloras/bash-preexec).

- Append this command to your shell startup file (e.g. $BASH_HOME/.bashrc):

    source $BASH_HOME/bash_history.sh
- Open a new bash window. You should see the message "bash-preexec is loaded."

Enjoy! Please share your feedback in the gist comments section.

(Memo to self: Package this up in a repo with a proper README.)

Re: Atuin replaces your existing shell history with a SQLite database

#198
post #15

Earlier quoted context omitted.

I think the advantage of the sqlite database is that you retain more context for any given command (e.g. what the current working directory was, ...) in a structured way (it is a database after all). That stored context can then be used to query the database (e.g. filter the history to only show commands that were executed in the cwd). These queries are the point of using sqlite, not anything security as far as I can…

I just don't see how sqlite is up to this; the problem clearly requires PostgreSQL.

No you know what it requires? A 100 person company with 20 microservices and a ceph cluster. /s

I am not sure how many writes per second you have on your shell history, but sqlite is not only up to the task, everything else is overkill.

Additionally not having to run a database, but having your history in a file has advantages for that usecase as well.

Post reply on HN