Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bash patterns I use weekly

will-keleher.com

Post reply on HN
Print view View original

71–80 of 115 posts

Re: Bash patterns I use weekly

#71

Post by DylanSp »

deckard1 wrote: post #48
barbazoo wrote: post #15

> git bisect is the "real" way to do this, but it's not something I've ever needed uh, yeah, you did need it, that's why you came up with "2. Track down a commit when a command started failing". Seriously though, git bisect is really useful to track down that bug in O(log n) rather than O(n).

For the given command, if the assumption is the command failed recently, it's likely faster than bisect. You can start it and go grab a coffee. It's automatic. I wish my usage of bisect were that trivial, though. Usually I need to find a bug in a giant web app. Which means finding a good commit, doing the npm install/start dance, etc. for each round.

As long as you can detect your bug from a script, you can pass the script to git bisect run and it'll run automatically.

Re: Bash patterns I use weekly

#72

Post by themk »

1vuio0pswjnm7 wrote: post #70

To conserve host resources RFC 2616 recommends making multiple HTTP requests over a single TCP connection ("HTTP/1.1 pipelining"). The cURL project said it never properly suported HTTP/1.1 pipelining and in 2019 it said it was removed once and for all. https://daniel.haxx.se/blog/2019/04/06/curl-says-bye-bye-to-... Anyway, curl is not needed. One can write a small program in their language of choice to generate HTTP/…

The curl binary will reuse the TCP connection when fed multiple URLs. Infact it can even use HTTP2 and make the requests in parallel over a single TCP connection. Common pattern I use is to construct URLs with a script and use xargs to feed to curl.

Re: Bash patterns I use weekly

#73

Post by phist_mcgee »

deckard1 wrote: post #48
barbazoo wrote: post #15

> git bisect is the "real" way to do this, but it's not something I've ever needed uh, yeah, you did need it, that's why you came up with "2. Track down a commit when a command started failing". Seriously though, git bisect is really useful to track down that bug in O(log n) rather than O(n).

For the given command, if the assumption is the command failed recently, it's likely faster than bisect. You can start it and go grab a coffee. It's automatic. I wish my usage of bisect were that trivial, though. Usually I need to find a bug in a giant web app. Which means finding a good commit, doing the npm install/start dance, etc. for each round.

I've had to git bisect a react native app in a monorepo before. Took me nearly a day to track down the commit, due to the size of node_modules and the need to do a `pod install` every time.

Re: Bash patterns I use weekly

#74

Post by adityaathalye »

stewartbutler wrote: post #33

I've always had trouble getting `for` loops to work predictably, so my common loop pattern is this: grep -l -r pattern /path/to/files | while read x; do echo $x; done or the like. This uses bash read to split the input line into words, then each word can be accessed in the loop with variable `$x`. Pipe friendly and doesn't use a subshell so no unexpected scoping issues. It also doesn't require futzing around with arr…

Once I discovered functions are pipeline-friendly, I started pipelining all the things. Almost any `for arg` can be re-designed as a `while read arg` with a function.

Here's what it can look like. Some functions I wrote to bulk-process git repos. Notice they accept arguments and stdin:

  # ID all git repos anywhere under this directory
  ls_git_projects ~/src/bitbucket |
      # filter ones not updated for pre-set num days    
      take_stale |
      # proc repo applies the given op. (a function in this case) to each repo
      proc_repos git_fetch
Source: https://github.com/adityaathalye/bash-toolkit/blob/master/bu...

The best part is sourcing pipeline-friendly functions into a shell session allows me to mix-and-match them with regular unix tools.

Overall, I believe (and my code will betray it) functional programming style is a pretty fine way to live in shell!

Re: Bash patterns I use weekly

#76

Post by Something1234 »

marginalia_nu wrote: post #45
marcodiego wrote: post #41

We need a simpler regex format. One that allows easy searching and replacing in source code. Of course, some IDE's already to that pretty well, but I'd like to be able to do it from the command line with a stand alone tool I can easily use in scripts. The simplest thing I know that is able to do that is coccinelle, but even coccinelle is not handy enough.

I think what we really need is one regex format. You have POSIX, PCRE, and also various degrees of needing to double-escape the slashes to get past whatever language you're using the regex in. Always adds a large element of guesswork even when you are familiar with regular expressions.

https://regex101.com/

Re: Bash patterns I use weekly

#77

Post by notatoad »

I really love this style of blog post. Short, practical, no backstory, and not trying to claim one correct way to do anything. Just an unopinionated share that was useful to the author.

It seems like a throwback to a previous time, but honestly can't remember when that was. Maybe back to a time when I hoped this was what blogging could be.

Re: Bash patterns I use weekly

#79

Post by Izkata »

phist_mcgee wrote: post #73
deckard1 wrote: post #48

Earlier quoted context omitted.

For the given command, if the assumption is the command failed recently, it's likely faster than bisect. You can start it and go grab a coffee. It's automatic. I wish my usage of bisect were that trivial, though. Usually I need to find a bug in a giant web app. Which means finding a good commit, doing the npm install/start dance, etc. for each round.

I've had to git bisect a react native app in a monorepo before. Took me nearly a day to track down the commit, due to the size of node_modules and the need to do a `pod install` every time.

> and the need to do a `pod install` every time.

Phrasing makes it sound like you were doing it manually, so if you didn't know, you can just drop a script in the directory, say, "check.sh":

  #!/bin/bash
  if ! pod install; then
      exit 125
  fi
  do-the-actual-test
make it executable, and:

  git bisect run ./check.sh
The special exit code 125 tells bisect "this revision can't be tested, skip it", otherwise it just uses exit code 0 for "good" and 1-127 (except for 125) for "bad".
Post reply on HN