Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bash patterns I use weekly

will-keleher.com

Post reply on HN
Print view View original

81–90 of 115 posts

Re: Bash patterns I use weekly

#81

Post by 1vuio0pswjnm7 »

1vuio0pswjnm7 wrote: post #70

To conserve host resources RFC 2616 recommends making multiple HTTP requests over a single TCP connection ("HTTP/1.1 pipelining"). The cURL project said it never properly suported HTTP/1.1 pipelining and in 2019 it said it was removed once and for all. https://daniel.haxx.se/blog/2019/04/06/curl-says-bye-bye-to-... Anyway, curl is not needed. One can write a small program in their language of choice to generate HTTP/…

Heres another way to do it without the subshell, using tr.

       #!/bin/sh
       IFS=/;while read w x y z;do
       v=$(echo x|tr x '\34');
       case $w in http:|https:);;*)exit;esac;
       case $x in "");;*)exit;esac;
       echo $y > .host
       printf '%s\r\n' "GET /$z HTTP/1.1";
       printf '%s\r\n' "Host: $y";
       printf 'Connection: keep-alive'$v$v;done \
       |sed '$s/keep-alive/close/'|tr '\34\34' '\r\n' > .http;
       read x

Re: Bash patterns I use weekly

#82

Post by phist_mcgee »

Izkata wrote: post #79
phist_mcgee wrote: post #73

Earlier quoted context omitted.

I've had to git bisect a react native app in a monorepo before. Took me nearly a day to track down the commit, due to the size of node_modules and the need to do a `pod install` every time.

> and the need to do a `pod install` every time. Phrasing makes it sound like you were doing it manually, so if you didn't know, you can just drop a script in the directory, say, "check.sh": #!/bin/bash if ! pod install; then exit 125 fi do-the-actual-test make it executable, and: git bisect run ./check.sh The special exit code 125 tells bisect "this revision can't be tested, skip it", otherwise it just uses exit cod…

Thanks so much for posting this :D

It did require manual user interaction, so I had to interact with the app for quite a while in between bisect checkouts, the waiting for the installs was more the worse part over the actual tedium of running commands.

Re: Bash patterns I use weekly

#83

Post by spondyl »

masklinn wrote: post #25
michaelhoffman wrote: post #24

I have something like this in my bashrc: preexec () { # shellcheck disable=2034 _CMD_START="$(date +%s)" } trap 'preexec; trap - DEBUG' DEBUG PROMPT_COMMAND="_CMD_STOP=\$(date +%s) let _CMD_ELAPSED=_CMD_STOP-_CMD_START if [ \$_CMD_ELAPSED -gt 5 ]; then _TIME_STR=\" (\${_CMD_ELAPSED}s)\" else _TIME_STR='' fi; " PS1="\n\u@\h \w\$_TIME_STR\n\\$ " PROMPT_COMMAND+="trap 'preexec; trap - DEBUG' DEBUG" Whenever a command ta…

FWIW that is supported by many zsh themes like pure, p9k, p10k, ... (and often enabled by default). Also: REPORTTIME If nonnegative, commands whose combined user and system execution times (measured in seconds) are greater than this value have timing statistics printed for them. which is slightly different but generally useful, and built-in.

Came here to +1 REPORTTIME for zsh

Re: Bash patterns I use weekly

#84

Post by eMSF »

chasil wrote: post #65
edgyquant wrote: post #63

Earlier quoted context omitted.

My problem was that I had a directory with probably 200+ subdirectories each one, and the files and subdirectories below them, had a couple of spaces in the name. I typically use for f in ‘ls’; For operations like that but it was obviously built on windows (but I run steam on Ubuntu) and I never interact with windows so tbh I had never thought of this problem before.

The GNU way for handling files that have inconvenient characters in their names is: find ... -print0 | xargs -0 ... It makes all the problems go away.

Also you can use readarray to store the found filenames in a bash array (to use with a for loop).

Re: Bash patterns I use weekly

#85

Post by 1vuio0pswjnm7 »

themk wrote: post #72
1vuio0pswjnm7 wrote: post #70

To conserve host resources RFC 2616 recommends making multiple HTTP requests over a single TCP connection ("HTTP/1.1 pipelining"). The cURL project said it never properly suported HTTP/1.1 pipelining and in 2019 it said it was removed once and for all. https://daniel.haxx.se/blog/2019/04/06/curl-says-bye-bye-to-... Anyway, curl is not needed. One can write a small program in their language of choice to generate HTTP/…

The curl binary will reuse the TCP connection when fed multiple URLs. Infact it can even use HTTP2 and make the requests in parallel over a single TCP connection. Common pattern I use is to construct URLs with a script and use xargs to feed to curl.

For HTTP/1.1 pipelining, not HTTP/2 which not all websites support, the curl binary must be slower because the program tries to do more than just make HTTP from URLs and send the text over TCP. It tries to be "smart" and that slows it down. But dont't take my word for it, test it.

For example, compare the retrieval speed of the above to something like

     sed -n '/^http/s/^/url=/p' URLs.txt|curl -K- --http1.1

Re: Bash patterns I use weekly

#86

Post by Izkata »

phist_mcgee wrote: post #82
Izkata wrote: post #79

Earlier quoted context omitted.

> and the need to do a `pod install` every time. Phrasing makes it sound like you were doing it manually, so if you didn't know, you can just drop a script in the directory, say, "check.sh": #!/bin/bash if ! pod install; then exit 125 fi do-the-actual-test make it executable, and: git bisect run ./check.sh The special exit code 125 tells bisect "this revision can't be tested, skip it", otherwise it just uses exit cod…

Thanks so much for posting this :D It did require manual user interaction, so I had to interact with the app for quite a while in between bisect checkouts, the waiting for the installs was more the worse part over the actual tedium of running commands.

Since we're in a "bash patterns" thread anyway... the script it's running can do whatever, so let's extend it so it works in your case too, by asking you if it's good or not, so you can still do manual app interaction with "bisect run":

   #!/bin/bash

   if ! pod install; then
       exit 125
   fi

   Q=
   while [[ "$Q" != 'y' && "$Q" != 'n' ]]; do
       read -p 'Is this revision good? [y/n] ' Q
   done

   [[ "$Q" == 'y' ]]

Re: Bash patterns I use weekly

#87

Post by penguin_booze »

If I find myself running a set of commands in parallel, I'd keep a cheap Makefile around: individual commands I want to run in parallel will be written as phony targets:

  all: cmd1 cmd2
  
  cmd1:
      sleep 5
  
  cmd2:
      sleep 10
And then

  make -j[n]

Re: Bash patterns I use weekly

#89

Post by DocTomoe »

l0b0 wrote: post #31

> 1. Find and replace a pattern in a codebase with capture groups > git grep -l pattern | xargs gsed -ri 's|pat(tern)|\1s are birds|g' Or, in IDEA, Ctrl-Shift-r, put "pat(tern)" in the first box and "$1s are birds" in the second box, Alt-a, boom. Infinitely easier to remember, and no chance of having to deal with any double escaping.

What you are doing here is to propose a very specialist approach ("Why not just use a SpaceX Merlin Engine, this is ?") when a slightly more cumbersome general approach ("This is how you get from A to B") was described.

IDEA is nice if you have IDEA.

"But not everyone uses Bash" - very correct (more fond of zsh, personally), but this article is specifically about Bash.

Post reply on HN