151–159 of 159 posts
In the olden times we used wget when we wanted to mirror a website. It is a specialized tool. Curl is a general purpose request library with a cli frontend (also used embedded from other programs, or as a standard library API in PHP etc).
I've never seen them as competitors! wget is my goto if I need to download a file now, with the minimum of fuss. curl is used when I need to do something fancy with a url to make it work, or when I'm fiddling with params to make an API work/debug it.
Earlier quoted context omitted.
A trick I've found useful when searching large man pages for a flag --foo is to search for `␣␣--foo` (note the two leading spaces). In my experience this always hits the line where the flag is defined instead of irrelevant mentions of that flag, and it's faster than paging through the manual by hand.
Or for something more flexible: ^ *--foo . But unfortunately this kind of pattern will be defeated by -f, --foo indexes.
Earlier quoted context omitted.
A server configured with a docroot to serve a static site will map requested URLs to hierarchical filesystem paths, but that isn't the only possibility; it's a common but quite loose coupling of ideas. But the filename directive of the content-disposition response header is entirely coupled to the idea of a filename. Therefore, it ought to take precedence.
I don't want the stranger deciding to save the content as .bash_profile or such.
Earlier quoted context omitted.
> It is specifying the `-C` argument with obscure special value of `-` It is not an “obscure special value”. Not only is `-C -` (or `--continue-at -` for the long form) well documented in the correct place in the manual, `-` is a common value in command-line tools (e.g. when specifying that a tool’s input will be STDIN instead of a file).
In what sense is "read offset to continue at from STDIN" a meaningful interpretation of `-C -`? That's not what it does.
Earlier quoted context omitted.
Also add -i which lets wget read URLs from a file. In particular wget -i - which makes it read from standard input, and is very useful in pipelines. curl cannot, AFAIK, do this. People usually suggest using xargs, which is a mediocre substitute because it waits for all the URLs to arrive before invoking curl, giving up any chance at parallelism between the command generating the URLs and the one downloading them.
xargs doesn't have to wait, you can specify the number of items to include in a single sub-command and it'll batch things as they come in. For instance: ds@swann3:~# (for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | xargs -L5 echo 1 2 3 4 5 1 2 3 4 5 6 7 8 9 10 6 7 8 9 10 11 12 [... and so on ...] If the xargs call uses -I then --max-lines=1 is implied anyway. If you replace echo with something that sl…
For an artificial example, change
(for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | xargs -L5 echo
(for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | sort | xargs -L5 echo
----
[1] An unnecessary sort in an ETL process causing the overall wall-clock time to increase significantly
Earlier quoted context omitted.
We were way out in the sticks, so interruptions were very common. Still nothing but dialup, satellite, and LTE (no 5G) out there.
I think our setup was very particular to the UK. We didn't (and still don't?) have free local calls like the US, so we paid per minute for ISPs. Almost all ISPs went through a scheme setup by British Telecom - you could either have free internet but you paid for your calls, or you could pay for your internet, and have access via a freefone number - so effectively flat-rate. But the flat-rate option disconnected after…
That was probably one of the biggest death-bringers for the BBS era, no more long distance calls to get to what you wanted.
> Which was hugely frustrating because we had a voicemail variant that was hosted by the telco, and let you know you had messages waiting by pulsing the dialtone. And my modem did not recognise the pulsed dialtone as a valid dialtone, and refused to connect until we called the number and marked them read.
Yeah, some VM providers in the USA did that too, and it similarly confused modems. It's called "stutter dialtone" here, and the usual fix was to put some delay elements in the dial string, which were commas for Hayes command set modems.
> and why they weren't deathstars
They sure did earn that name! I was so hesitant to switch to HGST for ZFS pools due to my 90s/2000s deathstar experiences. Wouldn't run them in production for a while, of course now that I'm over it and trust them as well as any other enterprise brand, they'll screw it up again!
1. wget can resolve onion links. curl can't(yet). You'll get a
curl: (6) Not resolving .onion address (RFC 7686)
curl -s -A "Mozilla/5.0 (Windows NT 10.0; rv:102.0) Gecko/20100101 Firefox/102.0" https://old.reddit.com/r/GonewildAudible/comments/wznkop/f4m_mi_coño_esta_mojada_summer22tomboy/.json
{"message": "Bad Request", "error": 400}
I've searched the curl manpage and couldn't find a way to solve this. Please help.
I'm having to use `xh --curl` [1] to "fix" the links before I pass them to curl.