Earlier quoted context omitted.
I thought that `-C -` checks the output file for the offset
Yes it does. How does the wget --continue work?
I don't know
The curl-wget Venn diagram
141–150 of 159 posts
Re: The curl-wget Venn diagram
#142Re: The curl-wget Venn diagram
#143Earlier quoted context omitted.
No, `-C -` is not a flag. It is specifying the `-C` argument with obscure special value of `-`, which causes curl to determine the offset to continue from the output file length. This might be obvious to you if you are well-versed in curl command line, but it's by no means expected or obvious like a simple flag. > But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong o…
> It is specifying the `-C` argument with obscure special value of `-` It is not an “obscure special value”. Not only is `-C -` (or `--continue-at -` for the long form) well documented in the correct place in the manual, `-` is a common value in command-line tools (e.g. when specifying that a tool’s input will be STDIN instead of a file).
In what sense is "read offset to continue at from STDIN" a meaningful interpretation of `-C -`? That's not what it does.
Re: The curl-wget Venn diagram
#144I would also add at least "sane default options", "continues downloads" and "retries on error" to the Wget column. I recently had to write a script that downloads a very large file over a somewhat unreliable connection. The common wisdom among the engineers is that you need to use Wget for this job. I tried using curl but out of the box it could not resume or retry the download. I would have to study the manual and s…
One point where I would argue wget doesn't have a sensible default is on filenames - it really should make --content-disposition the default, at least for single file downloads. Otherwise it will often use the wrong name if there is a redirect or similar in the chain, which seems increasingly common.
I'll argue not trusting the server to dictate the saved file name is the only correct default behavior, and taking the filename from the user input (URL) is reasonable.
Re: The curl-wget Venn diagram
#145Earlier quoted context omitted.
Our dialup used to disconnect every 2 hours by design. wget and wvdial were the only alternative to mail-ordering CDs
We were way out in the sticks, so interruptions were very common. Still nothing but dialup, satellite, and LTE (no 5G) out there.
I think our setup was very particular to the UK. We didn't (and still don't?) have free local calls like the US, so we paid per minute for ISPs.
Almost all ISPs went through a scheme setup by British Telecom - you could either have free internet but you paid for your calls, or you could pay for your internet, and have access via a freefone number - so effectively flat-rate.
But the flat-rate option disconnected after two hours, on the dot. Which was hugely frustrating because we had a voicemail variant that was hosted by the telco, and let you know you had messages waiting by pulsing the dialtone. And my modem did not recognise the pulsed dialtone as a valid dialtone, and refused to connect until we called the number and marked them read.
Which lead to one of my most UK-centric retro stories. I tried to connect to the internet, and it refused to dial. I blew away my wvdial config, and it refused to dial. I blew away my ppp config, and it refused to dial. I grepped / for the error message and it didn't exist. I ended up blowing away my OS (and accidentally installing onto the wrong drive, and blowing away everything non-OS too), and it still wouldn't dial.
So I dragged the modem & extension cord to my mother's PC, and shot off a mail to my preferred mailing list (one hosted by John @ linuxemporium, my preferred source of mail-order distros), and swiftly received the response that in order to be certified by BT to operate on their network, one of the rules equipment had to obey was to refuse to redial the same number x many times. And that all I needed to do was power-cycle the modem. Which I'd done by dragging it upstairs to my mother's PC. And my own machine had been wiped twice over needlessly.
Aside, there was a lady named Helen on that mailing list who knew everything about everything, and is everything I aspire to be today. She had opinions on which harddrives best survived salt/sea air, and why they weren't deathstars. Just an incredible amount of lived experience. I miss mailing lists.
Re: The curl-wget Venn diagram
#146Can anyone explain "happy eyeballs"? Did find one page about it, but wasn't 100% clear what the use case for it being an option was, or where on earth the name came from...
Happy Eyeballs makes a simultaneous connection over IPv4 and IPv6 to an HTTP server, and uses the first connection that gets a server acknowledgement. This is useful because many networks have noticeably different response times for IPv4 and IPv6, and many have one of them configured but not working properly (usually IPv6).
Without Happy Eyeballs web browsers can be slow fetching some web pages, for some users, waiting for a request timeout on IP addresses that don't work before trying one that works, or working but with the slower IP.
It's called Happy Eyeballs because it improves the visible page load time in web browsers for many users.
Re: The curl-wget Venn diagram
#147Earlier quoted context omitted.
Another pet peeve of mine is that curl's URL parser is a lot more strict compared to wget. For example: $ curl -sSLOJ 'example.com/file name.txt' curl: (3) URL using bad/illegal format or missing URL $ curl -sSLOJ 'example.com/file%20name.txt' $ ls file%20name.txt On the other hand, wget (without any additional flags) will produce a file called "file name.txt" for both URLs. Well, technically you also need to add a -…
I thought curl had the ability to encode html entities if you asked it to?
HTML entities (&.....;) are distinct from URL encoding (%..).
Re: The curl-wget Venn diagram
#148Can anyone explain "happy eyeballs"? Did find one page about it, but wasn't 100% clear what the use case for it being an option was, or where on earth the name came from...
Happy Eyeballs makes a simultaneous connection over IPv4 and IPv6 to an HTTP server, and uses the first connection that gets a server acknowledgement. This is useful because many networks have noticeably different response times for IPv4 and IPv6, and many have one of them configured but not working properly (usually IPv6). Without Happy Eyeballs web browsers can be slow fetching some web pages, for some users, waiti…
Thanks...so curl always does this but wget never does?
I got the impression initially it was an option.
Re: The curl-wget Venn diagram
#149Earlier quoted context omitted.
One point where I would argue wget doesn't have a sensible default is on filenames - it really should make --content-disposition the default, at least for single file downloads. Otherwise it will often use the wrong name if there is a redirect or similar in the chain, which seems increasingly common.
I'll argue not trusting the server to dictate the saved file name is the only correct default behavior, and taking the filename from the user input (URL) is reasonable.
A server configured with a docroot to serve a static site will map requested URLs to hierarchical filesystem paths, but that isn't the only possibility; it's a common but quite loose coupling of ideas.
But the filename directive of the content-disposition response header is entirely coupled to the idea of a filename. Therefore, it ought to take precedence.
Re: The curl-wget Venn diagram
#150Earlier quoted context omitted.
I'll argue not trusting the server to dictate the saved file name is the only correct default behavior, and taking the filename from the user input (URL) is reasonable.
A server configured with a docroot to serve a static site will map requested URLs to hierarchical filesystem paths, but that isn't the only possibility; it's a common but quite loose coupling of ideas. But the filename directive of the content-disposition response header is entirely coupled to the idea of a filename. Therefore, it ought to take precedence.
I don't want the stranger deciding to save the content as .bash_profile or such.