Live data from Hacker News

The curl-wget Venn diagram

daniel.haxx.se

21–30 of 159 posts

Re: The curl-wget Venn diagram

#21
Seems maybe dated. For example, it excludes both of these from wget in the diagram

> HTTP PUT

wget --method=PUT --body-data=

> proxies ... HTTPS

wget --use-proxy=on --https_proxy=https://example.com

Curl consistently has more options and flexibility, but there's several things on the right side of the venn diagram where wget does have some capability.

Re: The curl-wget Venn diagram

#22

I would also add at least "sane default options", "continues downloads" and "retries on error" to the Wget column. I recently had to write a script that downloads a very large file over a somewhat unreliable connection. The common wisdom among the engineers is that you need to use Wget for this job. I tried using curl but out of the box it could not resume or retry the download. I would have to study the manual and s…

[deleted]

Re: The curl-wget Venn diagram

#23

I would also add at least "sane default options", "continues downloads" and "retries on error" to the Wget column. I recently had to write a script that downloads a very large file over a somewhat unreliable connection. The common wisdom among the engineers is that you need to use Wget for this job. I tried using curl but out of the box it could not resume or retry the download. I would have to study the manual and s…

Both tools have their use-cases. I think with the advent of LLMs like ChatGPT it has also become a lot easier to get the proper command line incantation for whatever tool you're using. Even if you've read the manual previously, it's easy to forget the exact flags you want to use, and validating the generated command line is usually less effort than having to build it up yourself from scratch by reading the manual.

With the modern web, sometimes it's easier to use a tool like Puppeteer from a custom script. Especially if the sites you're interacting with are using a lot of JS.

Re: The curl-wget Venn diagram

#24
Daniel Stenberg is among those rare breed of developers who put their heart and soul into their creation, a fading trait in the modern world of big tech that shadowy developers seem to be replaceable cogs of a money-making machine.

It's as if he treats curl as his mark on the world of IT.

Re: The curl-wget Venn diagram

#25
post #6

What are happy eyeballs in the curl circle ?

A fast(er) & standards compatible way of doing ipv6+ipv4 connections. - https://en.wikipedia.org/wiki/Happy_Eyeballs

Interestingly, it would make more sense for Wget supporting these, since it is more of an "user" agent.

Re: The curl-wget Venn diagram

#26

I would also add at least "sane default options", "continues downloads" and "retries on error" to the Wget column. I recently had to write a script that downloads a very large file over a somewhat unreliable connection. The common wisdom among the engineers is that you need to use Wget for this job. I tried using curl but out of the box it could not resume or retry the download. I would have to study the manual and s…

Both tools have their use-cases. I think with the advent of LLMs like ChatGPT it has also become a lot easier to get the proper command line incantation for whatever tool you're using. Even if you've read the manual previously, it's easy to forget the exact flags you want to use, and validating the generated command line is usually less effort than having to build it up yourself from scratch by reading the manual. Wi…

I just look at my curl / wget note for some sane examples. i cant believe one needed chatgpt to be productive

Re: The curl-wget Venn diagram

#29
post #10
post #9

For me the killer feature of wget is that by default it downloads a file with a name derived from the url. You do: wget url://to/file.htm and a file named "file.htm" appears in your cwd. Using curl, you would have to do curl url://to/file.htm > file.htm or some other, less ergonomical, incantation.

curl -O https://curl.se/docs/manpage.html#-O

Also -OJ : with -O you get the name derived from the URL (the initial one, I think, even if redirects are being followed), with -OJ you get the one from the Content-Disposition header or the final URL, the way browsers do it. Of course, plain -O is safer. (For parity with Wget, you might also want to add -R to set the downloaded file’s mtime according to the Last-Modified header.)

Re: The curl-wget Venn diagram

#30

I would also add at least "sane default options", "continues downloads" and "retries on error" to the Wget column. I recently had to write a script that downloads a very large file over a somewhat unreliable connection. The common wisdom among the engineers is that you need to use Wget for this job. I tried using curl but out of the box it could not resume or retry the download. I would have to study the manual and s…

For downloading large files I would rather just use aria2c TBH.
Post reply on HN