Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The curl-wget Venn diagram

daniel.haxx.se

Post reply on HN
Print view View original

31–40 of 159 posts

Re: The curl-wget Venn diagram

#31

Post by augusto-moura »

thrdbndndn wrote: post #30
vesinisa wrote: post #13

I would also add at least "sane default options", "continues downloads" and "retries on error" to the Wget column. I recently had to write a script that downloads a very large file over a somewhat unreliable connection. The common wisdom among the engineers is that you need to use Wget for this job. I tried using curl but out of the box it could not resume or retry the download. I would have to study the manual and s…

For downloading large files I would rather just use aria2c TBH.

Aria2c currently looks unmaintained https://github.com/aria2/aria2/pulse

Re: The curl-wget Venn diagram

#32

Post by cinntaile »

bdhcuidbebe wrote: post #26
TheAceOfHearts wrote: post #23

Earlier quoted context omitted.

Both tools have their use-cases. I think with the advent of LLMs like ChatGPT it has also become a lot easier to get the proper command line incantation for whatever tool you're using. Even if you've read the manual previously, it's easy to forget the exact flags you want to use, and validating the generated command line is usually less effort than having to build it up yourself from scratch by reading the manual. Wi…

I just look at my curl / wget note for some sane examples. i cant believe one needed chatgpt to be productive

If chatgpt makes someone more productive, then why not. It's just a tool as any other tool.

Re: The curl-wget Venn diagram

#33

Post by jraph »

scatters wrote: post #20
johnchristopher wrote: post #18

Earlier quoted context omitted.

Yeah, but: wget "url://to/file.htm?uid=foo&q=bar&rnd=4"

`&unused=.htm`. It usually works.

At this point you might as well use the -o option (-o file.htm). It's easier and easier to understand.

I'd prefer wget to be a bit more clever when handling URLs query strings though, but I guess changing this behavior now might break some scripts.

Re: The curl-wget Venn diagram

#34

Post by TheAceOfHearts »

bdhcuidbebe wrote: post #26
TheAceOfHearts wrote: post #23

Earlier quoted context omitted.

Both tools have their use-cases. I think with the advent of LLMs like ChatGPT it has also become a lot easier to get the proper command line incantation for whatever tool you're using. Even if you've read the manual previously, it's easy to forget the exact flags you want to use, and validating the generated command line is usually less effort than having to build it up yourself from scratch by reading the manual. Wi…

I just look at my curl / wget note for some sane examples. i cant believe one needed chatgpt to be productive

You don't need much of anything, it's just another tool to help people figure stuff out. If you've already invested the time in reading through the manuals for these tools then maybe you don't get much out of something like ChatGPT, but consider that there's thousands of new people entering the industry every year and the number of tools which they're expected to learn to use has increased over time.

Re: The curl-wget Venn diagram

#35

Post by [deleted] »

OskarS wrote: post #19
vesinisa wrote: post #13

I would also add at least "sane default options", "continues downloads" and "retries on error" to the Wget column. I recently had to write a script that downloads a very large file over a somewhat unreliable connection. The common wisdom among the engineers is that you need to use Wget for this job. I tried using curl but out of the box it could not resume or retry the download. I would have to study the manual and s…

> I tried using curl but out of the box it could not resume or retry the download. Maybe I'm misunderstanding, but curl has exactly that feature, it's the `-C` flag. If you want retries, there's `--retry`. I find curls defaults pretty sane, personally, I wouldn't want either of those by default for a tool like curl.

[deleted]

Re: The curl-wget Venn diagram

#36

Post by throw0101a »

dwb wrote: post #10
enriquto wrote: post #9

For me the killer feature of wget is that by default it downloads a file with a name derived from the url. You do: wget url://to/file.htm and a file named "file.htm" appears in your cwd. Using curl, you would have to do curl url://to/file.htm > file.htm or some other, less ergonomical, incantation.

curl -O https://curl.se/docs/manpage.html#-O

> curl -O

Yes. But the GP said by default.

Re: The curl-wget Venn diagram

#37

Post by throw0101a »

OskarS wrote: post #16
dwb wrote: post #10

Earlier quoted context omitted.

curl -O https://curl.se/docs/manpage.html#-O

I much prefer the fact that curl doesn't do this by default (but has the option), it much closer matches the behavior of most unix-y tools. Makes it so much easier to put it into pipelines.

What's "better" depends on whether you 'just' want to download a file, or if you want to build an larger workflow.

For me, I usually want to download files and I'm usually not doing any more processing, so I tend to type in *wget" first.

Re: The curl-wget Venn diagram

#38

Post by inopinatus »

enriquto wrote: post #9

For me the killer feature of wget is that by default it downloads a file with a name derived from the url. You do: wget url://to/file.htm and a file named "file.htm" appears in your cwd. Using curl, you would have to do curl url://to/file.htm > file.htm or some other, less ergonomical, incantation.

I’ve always seen this as a misfeature of wget, on the general principle that command-line utilities should write their principal result to stdout unless otherwise instructed.

Re: The curl-wget Venn diagram

#40

Post by em-bee »

jraph wrote: post #33
scatters wrote: post #20

Earlier quoted context omitted.

`&unused=.htm`. It usually works.

At this point you might as well use the -o option (-o file.htm). It's easier and easier to understand. I'd prefer wget to be a bit more clever when handling URLs query strings though, but I guess changing this behavior now might break some scripts.

well, depends on the usecase. sometimes you want the whole url, like when i want to mirror a site and it has stuff like foo.html?page=1 foo.html?page=2 ...

wget does have options to use the name proposed by the server, and so another option to remove the query arguments would be useful, and in line with those.

Post reply on HN