Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl vs. Wget

daniel.haxx.se

Post reply on HN
Print view View original

161–170 of 205 posts

Re: Curl vs. Wget

#161

Post by laumars »

foota wrote: post #154
hmpc wrote: post #67

Earlier quoted context omitted.

Actually no. From the manpage: Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

Stupid question, but how do things like this resume from where they left off? Wouldn't the server need to be cooperating in this? Is that build into HTTP?

Yes the server would need to support it. The request is made via a HTTP header (Range[1] IIRC).

Also, I wouldn't consider that a stupid question. :)

[1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#Rang...

Re: Curl vs. Wget

#162

Post by ucho »

JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

> Wget's [..] defaults are more sane

Does it finally use filename from "Content-Disposition" without need for any switches?

Re: Curl vs. Wget

#163

Post by pherq »

timgilbert wrote: post #121
geerlingguy wrote: post #13

Also putting this out there—for nicer REST API interaction on the CLI, and a little more user-friendliness, you might also want to add HTTPie[1] to your toolbelt. It's not going to replace curl or wget usage, but it is a nicer interface in certain circumstances. [1] https://github.com/jkbrzt/httpie

HTTPie is amazing for working with JSON/REST interfaces. It's really succinct and generally designed with user-friendliness in mind, versus supporting every nook and cranny of the HTTP RFCs. It's installable via homebrew for Mac users. Whenever I go back to curl I feel the same minor aggravation I do when moving from homebrew back to apt (want to install? apt-get. Search? apt-cache search. List? dpkg. Remove? apt-get…

On Debian these days, they provide an executable named apt that does all the common stuff in one place (install, search, list (mimicks dpkg), remove, show, etc.).

Re: Curl vs. Wget

#164

Post by awjr »

Useful to know: If you use the Chrome dev tools, in the network tab, you can right click on a request and "Copy as cURL".

Re: Curl vs. Wget

#165

Post by Kurtz79 »

JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

"This doesn't really need to be an "emacs vs. vim" or "tabs vs. spaces"-type dichotomy."

Agreed, and even "emacs vs. vim" or "tabs vs. spaces" do not really need to be "emacs vs. vim" or "tabs vs. spaces"-type dichotomies.

Re: Curl vs. Wget

#166

Post by jsingleton »

manyxcxi wrote: post #107
JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

> This doesn't really need to be an "emacs vs. vim" or "tabs vs. spaces"-type dichotomy: wget and curl do different things well and there's no reason why both shouldn't coexist in one's workflow. Totally agree. I love curl for testing API request/responses manually. It's usually a huge part of navigating my way around a new API that doesn't have a client library available for whatever language I'm using at that time.…

Yes, wget is fantastic for mirroring www and ftp sites and I use it a lot for that purpose. It's magic [0]. I hadn't realized that it didn't support compression though, which might explain why it's so slow in some cases. Not normally a problem as it just runs in the background on a schedule.

Curl supports gzip and deflate. It would be great if support for sdch and br were added too. Brotli is in Firefox 44 and can be enabled in Chrome Canary with a flag. SDCH has been in Chrome for a while and is used on Google servers.

[0] http://www.litkicks.com/WgetMagic

Re: Curl vs. Wget

#167

Post by heinrich5991 »

ucho wrote: post #162
JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

> Wget's [..] defaults are more sane Does it finally use filename from "Content-Disposition" without need for any switches?

That sounds like a security issue, allowing the attacker to name a file like they want.

Re: Curl vs. Wget

#168

Post by heinrich5991 »

falcolas wrote: post #28

My favorite use of wget: mirroring web documentation to my local machine. wget -r -l5 -k -np -p https://docs.python.org/2/ Rewrites the links to point local where appropriate, and the ones which are not local remain links to the online documentation. Makes for a nice, seamless experience while browsing documentation. I also prefer wget to `curl -O` for general file downloads, simply because wget will handle redirects…

For docs specifically, you could also try Zeal, which allows you to browse documentation for quite a few programming languages offline, including Mozilla's JS docs and an HTML spec, as you mentioned in some other comment in this thread.

https://zealdocs.org/

Re: Curl vs. Wget

#169

Post by stephengillie »

jsingleton wrote: post #166
manyxcxi wrote: post #107

Earlier quoted context omitted.

> This doesn't really need to be an "emacs vs. vim" or "tabs vs. spaces"-type dichotomy: wget and curl do different things well and there's no reason why both shouldn't coexist in one's workflow. Totally agree. I love curl for testing API request/responses manually. It's usually a huge part of navigating my way around a new API that doesn't have a client library available for whatever language I'm using at that time.…

Yes, wget is fantastic for mirroring www and ftp sites and I use it a lot for that purpose. It's magic [0]. I hadn't realized that it didn't support compression though, which might explain why it's so slow in some cases. Not normally a problem as it just runs in the background on a schedule. Curl supports gzip and deflate. It would be great if support for sdch and br were added too. Brotli is in Firefox 44 and can be…

The Win64 latest version of curl doesn't seem to support gzip nor deflate. I have to remove those options when copying from Chrome developer tools and pasting into a script. I'd report a bug but their page doesn't seem to have an obvious link.
Post reply on HN