Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl vs. Wget

daniel.haxx.se

Post reply on HN
Print view View original

111–120 of 205 posts

Re: Curl vs. Wget

#111

Post by gutnor »

barbs wrote: post #85
kayone wrote: post #70

Earlier quoted context omitted.

I'm not sure why you feel like that, I switched back to Windows after ~1 year of using OSX, I wouldn't say its "miserable" there is really nothing I could do in OSX's command line that can't do in windows.

What is your Windows command-line environment? Plain CMD prompt or Cygwin? Or something else?

Powershell is very powerful and for windows it beats cygwin/mingw. Not quite sure how it measure against Linux shells running on Linux, but Microsoft has really made a proper shell for windows, too bad it looks so different.

Obviously if you work in a cross-platform environment, cygwin/mingw is still the only thing that will provide you some sort of consistency in your workflow on Windows machines.

Re: Curl vs. Wget

#113

Post by thrownaway2424 »

falcolas wrote: post #28

My favorite use of wget: mirroring web documentation to my local machine. wget -r -l5 -k -np -p https://docs.python.org/2/ Rewrites the links to point local where appropriate, and the ones which are not local remain links to the online documentation. Makes for a nice, seamless experience while browsing documentation. I also prefer wget to `curl -O` for general file downloads, simply because wget will handle redirects…

wget is ludicrously good for mirroring. I use it to mirror the entirety of EDGAR.

For my money another tool that belongs in the toolbox is perl's WWW::Mechanize and its component WWW::Mechanize::Shell.

Re: Curl vs. Wget

#114

Post by vacri »

lomnakkus wrote: post #19
Xophmeister wrote: post #9

Earlier quoted context omitted.

Streaming to stdout is more Unix-y, allowing you to pipe the response into further processes. For example: curl http://api.example.com/json | jq '.["someKey"]' # etc., etc.

It may be more unix-y, but it's less user-friendly if the expectation is to just download a file. EDIT: Wow, surprised by the downvotes. I don't think I said anything controversial (y'know principle of least surprise and all), but maybe I was being a bit too opaque: wget , by virtue of being the first on the scene, built an expectation that $THING_THAT_GETS_URLS would result in a file without any other input/argument…

Nonsense. You can use both tools. 'curl' is 'cat url', and by default is meant to behave like 'cat' - that is, send stuff to STDOUT. 'wget' is 'web get', and gets an object from (only) the web or ftp, and plonks it on your filesystem. They both do exactly what they're supposed to (according to name), by default.

Re: Curl vs. Wget

#115

Post by troyvit »

xd1936 wrote: post #56

wget has the amazing flag `--page-requisites` though, which downloads all of an html documents' css and images that you might need to display it properly. Lifesaver.

wget has another great flag, -k, which changes references to the css, js, and images to absolute URLs, resulting in a 1 page download that still looks like the original page. It's useful for making dummy pages for clients. I wish curl had this for my OSX friends who need the functionality above. Getting a wget binary onto OSX is a pain but curl is there by default.

Re: Curl vs. Wget

#116

Post by brobinson »

JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

curl is the one where I have to remember whether to use -o or -O when trying to download a file with the original filename and just use wget instead because it's faster than reading the curl man page.

Re: Curl vs. Wget

#117

Post by lomnakkus »

vacri wrote: post #114
lomnakkus wrote: post #19

Earlier quoted context omitted.

It may be more unix-y, but it's less user-friendly if the expectation is to just download a file. EDIT: Wow, surprised by the downvotes. I don't think I said anything controversial (y'know principle of least surprise and all), but maybe I was being a bit too opaque: wget , by virtue of being the first on the scene, built an expectation that $THING_THAT_GETS_URLS would result in a file without any other input/argument…

Nonsense. You can use both tools. 'curl' is 'cat url', and by default is meant to behave like 'cat' - that is, send stuff to STDOUT. 'wget' is 'web get', and gets an object from (only) the web or ftp, and plonks it on your filesystem. They both do exactly what they're supposed to (according to name), by default.

> 'curl' is 'cat url',

Whoa... TIL something! I don't know if that's the official etymology, but that's a great mnemonic!

EDIT: ... and yes, I use both tools :).

Re: Curl vs. Wget

#120

Post by iglookid »

thrownaway2424 wrote: post #113
falcolas wrote: post #28

My favorite use of wget: mirroring web documentation to my local machine. wget -r -l5 -k -np -p https://docs.python.org/2/ Rewrites the links to point local where appropriate, and the ones which are not local remain links to the online documentation. Makes for a nice, seamless experience while browsing documentation. I also prefer wget to `curl -O` for general file downloads, simply because wget will handle redirects…

wget is ludicrously good for mirroring. I use it to mirror the entirety of EDGAR. For my money another tool that belongs in the toolbox is perl's WWW::Mechanize and its component WWW::Mechanize::Shell.

How does wget compare with httrack for mirroring?
Post reply on HN