Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl vs. Wget

daniel.haxx.se

Post reply on HN
Print view View original

41–50 of 205 posts

Re: Curl vs. Wget

#41

Post by look_lookatme »

falcolas wrote: post #28

My favorite use of wget: mirroring web documentation to my local machine. wget -r -l5 -k -np -p https://docs.python.org/2/ Rewrites the links to point local where appropriate, and the ones which are not local remain links to the online documentation. Makes for a nice, seamless experience while browsing documentation. I also prefer wget to `curl -O` for general file downloads, simply because wget will handle redirects…

N.B., this will make curl redirect by default:

  echo "-L" >> ~/.curlrc

Re: Curl vs. Wget

#42

Post by Freaky »

ars wrote: post #32
Xophmeister wrote: post #9

Earlier quoted context omitted.

Streaming to stdout is more Unix-y, allowing you to pipe the response into further processes. For example: curl http://api.example.com/json | jq '.["someKey"]' # etc., etc.

One nice thing about saving a file vs piping is that wget sets the timestamp on the file based on the remote HTTP headers. curl -o does not.

curl -Ro does, however.

Re: Curl vs. Wget

#43

Post by MatthewPhillips »

wdmeldon wrote: post #37
geerlingguy wrote: post #13

Also putting this out there—for nicer REST API interaction on the CLI, and a little more user-friendliness, you might also want to add HTTPie[1] to your toolbelt. It's not going to replace curl or wget usage, but it is a nicer interface in certain circumstances. [1] https://github.com/jkbrzt/httpie

This is an extremely minor quibble, but the dependency on Python makes me less inclined to use it. I'm stuck on Windows for a lot of the work I do so configuring Python is never fun and there is no one line installation method from what I can tell. Putting it up on chocolatey[0] might be a good idea. Not sure how feasible that is however. [0] https://chocolatey.org/packages?q=httpie

That's true of running just about any command-line tool on Windows. We were promised an ecosystem of Powershell modlets that operate on objects rather than text, but that hasn't really happened.

Re: Curl vs. Wget

#44

Post by dsmithatx »

JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

The article is not coming up for me. Perhaps it's ycombinated. Anyway I agree curl is more of a developer tool although using it to download files is not the first thing that comes to mind. I use it daily to identify caching issues and redirect issues. The -SLIXGET flags in particular are very useful for this.

Re: Curl vs. Wget

#45

Post by wdmeldon »

recursive wrote: post #38
wdmeldon wrote: post #37

Earlier quoted context omitted.

This is an extremely minor quibble, but the dependency on Python makes me less inclined to use it. I'm stuck on Windows for a lot of the work I do so configuring Python is never fun and there is no one line installation method from what I can tell. Putting it up on chocolatey[0] might be a good idea. Not sure how feasible that is however. [0] https://chocolatey.org/packages?q=httpie

I've never needed to do more configuration for python than whatever the installer does. What problem are you having?

I've had enough problems in the past that I try to steer clear at this point. I don't do much with python so it's hard to justify spending much time on it. I think my bigger issue is the lack of being able to do cinst httpie

Re: Curl vs. Wget

#46

Post by ars »

lomnakkus wrote: post #19
Xophmeister wrote: post #9

Earlier quoted context omitted.

Streaming to stdout is more Unix-y, allowing you to pipe the response into further processes. For example: curl http://api.example.com/json | jq '.["someKey"]' # etc., etc.

It may be more unix-y, but it's less user-friendly if the expectation is to just download a file. EDIT: Wow, surprised by the downvotes. I don't think I said anything controversial (y'know principle of least surprise and all), but maybe I was being a bit too opaque: wget , by virtue of being the first on the scene, built an expectation that $THING_THAT_GETS_URLS would result in a file without any other input/argument…

Remove your edit, it makes it more likely for you to get downvotes, not less.

If you post something, stand by it, do not worry about downvotes. I don't really like them, but nevertheless I'm proud when I get a downvote - it means I don't have a hive-mind mentality.

Re: Curl vs. Wget

#47

Post by theoh »

eru wrote: post #35
JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

Looks like a frontend to curl that has wget's defaults would be useful.

Merge the code, hard link the filenames to the same executable and get it to change behavior based on the name it's invoked under.

Re: Curl vs. Wget

#48

Post by ank_the_elder »

wdmeldon wrote: post #37
geerlingguy wrote: post #13

Also putting this out there—for nicer REST API interaction on the CLI, and a little more user-friendliness, you might also want to add HTTPie[1] to your toolbelt. It's not going to replace curl or wget usage, but it is a nicer interface in certain circumstances. [1] https://github.com/jkbrzt/httpie

This is an extremely minor quibble, but the dependency on Python makes me less inclined to use it. I'm stuck on Windows for a lot of the work I do so configuring Python is never fun and there is no one line installation method from what I can tell. Putting it up on chocolatey[0] might be a good idea. Not sure how feasible that is however. [0] https://chocolatey.org/packages?q=httpie

Previously on HN: https://news.ycombinator.com/item?id=10418882 (that's the thread where I first found out about httpie)

Re: Curl vs. Wget

#49

Post by wdmeldon »

MatthewPhillips wrote: post #43
wdmeldon wrote: post #37

Earlier quoted context omitted.

This is an extremely minor quibble, but the dependency on Python makes me less inclined to use it. I'm stuck on Windows for a lot of the work I do so configuring Python is never fun and there is no one line installation method from what I can tell. Putting it up on chocolatey[0] might be a good idea. Not sure how feasible that is however. [0] https://chocolatey.org/packages?q=httpie

That's true of running just about any command-line tool on Windows. We were promised an ecosystem of Powershell modlets that operate on objects rather than text, but that hasn't really happened.

Any time I'm critical of Windows treatment of CLIs, I'm either met with "Why would I need a CLI, it's not the 80s" or "PowerShell is vastly superior to bash". The issue keeping the MS world from adopting adequate text only tools seems to be much more related to developer mindset than anything technical.

Re: Curl vs. Wget

#50

Post by arca_vorago »

Wget is under GPLv3 so thats what I use more often. Sometimes I will use curl in certain cases, but yes, I will use a GPL product over a non-gpl product if given a choice.
Post reply on HN