Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl vs. Wget

daniel.haxx.se

Post reply on HN
Print view View original

171–180 of 205 posts

Re: Curl vs. Wget

#171

Post by viperscape »

zo1 wrote: post #160
foota wrote: post #154

Earlier quoted context omitted.

Stupid question, but how do things like this resume from where they left off? Wouldn't the server need to be cooperating in this? Is that build into HTTP?

You specify a byte "range" when requesting a file via HTTP from the server. Not all servers support this, but most do these days. #Edit. More info here: https://en.wikipedia.org/wiki/Byte_serving

Just looked that up, that's a rather limited usefulness. The multi homed usage makes sense but feels out of place really.

Re: Curl vs. Wget

#172

Post by joosters »

viperscape wrote: post #171
zo1 wrote: post #160

Earlier quoted context omitted.

You specify a byte "range" when requesting a file via HTTP from the server. Not all servers support this, but most do these days. #Edit. More info here: https://en.wikipedia.org/wiki/Byte_serving

Just looked that up, that's a rather limited usefulness. The multi homed usage makes sense but feels out of place really.

Why? How could it be more useful? HTTP byte ranges are incredibly flexible, since you one request can specify many byte ranges at once (it's almost too flexible, since a dumb server can easily be overwhelmed by a maliciously complicated request)

It handles the basic case of fetching the remainder of an interrupted download, and can also support partial downloads e.g. for supporting a video stream with the user jumping to different places in the movie.

Re: Curl vs. Wget

#173

Post by emj »

MoSal wrote: post #59

I should probably write a "saldl vs. others" page someday. > Wget supports the Public Suffix List for handling cookie domains, curl does not. This is outdated info. (lib)curl can be built with libpsl support since 7.46.0.

Released on 2015-12-02, so it wont be in many dists for some years. :-)

Re: Curl vs. Wget

#174

Post by MoSal »

contingencies wrote: post #110
MoSal wrote: post #93

Earlier quoted context omitted.

Try saldl[1]. It depends on libcurl. So protocol support should be good and reliable. [1] https://github.com/saldl/saldl

https://github.com/saldl/saldl/wiki/saldl_vs._aria2

FWIW, this page is incomplete and outdated.

For example, `--mirror-url` was implemented. So, it is now possible to download from two sources concurrently.

Re: Curl vs. Wget

#175

Post by notfoss »

agumonkey wrote: post #25
et2o wrote: post #11

After vi vs. emacs, this is truly the great debate of our generation.

Come on people, who already emails with curl. Admit it.

This would be better directed at those who use Outlook. The ones using curl to send mails will be boasting about it ;)

Re: Curl vs. Wget

#176

Post by notfoss »

aria2 is much more reliable when downloading stuff, especially for links which involve redirections.

For example here's a link to download 7zip for windows from filehippo.com.

Results:

* Curl doesn't download it at all.

  curl -O 'http://filehippo.com/download/file/bf0c7e39c244b0910cfcfaef2af45de88d8cae8cc0f55350074bf1664fbb698d/'
gives:

  curl: Remote file name has no length!
* Wget manages to download the file, but with the wrong name.

  wget 'http://filehippo.com/download/file/bf0c7e39c244b0910cfcfaef2af45de88d8cae8cc0f55350074bf1664fbb698d/'
gives:

  2016-03-03 18:08:21 (75.9 KB/s) - ‘index.html’ saved [1371668/1371668]
* aria2 manages to download the file with the correct name with no additional switches.

  aria2c 'http://filehippo.com/download/file/bf0c7e39c244b0910cfcfaef2af45de88d8cae8cc0f55350074bf1664fbb698d/'
gives:

  03/03 18:08:45 [NOTICE] Download complete: /tmp/7z1514-x64.exe

Re: Curl vs. Wget

#177

Post by MoSal »

emj wrote: post #173
MoSal wrote: post #59

I should probably write a "saldl vs. others" page someday. > Wget supports the Public Suffix List for handling cookie domains, curl does not. This is outdated info. (lib)curl can be built with libpsl support since 7.46.0.

Released on 2015-12-02, so it wont be in many dists for some years. :-)

Stability and security comes first. So, let's ship an X years old curl release + patches ;)

Re: Curl vs. Wget

#178

Post by detaro »

stephengillie wrote: post #169
jsingleton wrote: post #166

Earlier quoted context omitted.

Yes, wget is fantastic for mirroring www and ftp sites and I use it a lot for that purpose. It's magic [0]. I hadn't realized that it didn't support compression though, which might explain why it's so slow in some cases. Not normally a problem as it just runs in the background on a schedule. Curl supports gzip and deflate. It would be great if support for sdch and br were added too. Brotli is in Firefox 44 and can be…

The Win64 latest version of curl doesn't seem to support gzip nor deflate. I have to remove those options when copying from Chrome developer tools and pasting into a script. I'd report a bug but their page doesn't seem to have an obvious link.

Github? https://github.com/curl/curl

Re: Curl vs. Wget

#179

Post by baldfat »

JonathonW wrote: post #31

For my usage: * Wget's the interactive, end-user tool, and my go-to if I just need to download a file. For that purpose, its defaults are more sane, its command line usage is more straightforward, its documentation is better-organized, and it can continue incomplete downloads, which curl can't. * Curl's the developer tool-- it's what I'd use if I were building a shell script that needed to download. The command line…

I totally agree except over the years I have been using aria2 more and more instead of wget. aria2 supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink with the same sane wget syntax and defaults.

example

$ aria2c http://yourlink.com/file.*

-x2 allows using 2 connections per host.

https://aria2.github.io/

Re: Curl vs. Wget

#180

Post by hobarrera »

mschuster91 wrote: post #150
manyxcxi wrote: post #107

Earlier quoted context omitted.

> This doesn't really need to be an "emacs vs. vim" or "tabs vs. spaces"-type dichotomy: wget and curl do different things well and there's no reason why both shouldn't coexist in one's workflow. Totally agree. I love curl for testing API request/responses manually. It's usually a huge part of navigating my way around a new API that doesn't have a client library available for whatever language I'm using at that time.…

Use the Chrome Postman extension together with Postman Interceptor. It's really awesome.

I'd just like to mention HttpRequester here, which is a very similar FLOSS addon for Firefox.
Post reply on HN