> And still better than wget, which only does http 1.0 and this has problems due to lacking a 'host' header. Well I had to check, but this is not true at all: $ nc -l -p 9999 GET / HTTP/1.1 User-Agent: Wget/1.15 (linux-gnu) Accept: */* Host: localhost:9999 Connection: Keep-Alive Also wget can recursively mirror webpages and there are nice options to carefully select contents you want to download. It's quite dated tho…
I love aria2c I use it all the time even as the down loader for ArchLinux's pacman (package manager).
It's also used internally in apt-fast[1] and speeds up the process by an order of magnitude for me.
> And still better than wget, which only does http 1.0 and this has problems due to lacking a 'host' header. Well I had to check, but this is not true at all: $ nc -l -p 9999 GET / HTTP/1.1 User-Agent: Wget/1.15 (linux-gnu) Accept: */* Host: localhost:9999 Connection: Keep-Alive Also wget can recursively mirror webpages and there are nice options to carefully select contents you want to download. It's quite dated tho…
I love aria2c I use it all the time even as the down loader for ArchLinux's pacman (package manager).
multiple connection/source downloads is the most significant feature. Great program.
Those were the early days of HTTP, so no, people didn't really get a lot of HTTP "manually" like that. We used other existing tools to get gopher and ftp though. When I started my work on httpget in 1996, I didn't know about wget and I didn't become aware of it until several years later....
I would figure that by 1997 most people stopped using gopher/archive/veronica. I think I stopped by 1995 or so. now days we use curl/wget for web dev it seems? probably used it for the same thing back then but I also image that lots of people just used their own ad-hoc scripts or whatever; i wish i had firsthand knowledge of that. i used wget to download copies of websites when i found one that was mostly documents/i…
I use wget for plain, old regular downloads a lot. If I need to save a file to a specific place (e.g. datasheets to a doc/ directory in my project) and I have a terminal already open there (which I usually do), doing wget and middle-click is far more convenient than dealing with the browsers' "Save to..." dialog. Especially ever since they started considering a sane download manager to be one of those confusing features that users totally don't want.
Curl is pretty great to use, but be warned it has old smelly code and is probably full of security issues. I wouldn't use it for anything too critical.
It's slightly weird to me that I am older than Curl and Wget. They always seemed like Unix Monoliths to me; I had just assumed they had always existed.
And still better than wget, which only does http 1.0 and this has problems due to lacking a 'host' header. Curl just works.
> And still better than wget, which only does http 1.0 and this has problems due to lacking a 'host' header. Well I had to check, but this is not true at all: $ nc -l -p 9999 GET / HTTP/1.1 User-Agent: Wget/1.15 (linux-gnu) Accept: */* Host: localhost:9999 Connection: Keep-Alive Also wget can recursively mirror webpages and there are nice options to carefully select contents you want to download. It's quite dated tho…
Do note that just a plain wget command isn't enough to properly archive/"back up" a webpage, because it doesn't save headers and other important metadata. Make sure to always use WARCs, which preserve metadata and have wonderful support, including the possibility to get absorbed into the Wayback Machine. More info: