They started in 1991, when a terabyte was an unimaginably huge quantity of data and it was common for anonymous FTP servers like xxx.lanl.gov to request that you not connect until after business hours to avoid interfering with the main purpose of the machines. When I joined the internet in 1992, our 7.5-MHz VAX had a 56-kbps frame-relay link to New Mexico Technet (TECNET on our DECNET), which I think may also have provided LANL's rather beefier internet connection. They started providing WWW access in 1993, before Apache added preforking to NCSA HTTPD, and in fact I think before NCSA HTTPD itself. This means that initially every new HTTP request involved forking a new child process from the HTTP server, which took a few hundred milliseconds. This is the context in which the arXiv's hostile stance toward spidering was established.
I agree that it would be an extremely valuable course of action to seed a series of torrents, since a single torrent wouldn't work; it would have to be replaced every time a new paper was uploaded, fragmenting the swarm enough to render it useless. Also, they could surely use Fastly and permit spidering.