Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

High Performance SSH/SCP

psc.edu

Post reply on HN
Print view View original

31–40 of 75 posts

Re: High Performance SSH/SCP

#31

Post by chasil »

wahern wrote: post #13
Almondsetat wrote: post #4

Earlier quoted context omitted.

OpenSSH is from the people at OpenBSD, which means performance improvements have to be carefully vetted against bugs, and, judging by the fact that they're still on fastfs and the lack of TRIM in 2025, that will not happen.

There's nothing inherently slow about UFS2; the theoretical performance profile should be nearly identical to Ext4. For basic filesystem operations UFS2 and Ext4 will often be faster than more modern filesystems. OpenBSD's filesystem operations are slow not because of UFS2, but because they simply haven't been optimized up-and-down the stack the way Ext4 has been Linux or UFS2 on FreeBSD. And of course, OpenBSD's imp…

OpenBSD UFS did have "soft updates" which were some kind of alternative to journaling.

I believe that these were recently removed. Perhaps they don't play well with SMP.

Re: High Performance SSH/SCP

#32

Post by gorgoiler »

josephg wrote: post #2

Any chance this work can be upstreamed into mainline SSH? I'd love to have better performance for SSH, but I'm probably not going to install and remember to use this just for the few times it would be relevant.

There’s a third party ZFS utility (zrepl, I think) that solves this in a nice way: ssh is used as a control channel to coordinate a new TLS connection over which the actual data is sent. It is considerably faster, apparently.

Re: High Performance SSH/SCP

#33

Post by rapier1 »

harvie wrote: post #3
josephg wrote: post #2

Any chance this work can be upstreamed into mainline SSH? I'd love to have better performance for SSH, but I'm probably not going to install and remember to use this just for the few times it would be relevant.

Also upstream is extremely well audited. That's a huge benefit i don't want to loose by using fork.

I do want to say that HPN-SSH is also well audited; you can see the results of CI tests on the github. We also do fuzz testing, static analysis, extensive code reviews, and functionality testing. We build directly on top of OpenSSH and work with them when we can. We don't touch the authentication code and the parallel ciphers are built directly on top of OpenSSL.

I've been developing it for 20+ years and if you have any specific questions I'd be happy to answer them.

Re: High Performance SSH/SCP

#34

Post by rapier1 »

yjftsjthsd-h wrote: post #25
gear54rus wrote: post #10

Earlier quoted context omitted.

It could safely be used on public internet, all this fearmongering has no basis under it. Better question is 'does it have any actual improvements in day-to-day operations'? Because it seems like it mostly changes up some ciphering which is already very fast.

> It could safely be used on public internet, all this fearmongering has no basis under it. On what basis are making that claim? Because AFAICT, concern about it being less secure is entirely reasonable and is one of the big caveats to it.

Concern about it being less secure is fully justified. I'm the lead developer and have been for the past 20 years. I'm happy to answer any questions you might happen to have.

Re: High Performance SSH/SCP

#35

Post by joecool1029 »

This has been around for years (like at least mid-2000’s). Gentoo used to have this patchset available as a USE flag on net-misc/openssh, but some time ago it was moved to net-misc/openssh-contrib (also configurable by useflag).

There are some minor usability bugs and I think both endpoints need to have it installed to take advantage. I remember asking ages ago why it wasn’t upstreamed, there were reasons…

Re: High Performance SSH/SCP

#36

Post by rapier1 »

ollybee wrote: post #18

It's not clear if you need it on both ends to get an advantage?

The bottleneck in SSH is entirely on the receiving side. So as long at the receiver is using HPN-SSH you will see some performance improvements if the BDP of the path exceeds 2MB. Note: because of changes made to OpenSSH in 8.8 the maximum buffer with OpenSSH as the sender is 16MB. In an HPN to HPN connection that maximum receive buffer is 128MB.

Re: High Performance SSH/SCP

#37

Post by rapier1 »

tristor wrote: post #28

I don't think it comes as a surprise that you can improve performance by re-implementing ciphers, but what is the security trade-off? Many times, well audited implementations of ciphers are intentionally less performant in order to operate in constant time and avoid side channel attacks. Is it even possible to do constant time operations while being multithreaded? The only change I see here that is probably harmless…

The parallel ciphers are built using OpenSSL primitives. We aren't reimplementing the cipher itself in anyway. Since counter ciphers use an atomically increasing counter you can precompute the blocks in advance. Which is what we do - we have a cache of ketstream data that is precomputed and we pull the correct block off as needed - this gets around the need to have the application compute the blocks serially which can be a bottleneck at higher throughput rates.

The main performance improvement is from the buffer normalization. This can provide, on the right path, a 100x improvement in throughput performance without any compromise in security.

Re: High Performance SSH/SCP

#38

Post by rapier1 »

josephg wrote: post #2

Any chance this work can be upstreamed into mainline SSH? I'd love to have better performance for SSH, but I'm probably not going to install and remember to use this just for the few times it would be relevant.

I'm the lead developer. I can go into this a bit more when I get from an appointment if people are interested.

Re: High Performance SSH/SCP

#39

Post by joecool1029 »

Zambyte wrote: post #12
gear54rus wrote: post #10

Earlier quoted context omitted.

It could safely be used on public internet, all this fearmongering has no basis under it. Better question is 'does it have any actual improvements in day-to-day operations'? Because it seems like it mostly changes up some ciphering which is already very fast.

I'm not fear mongering. I'm just saying - IF you don't trust it - AND you want to use it => run it on a private network You don't have to trust it for security to use it. Putting services on secure networks when the public doesn't need access is standard practice.

I remember the last time I really cared to look into this was in the 2000’s, I had these wdtv embedded boxes that had a super anemic cpu that doing local copies with scp was slow as hell from the cipher overhead. I believe at the time it was possible to disable ciphers in scp but it was still slower than smbfs. NFS was to be avoided as wifi was shit then and losing connection meant risking system locking up. This of course was local LAN so I did not really care about encryption.

But I don’t miss having those limitations.

Re: High Performance SSH/SCP

#40

Post by nickcw »

The fact that sftp is not the fastest protocol is well known by rclone users.

The main problem is that it packetizes the data and waits for responses, effectively re-implementing the TCP window inside a TCP stream. You can only have so many packets outstanding in the standard SFTP implementation (64 is the default) and the buffers are quite small (32k by default) which gives a total outstanding data of 2MB. The highest transfer rate you can make depends on the latency of the link. If you have 100 ms of latency then you can send at most 20 MB/s which is about 200 Mbit/s - nowhere near filling a fast wide pipe.

You can tweak the buffer size (up to 256k I think) and the number of outstanding requests, but you hit limits in the popular servers quite quickly.

To mitigate this rclone lets you do multipart concurrent uploads and downloads to sftp which means you can have multiple streams operating at 200 Mbit/s which helps.

The fastest protocols are the TLS/HTTP based ones which stream data. They open up the TCP window properly and the kernel and networking stack is well optimized for this use. Webdav is a good example.

Post reply on HN