Live data from Hacker News

High Performance SSH/SCP

psc.edu

21–30 of 75 posts

Re: High Performance SSH/SCP

#21

This is cool very cool and I think I'll give it a try, though I'm wary about using a forked SSH so would love to see things land upstream. I've been using mosh now for over a decade and it is amazing. Add on rsync for file transfers and I've felt pretty set. If you haven't checked out mosh, you should definitely do so!

wary (cautious or skeptical), not weary (tired)

At this point, maybe both. :) Can we have a portmanteau?

Re: High Performance SSH/SCP

#22

This is cool very cool and I think I'll give it a try, though I'm wary about using a forked SSH so would love to see things land upstream. I've been using mosh now for over a decade and it is amazing. Add on rsync for file transfers and I've felt pretty set. If you haven't checked out mosh, you should definitely do so!

wary (cautious or skeptical), not weary (tired)

Heh, thank you! Edited

Re: High Performance SSH/SCP

#24
post #2

Any chance this work can be upstreamed into mainline SSH? I'd love to have better performance for SSH, but I'm probably not going to install and remember to use this just for the few times it would be relevant.

I admittedly don't really know how SSH is built but it looks to me like the patch that "makes" it HPN-SSH is already present upstream[1], it's just not applied by default? Nixpkgs seems to allow you to build the pkg with the patch [2]. [1] https://github.com/freebsd/freebsd-ports/blob/main/security/... [2] https://github.com/NixOS/nixpkgs/blob/d85ef06512a3afbd6f9082...

Upstream is either OpenBSD itself or https://github.com/openssh/openssh-portable , not the FreeBSD port. I'm... not sure why nix is pulling the patch from FreeBSD, that's odd.

Re: High Performance SSH/SCP

#25
post #9

Earlier quoted context omitted.

So could this safely be used on Tailscale then ? I’m very curious though also a bit paranoid.

It could safely be used on public internet, all this fearmongering has no basis under it. Better question is 'does it have any actual improvements in day-to-day operations'? Because it seems like it mostly changes up some ciphering which is already very fast.

> It could safely be used on public internet, all this fearmongering has no basis under it.

On what basis are making that claim? Because AFAICT, concern about it being less secure is entirely reasonable and is one of the big caveats to it.

Re: High Performance SSH/SCP

#26
post #3
post #2

Any chance this work can be upstreamed into mainline SSH? I'd love to have better performance for SSH, but I'm probably not going to install and remember to use this just for the few times it would be relevant.

Also upstream is extremely well audited. That's a huge benefit i don't want to loose by using fork.

lose*

Re: High Performance SSH/SCP

#27
post #11
post #2

Any chance this work can be upstreamed into mainline SSH? I'd love to have better performance for SSH, but I'm probably not going to install and remember to use this just for the few times it would be relevant.

I doubt this would ever be accepted upstream. That said if one wants speed play around with lftp [1]. It has a mirror subsystem that can replicate much of rsync functionality in a chroot sftp-only destination and can use multiple TCP/SFTP streams in a batch upload and per-file meaning one can saturate just about any upstream. I have used this for transferring massive postgres backups and then because I am paranoid wh…

Wow, I hadn't heard of this before. You're saying it can "chunk" large files when operating against a remote sftp-subsystem (OpenSSH)?

I often find myself needing to move a single large file rather than many smaller ones but TCP overhead and latency will always keep speeds down.

Re: High Performance SSH/SCP

#28
I don't think it comes as a surprise that you can improve performance by re-implementing ciphers, but what is the security trade-off? Many times, well audited implementations of ciphers are intentionally less performant in order to operate in constant time and avoid side channel attacks. Is it even possible to do constant time operations while being multithreaded?

The only change I see here that is probably harmless and a speed boost is using AES-NI for AES-CTR. This should probably be an upstream patch. The rest is more iffy.

Re: High Performance SSH/SCP

#29
post #11

Earlier quoted context omitted.

I doubt this would ever be accepted upstream. That said if one wants speed play around with lftp [1]. It has a mirror subsystem that can replicate much of rsync functionality in a chroot sftp-only destination and can use multiple TCP/SFTP streams in a batch upload and per-file meaning one can saturate just about any upstream. I have used this for transferring massive postgres backups and then because I am paranoid wh…

Wow, I hadn't heard of this before. You're saying it can "chunk" large files when operating against a remote sftp-subsystem (OpenSSH)? I often find myself needing to move a single large file rather than many smaller ones but TCP overhead and latency will always keep speeds down.

I use lftp a lot because of it's better UI compared to sftp. However, for large files, even with scp I can pin GigE with an old Xeon-D system acting as a server.

Re: High Performance SSH/SCP

#30
post #11
post #2

Any chance this work can be upstreamed into mainline SSH? I'd love to have better performance for SSH, but I'm probably not going to install and remember to use this just for the few times it would be relevant.

I doubt this would ever be accepted upstream. That said if one wants speed play around with lftp [1]. It has a mirror subsystem that can replicate much of rsync functionality in a chroot sftp-only destination and can use multiple TCP/SFTP streams in a batch upload and per-file meaning one can saturate just about any upstream. I have used this for transferring massive postgres backups and then because I am paranoid wh…

The normal answer that I have heard to the performance problems in the conversion from scp to sftp is to use rsync.

The design of sftp is such that it cannot exploit "TCP sliding windows" to maximize bandwidth on high-latency connections. Thus, the migration from scp to sftp has involved a performance loss, which is well-known.

https://daniel.haxx.se/blog/2010/12/08/making-sftp-transfers...

The rsync question is not a workable answer, as OpenBSD has reimplemented the rsync protocol in a new codebase:

https://www.openrsync.org/

An attempt to combine the BSD-licensed rsync with OpenSSH would likely see it stripped out of GPL-focused implementations, where the original GPL release has long standing.

It would be more straightforward to design a new SFTP implementation that implements sliding windows.

I understand (but have not measured) that forcibly reverting to the original scp protocol will also raise performance in high-latency conditions. This does introduce an attack surface, should not be the default transfer tool, and demands thoughtful care.

https://lwn.net/Articles/835962/

Post reply on HN