Live data from Hacker News

SFTPGo: A Full Featured SFTP Server in Go

github.com

21–30 of 59 posts

Re: SFTPGo: A Full Featured SFTP Server in Go

#21
post #15

This looks like a very interesting project to make a lightweight sftp server, though the dependency on an SQL db makes it a bit harder to install. Go has a really good set of crypto primitives, an excellent ssh implementation maintained by the core team, and a great sftp server and client library which makes this project possible. I recently added an sftp server to rclone: https://rclone.org/commands/rclone_serve_sft…

Thanks for the kind words Nick... And thanks for rclone. :)

Re: SFTPGo: A Full Featured SFTP Server in Go

#22

Part of the utility of SFTP to clients and admins is the fact there are no additional requirements other than SSHD to function. This seems to fly in the face of that. If you are willing to install more software, why not use a more feature filled file server?

I agree, unless some robust solution is needed - I prefer just the daemon and using the CLI to push/pull content

Re: SFTPGo: A Full Featured SFTP Server in Go

#23
post #20
post #18

Earlier quoted context omitted.

On the other hand it requires a C compiler, so it's definitely heavier than a pure go application

Serious question, why hasn't anyone rewritten it in Go? Been awhile since I've looked at it but C to Go doesn't seem that difficult to translate. Edit: just looked at the amalgamation again, it's huge... Still surprised no one has tried it yet that I know of. Pre-processor directives would probably make it even more difficult though. Edit2: and it already works so what's the point, other than an extremely challenging…

Sqlite databases are often not shared between applications. Do you know a similar sql, in process database in pure go (even if not completely similar feature wise ?)

Re: SFTPGo: A Full Featured SFTP Server in Go

#24
post #20
post #18

Earlier quoted context omitted.

On the other hand it requires a C compiler, so it's definitely heavier than a pure go application

Serious question, why hasn't anyone rewritten it in Go? Been awhile since I've looked at it but C to Go doesn't seem that difficult to translate. Edit: just looked at the amalgamation again, it's huge... Still surprised no one has tried it yet that I know of. Pre-processor directives would probably make it even more difficult though. Edit2: and it already works so what's the point, other than an extremely challenging…

Sqlite has been rewritten in both Go and C#.

It's really not that hard. The code is clearly documented and everything works in roughly the same way. Easily achievable by a single developer in a few months.

Sqlite code is emulating classes with structs and methods (they're even called classes in the code). This it's pretty easy to translate into languages with reference and struct support.

Memory allocation is separated into an allocation provider, which is easy to implement in GC languages (just create the struct requested).

The parser and VM opcode file is tricky. The opcodes are parsed from comments in one big file. I'm not a fan of that. But once generated for a particular version, pretty easy to translate.

Testing is another can of worms. Translating the exact version of Tcl, too is the best course of action. This is required to run and pass the (hundreds of thousands of) base line tests. Other correctness tests (in sqlite because of branch coverage requirements for aircraft software) are harder to translate. The prerelease burn in test is also not publicly available.

Preprocessor directives are not used as macros. Rather they are used to enable features. In a port, you can for example choose to exclude WAL or specific optimizations. FTS can be left out if not needed. The different lock implementations can be deferred to the stdlib.

If you interested in the perf overhead, here's a small benchmark of a C# implementation I maintain:

https://drive.google.com/file/d/11Bgfh1WgEreDenosoEdWkYAOb6u...

Re: SFTPGo: A Full Featured SFTP Server in Go

#25
Can this project be used as a library and integrated into an existing program?

I am asking, b/c I am currently working on a little document management system for home-use. It looks like the only sane way to integrate with document scanners is (S)FTP upload or Email (SMPT). I have tried to use off-the-shelf FTP servers for this and inotify to get notified of new uploads. The the solutions work OK, but are hard to setup and rather brittle (and limited to Linux).

Go seems to be the ideal language for this project, because its concurrency model and the ease of distribution (as single binary files).

- Has anyone here experience with building systems like this (integrating via FTP/SMTP)?

- Any recommendations for languages and libraries?

Re: SFTPGo: A Full Featured SFTP Server in Go

#26

Can this project be used as a library and integrated into an existing program? I am asking, b/c I am currently working on a little document management system for home-use. It looks like the only sane way to integrate with document scanners is (S)FTP upload or Email (SMPT). I have tried to use off-the-shelf FTP servers for this and inotify to get notified of new uploads. The the solutions work OK, but are hard to setu…

The linked project seems to be a wrapper around

https://github.com/pkg/sftp

Re: SFTPGo: A Full Featured SFTP Server in Go

#27
post #11

Earlier quoted context omitted.

I'd second this, I've longed struggled to come up with a easy-to-deploy sshd/sftp/chroot configuration that permitted easy database-driven configuration w/o extra shell access. You have to fight a lot of defaults to get this just right. Would the OpenSSH upstream accept patches for an unprivileged sshd/sftp-subsystem to make this easier to use their battle tested code?

Proftpd can do sftp-only out of the box.

I am not sure why I didn't realize that sooner, but that's awesome, thank you for pointing that out.

Re: SFTPGo: A Full Featured SFTP Server in Go

#28

Can this project be used as a library and integrated into an existing program? I am asking, b/c I am currently working on a little document management system for home-use. It looks like the only sane way to integrate with document scanners is (S)FTP upload or Email (SMPT). I have tried to use off-the-shelf FTP servers for this and inotify to get notified of new uploads. The the solutions work OK, but are hard to setu…

The linked project seems to be a wrapper around https://github.com/pkg/sftp

Ah Thx! Should have checked the code myself. The library looks good. Not too much documentation (https://github.com/pkg/sftp/issues/267, https://github.com/pkg/sftp/issues/247), but straight forward and battle tested.

Re: SFTPGo: A Full Featured SFTP Server in Go

#29
post #20

Earlier quoted context omitted.

Serious question, why hasn't anyone rewritten it in Go? Been awhile since I've looked at it but C to Go doesn't seem that difficult to translate. Edit: just looked at the amalgamation again, it's huge... Still surprised no one has tried it yet that I know of. Pre-processor directives would probably make it even more difficult though. Edit2: and it already works so what's the point, other than an extremely challenging…

Sqlite has been rewritten in both Go and C#. It's really not that hard. The code is clearly documented and everything works in roughly the same way. Easily achievable by a single developer in a few months. Sqlite code is emulating classes with structs and methods (they're even called classes in the code). This it's pretty easy to translate into languages with reference and struct support. Memory allocation is separat…

Can you point to the pure Go re-implementation of Sqlite? I was looking for such a thing but never found it. Instead I had to go with a k/v store like Bolt.

Re: SFTPGo: A Full Featured SFTP Server in Go

#30
post #15

This looks like a very interesting project to make a lightweight sftp server, though the dependency on an SQL db makes it a bit harder to install. Go has a really good set of crypto primitives, an excellent ssh implementation maintained by the core team, and a great sftp server and client library which makes this project possible. I recently added an sftp server to rclone: https://rclone.org/commands/rclone_serve_sft…

"though the dependency on an SQL db makes it a bit harder to install"

There's only one table..."users". SQL seems like a bit of overkill for this. Maybe there are future plans to save more state.

Post reply on HN