Live data from Hacker News

FTP is 50 years old

filestash.app

91–100 of 187 posts

Re: FTP is 50 years old

#91
post #82
post #61

Earlier quoted context omitted.

Just curious on the motivation. Why not run a regular FTP server and have your application periodically look for new files to process? For horizontal scaling, you just take a distributed lock on the file name.

Because nodejs kubernetes modern cloud.

> Because nodejs kubernetes modern cloud.

Not necessarily so. The history of FTP servers is ridden by bugs with practically no exceptions. At some point some folks decided they finally implement a bug-free implementation and even dared to call it "Very Secure FTPd." Needless to say, it turned out it has bugs, too.

As most of these bugs were related to buffer overflows and similar issues, implementing a new FTP server in a safer language is not such a bad idea, and today's JavaScript is efficient enough to make it a reasonably well-working implementation. I pity the author though for the bugs they encounter and workarounds that will need to be implemented.

Re: FTP is 50 years old

#92
post #73

Not that I think we should go back, but I do miss the wildwest days of web development when it was still acceptable to FTP untested code straight to production. For my first dev job we would develop on production using an FTP client to push up changes on save. One day I was writing an SQL UPDATE statement and I forgot to include a WHERE clause. I basically nuked the entire product DB and it took days to recover becau…

Don’t worry, SFTP is the backbone of the US financial system.

But SFTP is nothing like FTP (in terms of protocol).

Re: FTP is 50 years old

#93
post #63

Earlier quoted context omitted.

I thought a byte ment “by eight” What’s a nibble of a byte is 8bits

By the way, the handy word for an 8-bit byte is “octet” (I have not observed octet used in the software industry, only in school. Don’t use it at work or you might come across as an ass...) I was curious and searched HN and found this 2012 comment explaining octet is essentially an anachronism, as bytes have been standardized at 8 bits: https://news.ycombinator.com/item?id=4649528 . Feeling old!

Octet is the word used for bytes in French. Data sizes in French locale are expressed most often in ko, Mo, Go, To ...

Re: FTP is 50 years old

#94
post #6

From rfc801 (transition from NCP to TCP/IP) : "FTP: This is specified in RFC 765. It is very similar to the FTP used with the NCP. The primary differences are that in addition to the changes for Telnet, that the data channel is limited to 8-bit bytes so FTP features to use other transmission byte sizes are eliminated." So FTP is older than the standardisation of byte size :-)

I thought a byte ment “by eight” What’s a nibble of a byte is 8bits

> I thought a byte ment “by eight”

There are differing sizes for bytes. This isn't so common, anymore, but that was the point being made.

> What’s a nibble of a byte is 8bits

A nibble is 4 bits not 8 bits.

Re: FTP is 50 years old

#96
post #63

Earlier quoted context omitted.

I thought a byte ment “by eight” What’s a nibble of a byte is 8bits

By the way, the handy word for an 8-bit byte is “octet” (I have not observed octet used in the software industry, only in school. Don’t use it at work or you might come across as an ass...) I was curious and searched HN and found this 2012 comment explaining octet is essentially an anachronism, as bytes have been standardized at 8 bits: https://news.ycombinator.com/item?id=4649528 . Feeling old!

> I have not observed octet used in the software industry, only in school.

Commonly, the word octet is used to refer to IP addresses, and the permission octets of files in nix systems.

Re: FTP is 50 years old

#97

It's been about a decade since I've used ftp. I'm trying to remember if there is anything it provides that sftp/scp does not? I remember the egress ports being a pain, and encrypting the traffic required ssl certs and being even more of a pain.

> It's been about a decade since I've used ftp. I'm trying to remember if there is anything it provides that sftp/scp does not?

The FTP protocol provides lots of features that SFTP doesn't. However, most of those distinctive features are rarely implemented in FTP clients for Unix-like systems or Windows. It is more common to find them implemented in mainframe or minicomputer FTP implementations:

- Record-oriented files (STRU R). Unix-like systems and Windows don't have any concept of record-oriented files (where the filesystem is aware of record boundaries–on Unix and Windows, record boundaries are an application-level concept only.) Platforms such as IBM mainframe operating systems, and OpenVMS RMS, do have such files, and so FTP implementations for them often support STRU R

- Block mode file transfer (MODE B). This is where transfers are broken into blocks and each block has a header with a length and flags. You need this to transfer STRU R files (especially binary STRU R files). Unix and Windows FTP programs generally only support stream transfer mode (MODE S)

- Flagging data blocks as corrupt (in a MODE B transfer). This was intended to be used when the file being FTPed is being read directly from a tape. If one of the tape blocks has an invalid checksum, and repeated reads fail to read it with a valid checksum, you can transfer what you read from the block, but set a bit in the block header to indicate the data could be corrupt

- Compressed transfers (MODE C). This is like MODE B but blocks can be compressed. Unfortunately the compression is really basic, just run-length encoding. Some non-standardised extensions to FTP add support for better compression formats (e.g. "MODE Z" for zlib), but those generally are adding compression to MODE S not MODE B.

- Built-in ASCII-EBCDIC conversion (TYPE A vs TYPE E). Unfortunately, this is not aware of the existence of different variants of ASCII and EBCDIC ("code pages"), although there are some non-standardised extensions to add that (which IBM mainframe FTP servers/clients commonly implement). There was also an Internet draft to define a TYPE U explicitly for UTF-8, but it never advanced to an RFC [0]

- Metadata to indicate the carriage control format used in a text file ("FORM"). ("Carriage control" is about telling old-fashioned line printers how to print each line.) You can mark a file as using either ASA carriage control [4] (TYPE A A, TYPE E A), or TELNET carriage control (TYPE A T, TYPE E T; this means carriage control using CR, LF, HT, VT, FF, etc). This feature is rarely implemented except on mainframe platforms where this metadata is supported by the filesystem. (It isn't in the FTP standard, but I believe some IBM mainframe FTP clients/servers also support the IBM proprietary 'machine code' [5] carriage control as well.)

- Non-8 bit bytes. This was commonly used with 36-bit operating systems, for example TOPS-10 or TOPS-20, to transfer files made up of 36-bit words (TYPE L 36).

- "Paged files" (STRU P). These are files composed of "pages" (blocks) where each page has a header, you can have sparse files (some pages not present), even different access control for each page. Unfortunately, the specification is overly specific to the needs of TOPS-10, and can't really be used on other platforms that have the same concept but implement it in a different way. RFC1123 recommends not to implement this feature.

- Account numbers (ACCT). This lets you supply an account number, as well as username (USER) and password (PASS). This was used on some mainframe systems, to bill each transfer to a particular account. (The same user may have access to multiple accounts, and this lets them choose which one to bill the transfer to – consider the case of an academic working on multiple research projects simultaneously.)

- Structure Mount (SMNT). This lets you mount a filesystem over FTP. Most commonly used for DOS/Windows FTP servers to change drive letters.

- Store Unique (STOU). Allows you to upload a file without choosing a name for it. The server chooses a unique name for you, and at the end of the transfer tells you the name it chose.

- Allocate space for a file (ALLO). This allows you to allocate disk space for a file upfront before you transfer it. For record-oriented files (STRU R) or page-oriented files (STRU P), also supports specifying the maximum size of a record/page. Especially used in IBM mainframe operating systems where you are expected to say how big your file (dataset) is going to be before you write any data to it. (In principle, this could be implemented on Linux using fallocate, but in practice Linux FTP servers just ignore the ALLO command.)

It is common for IBM mainframe systems to prefer FTPS (FTP over SSL/TLS) over SFTP, because they actively implemented and use some of the above features which are unique to FTP and lacking in SFTP. IBM mainframe FTP servers also often offer lots of proprietary (non-standard) features in their FTP servers, such as setting dataset allocation parameters, submitting batch jobs (JCL), checking batch job status and retrieving batch job output, executing MVS console commands, running SQL queries (especially with DB2). Since FTP commands are ASCII, it is easy to invoke commands not supported by a particular FTP client simply with "QUOTE SITE". That generally isn't possible with SFTP, since its commands are in a binary format.

There are also some interesting FTP extensions defined for use with high-performance computing – GridFTP [1] [2] [3], including an extended version of block mode (MODE E) with support for "striped" transfers, in which multiple connections (possibly even running across different hosts) transfer different parts of a very large file.

[0] https://tools.ietf.org/html/draft-klensin-ftpext-typeu-00

[1] https://www.ogf.org/documents/GFD.20.pdf

[2] https://www.ogf.org/documents/GFD.21.pdf

[3] https://www.ogf.org/documents/GFD.47.pdf

[4] https://en.wikipedia.org/wiki/ASA_carriage_control_character...

[5] https://en.wikipedia.org/wiki/IBM_Machine_Code_Printer_Contr...

Re: FTP is 50 years old

#100

Earlier quoted context omitted.

British naval signal and letter flags codified 1817. Morse code from 1840s. Ham radio still uses it when signal took weak fir voice. Proficiency was required for a ham license until 2006.

Braille is originally from a similar timeframe and carries six bits. Modern enhancements have focused on the protocol above the payload to make it more efficient.

If that counts, then so do Chinese characters (1200 BC or older)
Post reply on HN