Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Plain text has been around for decades and it’s here to stay

unsung.aresluna.org

Post reply on HN
Print view View original

141–150 of 172 posts

Re: Plain text has been around for decades and it’s here to stay

#141

Post by rmunn »

zzo38computer wrote: post #124
thaumasiotes wrote: post #35

Earlier quoted context omitted.

> Do you prefer using code pages so that the bytes in a file have no meaning unless you also supply code page information? Yes. Note that this is already how Unicode is supposed to work. See e.g. https://en.wikipedia.org/wiki/Byte_order_mark . A file isn't meaningful unless you know how to interpret it; that will always be true. Assuming that all files must be in a preexisting format defeats the purpose of having fil…

> A file isn't meaningful unless you know how to interpret it; that will always be true. There are multiple levels of meaning, though; character encoding is just one part of it. For example, a text file might be plain text, or HTML, or JSON, or a C source code, etc; a binary file might be DER, or IFF, or ZIP, etc; and then there will be e.g. what kind of data a JSON or DER or IFF contains and how that level of the da…

> ... (although I think UTF-8 should not be used for Japanese either) ...

The people putting up websites in Japanese disagree with you, it would seem. According to Wikipedia (in the Shift JIS article), as of March 2026 99% of websites in the .jp domain were in UTF-8, with only 1% being in Shift JIS.

Japan used to have two different encodings in common use, Shift JIS (usually used on Windows) and EUC-JP (more common on Unix servers). This resulted in characters being misinterpreted often enough that they coined the word mojibake to describe the phenomenon of text coming out completely garbled. These days, it seems Japanese website makers are more than happy to accept a slight inefficiency in encoding size, because what they gain from that is never having to see mojibake again.

Re: Plain text has been around for decades and it’s here to stay

#142

Post by ButlerianJihad »

d-us-vb wrote: post #70
carra wrote: post #40

Earlier quoted context omitted.

I read that article long time ago, and for me it's a hard disagree. A system as complex and quirky as Unicode can never be considered "plain", and even today it is common for many apps that something Unicode-related breaks. ASCII is still the only text system that will really work well everywhere, which I consider a must for calling something plain text. And yes, ASCII means mostly limiting things to English but for…

I feel like that isn’t exactly a very useful definition of plaintext. If you mean “ASCII” say ASCII. Plain text is text intended to be interpreted as bytes that map simply to characters. Complexity is irrelevant.

https://en.wikipedia.org/wiki/Plaintext

  With the advent of computing, the term plaintext expanded beyond human-readable documents to mean any data, including binary files, in a form that can be viewed or used without requiring a key or other decryption device. Information—a message, document, file, etc.—if to be communicated or stored in an unencrypted form is referred to as plaintext.
https://csrc.nist.gov/glossary/term/plaintext

    Unencrypted information that may be input to an encryption operation. Note: Plain text is not a synonym for clear text. See clear text.

    Intelligible data that has meaning and can be understood without the application of decryption.

Re: Plain text has been around for decades and it’s here to stay

#143

Post by rmunn »

wongarsu wrote: post #132
rmunn wrote: post #15

Earlier quoted context omitted.

Haven't watched the videos yet, but from the slides, it looks like part of the issue he was talking about was encodings (there's a slide illustrating UTF-16LE ve UTF-16BE, for example). Thankfully, with UTF-8 becoming the default everywhere (so that you need a really good reason not to use it for any given document), we're back at "yes, there is such a thing as plain text" again. It has a much larger set of valid cha…

Until you hit a CSV exported by Excel

I should have said "a text file with no byte-order mark". I would hope that Excel's CSV export, if it's writing UTF-16, is writing a byte-order mark first (though I don't have any Excel-exported CSVs lying around right now to check). The byte-order mark is necessary for UTF-16 since it has big-endian and little-endian variants, but unnecessary (and actually harmful in a few situations) for UTF-8. So naturally, if you assume something is UTF-8 but the first few bytes you encounter are FF FE or FE FF (both of which are illegal in UTF-8) then instead of throwing an error saying "Hey, that's illegal UTF-8, buddy!" you should just reparse in UTF-16 (and you now know the correct byte order to use). In fact, you should read four bytes just to make sure you're not seeing FF FE 00 00, because that would indicate a UTF-32LE document. (Which indicates an ambiguity in UTF-16, that UTF-8 doesn't have. A UTF-16 document that begins with a null byte is likely to be misinterpreted as UTF-32LE).

Before I go off on too much of a rabbit trail, I have two points I want to make:

1. Since UTF-8 should be the default assumption for any sensible software, a byte-order mark is not needed for UTF-8, but any non-UTF-8 encoding should use a byte-order mark. (And in fact needs a BOM, because both UTF-16 and UTF-32 have LE and BE variants).

2. Excel needs to fix its stupid CSV import/export defaults.

Re: Plain text has been around for decades and it’s here to stay

#144

Post by rmunn »

epistasis wrote: post #100

It's fun to see a plaintext accounting view as the example... I just switched from QuickBooks to Beancount+Fava for my sole proprietorship, and couldn't be happier. I've added a text-based simple invoice system, a text-based vehicle mileage tracker, and have validators that ensure that every expense with a tax status has a document attached to it. It's far easier and faster to use than QuickBooks, I don't have to put…

Thanks for mentioning Beancount; I didn't know about it before. I'm American but have a job in a different country, so I deal with two currencies routinely, and haven't found a good way of handling multiple currencies in Gnucash. So my wife and I have been keeping our records in text files. I'll look into what it would take to switch over to Beancount; I bet I could write a conversion script (or get an LLM to write most of it for me, even) that would do 95% of the work since we've been pretty consistent in our format, and throw up warning messages about any entries that it couldn't parse (we haven't necessarily been 100% consistent).

I just might join you in Beancount + Fava land.

Re: Plain text has been around for decades and it’s here to stay

#145

Post by shepherdjerred »

imoverclocked wrote: post #135

Does the community here consider HN as plaintext? Obviously, the site is html with hyperlinks etc ... but it's basically implementing a text interface that I can click on. In a cryptographic sense, html itself is plaintext since it is encoded as ascii/utf-8. However, the MIME type of text/plain is distinct from text/html to describe the html encoding of document style/structure information. A terminal is often consid…

HN is normal HTML, so I think that's fair to call 'plaintext'

Re: Plain text has been around for decades and it’s here to stay

#147

Post by majormajor »

cfiggers wrote: post #52
gf000 wrote: post #49

Earlier quoted context omitted.

But why? You easily have 4k pixels, why use a tiny subset of those in a very inefficient way? We have proper hardware to make a bunch of these computations actually fast, and yet we should stuck with drawing relatively expensive text everywhere? If you only care about the UX of TUIs, that I can stand behind (though mostly as a guideline, it doesn't fit every workflow), but you can do that with a proper GUI just as we…

> If you only care about the UX of TUIs, that I can stand behind This is a confusing concession. Of course we love TUIs because of the UX, what other reason is there? Constraint breeds consistency and consistency breeds coherence. Take 1,000 random TUI designers and 1,000 random GUI designers and plot the variations between them (use any method you like)—the TUI designers will be more tightly clustered together becau…

The gap between vi and emacs is larger than that of any GUI program I use as regularly as I use either of those.

Re: Plain text has been around for decades and it’s here to stay

#148

Post by lofaszvanitt »

godd2 wrote: post #71
lofaszvanitt wrote: post #51

All plaintext bullshit should be eradicated. Fucking useless as a medium when displaying and handling complex tasks.

Counterpoint: Adding in a custom, proprietary interpretation of your data makes your complex task more complicated.

Howso? :)

Re: Plain text has been around for decades and it’s here to stay

#149

Post by robcohen »

rmunn wrote: post #144
epistasis wrote: post #100

It's fun to see a plaintext accounting view as the example... I just switched from QuickBooks to Beancount+Fava for my sole proprietorship, and couldn't be happier. I've added a text-based simple invoice system, a text-based vehicle mileage tracker, and have validators that ensure that every expense with a tax status has a document attached to it. It's far easier and faster to use than QuickBooks, I don't have to put…

Thanks for mentioning Beancount; I didn't know about it before. I'm American but have a job in a different country, so I deal with two currencies routinely, and haven't found a good way of handling multiple currencies in Gnucash. So my wife and I have been keeping our records in text files. I'll look into what it would take to switch over to Beancount; I bet I could write a conversion script (or get an LLM to write m…

Hey I'm working on a rust implementation of Beancount, if you get a chance check it out. https://rustledger.github.io .

I'm also hoping to put together a standards body that can formalize plain-text accounting standards too so others can more easily implement their own systems that conform to the standard.

Re: Plain text has been around for decades and it’s here to stay

#150

Post by 3eb7988a1663 »

rmunn wrote: post #143
wongarsu wrote: post #132

Earlier quoted context omitted.

Until you hit a CSV exported by Excel

I should have said "a text file with no byte-order mark". I would hope that Excel's CSV export, if it's writing UTF-16, is writing a byte-order mark first (though I don't have any Excel-exported CSVs lying around right now to check). The byte-order mark is necessary for UTF-16 since it has big-endian and little-endian variants, but unnecessary (and actually harmful in a few situations) for UTF-8. So naturally, if you…

Another classic from Microsoft -the Language Server Protocol is UTF-16. Could be paying that price for the rest of time.
Post reply on HN