Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The curious case of the missing period

tjaart.substack.com

Post reply on HN
Print view View original

181–190 of 201 posts

Re: The curious case of the missing period

#181

Post by CRConrad »

Biganon wrote: post #75
aftoprokrustes wrote: post #60

I will not comment on the technical part, as others already did it better than I could, but it just reminded me of an anecdote that reminds of the importance of such trivial things as a period at the end of a sentence: In Germany, where I work, it is usual at the end of employement to ask for a letter of recommendation ("Zeugnis") that lists the tasks performed, and how good the employee was. It is an important docum…

Secret codes being used in recommandation letters are an urban legend. HR people have no incentive to create a secret code for them and their potential rivals, let alone teach it to new HR people while also keeping it secret. This legend comes from the fact that HR people cannot be too explicit about the fact that you've been a pain in the ass (you could probably sue if it's too transparent), so if they have nothing…

You skipped over the fact that "bless their heart" itself is (or at least used to be, before it became too well-known to really be a “secret” any more) precisely such a secret code. (Like, probably, most “HR talk”.)

Re: The curious case of the missing period

#183

Post by TeMPOraL »

throw10920 wrote: post #144
TeMPOraL wrote: post #140

Earlier quoted context omitted.

No, it's not reductionist and pedantic. It's a reminder that there is no magic. Building an abstraction layer that separates control and data doesn't win you anything if, like the people in the article, you then forget it's a thing and write directly to the level below it.

> No, it's not reductionist and pedantic. It's very reductionistic, because it intentionally ignores meaningful detail, and it's pedantic because it's making a meaningless distinction. > It's a reminder that there is no magic. This is irrelevant. Nobody is claiming that there's any magic. I'm pointing out the true fact that details about the abstraction layers matter . In this case, the abstraction layer was poorly-d…

> Good abstraction layer: length prefix, or JSON encoding.

> Bad abstraction layer: (...)

In this context, it shouldn't matter. Sure, "mostly plaintext except some characters in some special positions..." is considered bad in modern engineering practice, however it's not fundamentally different or more difficult that printf and family. You wouldn't start calling printf without at least skimming the docs for the format string language, would you?

> It is a bad engineering decision, and it also obfuscates the fact that there even is an abstraction layer unless you carefully read the spec.

There's the rub: you should have read the spec. You should always read the spec, at least if you're doing something serious like production-grade software. With a binary or JSON-based protocol, you wouldn't look at few messages and assume you understand the encoding. I suppose we can blame SMTP for design that didn't account for human nature: it looks simple enough to fool people into thinking they don't need to read the manual.

> There are very few good reasons to use a text-based data interchange format.

If you mean text without obvious and well-defined structure, then I completely agree.

> One of them is to make the format self-documenting, such that people can easily read and write it without consulting the spec.

"Self-documenting" is IMHO a fundamentally flawed idea, and expecting people to read and write code/markup without consulting the spec is a fool's errand.

> it should be binary - then you have to either read the spec or use someone else's implementation.

That's mitigating (and promoting) bad engineering practice with protocol design; see above. I'm not a fan of this, nor the more general attitude of making tools "intuitive". I'd rather promote the practice of reading the goddamn manual.

> But there's no excuse for the brain-dead approach that SMTP took. They didn't even use length prefixing,

The protocol predates both JSON and XML by several decades. It was created in times when C was roaming the world; length prefixing got unpopular then, and only recently seems to en vogue.

Re: The curious case of the missing period

#184

Post by TeMPOraL »

treflop wrote: post #176
pja wrote: post #170

Earlier quoted context omitted.

If your metadata about the data is in the same channel as the data then you’re doing in-band signalling.

Not in modern times. The terms harken back from the day of circuit switched networks but now that we have heavily transitioned to packets, bands are an artificial construct on top of packets and applying the term isn’t very clear cut. The main property of in-band data in the circuit-switched network days is that you could inject commands into your data stream. If we apply that criteria that to a modern protocol, even…

That's only true if you're not breaking the protocol abstraction layer. There is no "out-of-band" once you serialize your messages. If you start injecting random bytes into the data stream on the wire, you can absolutely start introducing commands, or confuse the receiver where the next piece of metadata/control is.

In this case, somewhere the protocol abstraction layer got broken, and the message text ended up being treated as already serialized. It's not a problem with the protocol per se, but with bad implementation of its API (or no implementation at all, just printf-ing into the wire format).

Re: The curious case of the missing period

#185

Post by TeMPOraL »

spankalee wrote: post #154
TeMPOraL wrote: post #122

Earlier quoted context omitted.

The real problem isn't the protocol, but the cowboy approach to interacting with it. It's not hard to "accept text and encode it properly per the SMTP protocol", you just need to realize you need to do it in the first place. There is a multitude of classes of errors and security vulnerabilities, including "SQL injection", XSS, and similar, that are all caused by the same mistake that this case of missing period was[0…

> [1] - Templating systems themselves are thus a mistake belonging to this class This is not universally true. JavaScript has an amazing feature called tagged template literals which let you tag a string with interpolations with a function that handles the literal and interpolation parts separately. This lets the tag function handle the literals as trusted developer written HTML or SQL, and the interpolations as untr…

That's exactly the kind of hack that worries me. Your example is still (seemingly[0]) gluing text at serialized level, ignoring the actual structure of HTML language. ${name} should never be able to insert any text that would end up being interpreted as markup. Not only when some code decides it's not user-provided; it's not even possible to make that test be 100% accurate, and it doesn't protect you from mistakes in "trusted" strings (like totally trusted `name` having a stray '>' in it).

The bulletproof way of doing this is working at the level of abstraction of your target language. With HTML, that would be a tree structure. For example, if your HTML generation looks more like:

  ["H1", "Hello, " + name]
and that is passed to code that actually builds up the tree and then serializes it down to HTML, then there is no way `name` could ever break the structure or inject anything.

--

[0] - I skimmed the docs of Lit, it seems there are restrictions on where interpolation can be placed, but I don't think they're actually building up the tree expressed by the static parts.

Re: The curious case of the missing period

#187

Post by throw10920 »

Animats wrote: post #178
throw10920 wrote: post #103

Earlier quoted context omitted.

The real problem is that SMTP is a "plain-text" protocol that includes in-band signaling. It literally happened because SMTP defined "a line that only contains a single period in it" as a control sequence and not a literal line that only contains a single period in it. SMTP is an example of an unnecessarily complex design, and the implementation bugs reflect it. SMTP shouldn't be hard for someone to correctly impleme…

SMTP is that way because running the entire contents of a message through some escape character processing was expensive back then, in the era of 0.25 MIPS machines.

Thank you for weighing in, John Nagle!

I understand that constraint, and it seems reasonable - but in that case, why not use a length prefix? That should be even more efficient than having to scan for a line containing a single period and nothing else.

Re: The curious case of the missing period

#188

Post by treflop »

TeMPOraL wrote: post #184
treflop wrote: post #176

Earlier quoted context omitted.

Not in modern times. The terms harken back from the day of circuit switched networks but now that we have heavily transitioned to packets, bands are an artificial construct on top of packets and applying the term isn’t very clear cut. The main property of in-band data in the circuit-switched network days is that you could inject commands into your data stream. If we apply that criteria that to a modern protocol, even…

That's only true if you're not breaking the protocol abstraction layer. There is no "out-of-band" once you serialize your messages. If you start injecting random bytes into the data stream on the wire, you can absolutely start introducing commands, or confuse the receiver where the next piece of metadata/control is. In this case, somewhere the protocol abstraction layer got broken, and the message text ended up being…

Injecting random data into any protocol will break it.

When we’re talking about whether someone can inject data into the link, we’re talking about the end user and not the software. If we’re talking protocol design, then you wouldn’t want regular data to be able to inject commands by simply existing.

Re: The curious case of the missing period

#189

Post by FabHK »

HankB99 wrote: post #98
Biganon wrote: post #80

Earlier quoted context omitted.

Please refrain from willingly picking the naive interpretation when you've understood my point perfectly fine, it's against the rules of this website. ...sigh: Secret codes as in "watermark-level omission of characters" are a myth. Lingo and jargon do however exist, and convey meaning in a particularly subtle way. They are shared and taught by culture, not by a secret handbook passed down from generation to generatio…

This reminds me of the joke whose punch line is > You will be lucky to have this person work for you.

I think the line is

> you would be lucky to get this employee to work for you!

Re: The curious case of the missing period

#190

Post by TeMPOraL »

treflop wrote: post #188
TeMPOraL wrote: post #184

Earlier quoted context omitted.

That's only true if you're not breaking the protocol abstraction layer. There is no "out-of-band" once you serialize your messages. If you start injecting random bytes into the data stream on the wire, you can absolutely start introducing commands, or confuse the receiver where the next piece of metadata/control is. In this case, somewhere the protocol abstraction layer got broken, and the message text ended up being…

Injecting random data into any protocol will break it. When we’re talking about whether someone can inject data into the link, we’re talking about the end user and not the software. If we’re talking protocol design, then you wouldn’t want regular data to be able to inject commands by simply existing.

> Injecting random data into any protocol will break it.

It shouldn't, unless you're bypassing the actual protocol serialization layer (or hitting a bug in the implementation). Which is what's the case here. Protocol design can't address the case of users just writing out some bytes and declaring it's a valid protocol message.

Post reply on HN