Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ASCII Delimited Text – Not CSV or TAB delimited text

ronaldduncan.wordpress.com

Post reply on HN
Print view View original

281–286 of 286 posts

Re: ASCII Delimited Text – Not CSV or TAB delimited text

#281

Post by sesqu »

slowmover wrote: post #59
mikestew wrote: post #5

Anyone that's ever had to parse arbitrary data knows of the approximately 14 jiggityzillion corner cases involved when sucking in or outputting CSV/TAB delimited formats. Yet much like virtual memory and virtual machines, we find that a solution has existed since the 60s. For those wondering about the history and use of all those strange characters in your ASCII table: http://www.lammertbies.nl/comm/info/ascii-charac…

> One might question why all control codes in the ASCII character set have low values, but the DEL control code has value 127. This is, because this specific character was defined for deleting data on paper tapes. Most paper tapes in that time used 7 holes to code the data. The value 127 represents a binary pattern were all seven bits are high, so when using the DEL character on an existing paper tape, all holes are…

There's a problem with that explanation: the binary representation of "deleted data" has a different meaning than the control code "delete data".

Re: ASCII Delimited Text – Not CSV or TAB delimited text

#282

Post by pjbrunet »

BorgHunter wrote: post #184
pjbrunet wrote: post #145

Earlier quoted context omitted.

Back in the day with a numeric keypad you could type Alt-(number) to get any ASCII character, but not sure if that still works. In Firefox apparently Alt-1 takes you to the first tab, Alt-2 second tab...

In most graphical *nix environments that I'm familiar with, if you hold Control and Shift¹ and type U, you can type any Unicode character by its (hex) code point. For instance, if you want an em dash, which is U+2014, hold Control and Shift, and type "U2014". ¹You can release Control and Shift after typing "U", in which case the character will appear after you type a space. Or, you can hold Control and Shift while ty…

I just tried in Debian, works! Very handy.

Re: ASCII Delimited Text – Not CSV or TAB delimited text

#283

Post by chinpokomon »

eli wrote: post #258
chinpokomon wrote: post #212

Earlier quoted context omitted.

Unix = LF Mac = CR DOS = CRLF The DOS line endings are inherited from previous systems, and while they precisely convey carriage and paper movement of a printer it can be a pain to deal with today.

Only old Macs. They switched to POSIX-style LF with OS X.

True, but since Mac OSX is already known by most people to be *nix based, at least this audience, I felt like that might just add to confusion. There are certainly other OSs like BeOS that I didn't mention, in part because I didn't think they added to the discussion, and also while having fond memories of BeOS, I honestly don't remember what it used. I want to say it was LF, but I no longer remember.

Re: ASCII Delimited Text – Not CSV or TAB delimited text

#284

Post by chinpokomon »

gpvos wrote: post #237
chinpokomon wrote: post #208

Earlier quoted context omitted.

Perhaps, but I think it still goes against the original intent. Ctrl-~ or Ctrl-^ should give you a record separator (RS) and Ctrl-Del or Ctrl-_ should give you a unit separator (US). For the same reason Ctrl-m or Ctrl-M should give you carriage return (CR). This is because ASCII values from 00-1F are control characters and effectively grounded the most significant bits 7 and 6. Shift similarly would toggle or ground…

This hasn't been true since IBM keyboards became popular. For example, on older keyboards shift+number would simply toggle a bit, so shift+2 would be a double quote, etc., but this hasn't been common for decades now. Unfortunately.

Probably on the path to scan codes. By using scan codes, they could abstract what a particular key meant and thereby remap the keys so that they didn't have to match the ASCII table layout. I still don't understand why we evolved scan codes the way we did. This requires the OS to be in sync to be able to map them back.

Re: ASCII Delimited Text – Not CSV or TAB delimited text

#285

Post by penguindev »

mikeash wrote: post #263
derefr wrote: post #128

Earlier quoted context omitted.

Examples of length-prefixed data abound in protocols and formats defined by systems and telecom engineers (e.g. the IETF). IP packets are length-prefixed. ELF-binary tables and sections are length-prefixed. PNG chunks are length-prefixed. It's just these worse-is-better text-based protocols like HTTP, created by application developers, that toss all the advantages of length-prefixing away. (And, even then, HTTP bodie…

The only problem with length prefixing is that it interferes with streaming data, because you need to know the full length in advance. Thus HTTP chunked encoding. Still, it works great in most scenarios. My favorite way to deal with this stuff is Consistent Overhead Byte Stuffing: http://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffi... In short, you take the data and encode it with a clever scheme that effect…

that is just freaking cool. took me about 4 times to grok it. it sort of reminds me of utf-8, and how you can synchronize that easily.

Re: ASCII Delimited Text – Not CSV or TAB delimited text

#286

Post by JadeNB »

rcthompson wrote: post #189
JadeNB wrote: post #167

Earlier quoted context omitted.

> It may as well be XML (i.e. technically text-based but not practically human-readable). Is this really a standard complaint about XML? I thought the main complaint was that it wasn't human- writeable . I wouldn't want to read novels in XML, but I've never had a problem opening up an XML file in a text editor to get at bits of it.

Well, you'll often get XML files with a single line and no whitespace between elements, and that makes things a lot more interesting. Basically, you can't rely it being practical to quickly poke around in the text of an XML file. I always feel sad when I have to read the text of an XML file to get information.

> Well, you'll often get XML files with a single line and no whitespace between elements, and that makes things a lot more interesting.

That can be fixed with `xmllint --format`; but I agree that, once you need to bring in external tools, it's not clear that calling it 'human-readable' is really appropriate any more.

Post reply on HN