Live data from Hacker News

There's no single best way to store information

quantamagazine.org

11–20 of 53 posts

Re: There's no single best way to store information

#11
The best way to store information depends on how you intend to use (query) it.

The query itself represents information. If you can anticipate 100% of the ways in which you intend to query the information (no surprises), I'd argue there might be an ideal way to store it.

Re: There's no single best way to store information

#15
post #11

The best way to store information depends on how you intend to use (query) it. The query itself represents information. If you can anticipate 100% of the ways in which you intend to query the information (no surprises), I'd argue there might be an ideal way to store it.

This is exactly right, and the article is clickbait junk.

Given the domain name, I was expecting something about the physics of information storage, and some interesting law of nature. Instead, the article is a bad introduction to data structures.

Re: There's no single best way to store information

#16
There are plenty of good enough ways:

* For lossless compression of generic data, gzip or zstd.

* For text, documentation, and information without fancy formatting, markdown, which is effectively a plain-text superset.

* For small datasets, blobs, objects, and what not, JSON.

* For larger datasets and durable storage, SQLite3.

Whenever there's text involved, use UTF-8. Whenever there's dates, use ISO8601 format (UTC timezone) or Unix timestamps.

Following these rules will keep you happy 80% of the time.

Re: There's no single best way to store information

#17
post #8

There are, however, several objectively bad ways. In "Service Model" (a novel that I recommend) a certain collection of fools decides to sort bits by whether it's a 1 or a 0, ending up with a long list of 0's followed by a long list of 1's.

It _does_ open up amazing opportunities for compression though.

[deleted]

Re: There's no single best way to store information

#19
post #16

There are plenty of good enough ways: * For lossless compression of generic data, gzip or zstd. * For text, documentation, and information without fancy formatting, markdown, which is effectively a plain-text superset. * For small datasets, blobs, objects, and what not, JSON. * For larger datasets and durable storage, SQLite3. Whenever there's text involved, use UTF-8. Whenever there's dates, use ISO8601 format (UTC…

One format I'm missing: storage for conversations and social media posts. Both are complex media (text + images/videos + metadata), and one is actually a collection of such posts.

How would you go about storing those in a somewhat human-readable format? My goal is to archive my chats and social media activity.

Post reply on HN