Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

πFS

github.com

Post reply on HN
Print view View original

121–130 of 224 posts

Re: πFS

#121

Post by janalsncm »

jamwise wrote: post #31

Reminds me of when I tried to use the library of babel as a data compression tool. It led me down a fun rabbit hole and was my first introduction to information theory. The conclusion being that you basically need the same amount of data to represent the address of your data as the data itself, so it's not really effective at compression, just a fun thought experiment. The cool part of this in modern times is that LL…

The level of compression is pretty impressive when you think about it. I wrote a comment a while back which is still true (although bytes should be bits, so in that sense it’s still wrong): https://news.ycombinator.com/item?id=39559969

Back of the envelope calculation for storing valid 4-grams (sequences of four words) is around 10 billion x 14 bits per word = 17 gb for all 10 billion. There are LLMs 100x smaller which can write coherent prose.

Re: πFS

#122

Post by smaudet »

mike_hock wrote: post #16
bobim wrote: post #6

This is disturbing to realize that pi then contains all the past and future knowledge, including when I'll pass away.

So does every other random infinite sequence of bits. The unintuitive part comes from infinity, not pi. It also doesn't contain all past and future knowledge because it also contains all possible falsehoods about the past and future in a way that's indiscernible from the truth. Encoding information as an offset into a pseudorandom sequence is no more storage efficient than storing the information directly.

Keyword is conjectured.

Infinities of random sequences exist that can be shown not to contain all data, 0-8 (base 10) is one such random sequence that is trivially proven to never contain 9...

There are no known patterns to pi, but, (I am legitimately curious about this), are there any known sequences e.g. of 1 million 0s and a single other digit within the decimal sequence of pi?

Given how it (pi) looks, I'm of the strong suspicion is that the answer is "no". But of course, proving that requires that some property of the randomness is provable. Which it does feel as if, given there are different infinities, there are also different randomnesses, hence the conjecture is ill-formed and probably incorrect...

Re: πFS

#124

Post by ithkuil »

hatthew wrote: post #110
adzm wrote: post #12

It is worth noting that as the length of data increases it becomes extremely unlikely that the index and length of the sequence within pi would actually be smaller than the data.

TFA addresses this > Now, we all know that it can take a while to find a long sequence of digits in π, so for practical reasons, we should break the files up into smaller chunks that can be more readily found. > In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in π.

Why stop at bytes? Let's split it in individual bits and then look up the bits in pi!

But Pi's binary expansion is not very practical for this purpose, since it's 11.0010...

OTOH. e is 10.1011...

Let's stick to fractional digits (the ones right of the binary point) at index 0 we have 1 and at index 1 we have 0.

So, to encode a stream of bytes so that each bit is encoded as the index of that bit in the e, all you need to do is to xor it with 0xFF

Re: πFS

#125

Post by MisterTea »

ndiddy wrote: post #106
MisterTea wrote: post #8

Reminds me of: https://www.spronck.net/sloot.html Further reading: https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System

I looked into this a bit a while ago, what Sloot did was at least a little novel. Basically the way his encoding scheme actually worked was that it would store each line of video into a database, encode each video frame as a series of line lookups, and then store that encoded frame into another database. Then each video is a series of frame lookups. When you hear accounts of him being able to demo smooth playback of…

Interesting. Do you have any resources you can share?

Re: πFS

#127

Post by [deleted] »

helterskelter wrote: post #112
dekhn wrote: post #104

Earlier quoted context omitted.

I once interviewed for a company and the interviewer was telling me how he (a vc) funded a project to generate large streams of random numbers; you would select an index at random, share that private key with somebody, and then the subsequent text could be used as a one-time-pad. NSA would be forced to buffer/save the entire stream, which could be generated at GB/sec, if they wanted to decrypt. It didn't seem very pr…

I wonder if we could mess with NSA-style surveillance by having a good chunk of the population streaming lots of random data over the internet. Essentially, Alice piping her /dev/random to Bob's /dev/null over netcat or something. Make a slick looking app that does it 24/7 in the background using excess bandwidth and tell people it sticks it to the NSA. Spy agencies would not only have to store it all in case it was…

[deleted]

Re: πFS

#128

Post by School-Cotton »

windward wrote: post #96

>One of the properties that π is conjectured to have is that it is normal conjectured Glad to see one of my pet points of pedantry come up. No non-constructed irrational number has never been proven to be normal or disjunctive.

What do you mean by "non-constructed" here?

Re: πFS

#129

Post by agnishom »

dekhn wrote: post #104
emptyroads wrote: post #79

Reminds me of nsafs, the National Security Agency Filesystem ("free" because the government pays for it) - https://github.com/freedomtools/nsafs

I once interviewed for a company and the interviewer was telling me how he (a vc) funded a project to generate large streams of random numbers; you would select an index at random, share that private key with somebody, and then the subsequent text could be used as a one-time-pad. NSA would be forced to buffer/save the entire stream, which could be generated at GB/sec, if they wanted to decrypt. It didn't seem very pr…

> generate large streams of random numbers; you would select an index at random, share that private key with somebody, and then the subsequent text could be used as a one-time-pad.

This is what stream ciphers are

Post reply on HN