Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A student’s desire to get out of a exam led to a compression algorithm

quantamagazine.org

Post reply on HN
Print view View original

81–90 of 132 posts

Re: A student’s desire to get out of a exam led to a compression algorithm

#81

Post by Timon3 »

dmytrish wrote: post #70
jakub_g wrote: post #67

Earlier quoted context omitted.

HTTPS everywhere is not only about privacy but also about integrity. You don't want internet randos (including ISPs) to swap your content with ads, bitcoin miners and put other batshit crazy JavaScript in it. Also, a lot of ISPs blackbox caching proxies were buggy and breaking websites.

Integrity without confidentiality is trivial.

How so? Your ISP is still the channel over which the integrity information is sent. Why couldn't they just swap the SRI hashes as well?

Re: A student’s desire to get out of a exam led to a compression algorithm

#82

Post by MauranKilom »

twotwotwo wrote: post #20

There was a series of articles on lossless and lossy and compression techniques in, I think, PC Magazine that I read as a kid and made a big impression on me. I didn't, like, end up ready to write code to build Huffman trees or anything, but it did change it from a total black box into a bunch of smaller pieces each of which a mere mortal can understand. The compression rabbit hole can be a rewarding one to go down i…

Another interesting facet is that, according to some schools of thought, compression and AI are equivalent problems: The better you understand something, the less you need to memorize to reproduce it.

https://en.wikipedia.org/wiki/Hutter_Prize (and https://en.wikipedia.org/wiki/AIXI)

Of course, large language models are (by definition) currently going the other direction, but it remains to be seen whether that leads to artificial intelligence (whatever that ends up meaning).

Re: A student’s desire to get out of a exam led to a compression algorithm

#83

Post by ninkendo »

Timon3 wrote: post #81
dmytrish wrote: post #70

Earlier quoted context omitted.

Integrity without confidentiality is trivial.

How so? Your ISP is still the channel over which the integrity information is sent. Why couldn't they just swap the SRI hashes as well?

You could sign the content with the same CA architecture we already use to encrypt it, but leave it plain text (just a thought.)

A browser could render a similar security warning to what it already does, if the signature doesn’t match or if the hash is wrong.

Re: A student’s desire to get out of a exam led to a compression algorithm

#84

Post by ninkendo »

bhickey wrote: post #65
throwaway892238 wrote: post #64

Earlier quoted context omitted.

This is why I was flabbergasted by the nonsense internet takeover of HTTPS. You can't cache it. Probably 99% of web traffic is generic public content, but we can't cache any of it, because 1% of it needs to be private. So then people complain "oh but The Illuminati can see what websites I'm going to!" Yeah, and they still can with HTTPS, it's called statistical network traffic analysis. Decades of research papers sho…

Tell me you've never worked on a CDN without telling me you've never worked on CDN. Compared to transit, last mile bandwidth is effectively limitless and free. Cache fill at the edge is important, last mile caching not so much.

You didn’t need the first paragraph there, the snark is a bit uncalled for.

Re: A student’s desire to get out of a exam led to a compression algorithm

#85

Post by ninkendo »

Null-Set wrote: post #77
iforgotpassword wrote: post #47

Earlier quoted context omitted.

Is it still? Afaik now with everything HTTPS, caching proxies on the ISP end are completely useless. We instead now see (proprietary?) cache boxes by big players like YouTube and Netflix that ISPs can install in their DCs, but that seems less elegant, even though it gives the content providers much greater control over what, when and how much gets cached. Still, as a smaller fish, without going with cloudflare, there…

How do these cache boxes work? Does the server send them its half of the ephemeral key or something? I can't imagine each isp has a box with a copy of netflix's private key.

They terminate TLS at the cache box and use a different connection altogether from the cache box to the upstream server. (Said tunnel to upstream could simply be another TLS connection, or wireguard, or IPsec, etc etc.)

Re: A student’s desire to get out of a exam led to a compression algorithm

#86

Post by vjerancrnjak »

amelius wrote: post #79

What would be interesting is something like an LLM attempting to compress some piece of data, and see how it gets better at it over time.

I think there’s a contest for this compressing Wikipedia. LMs are winning.

Re: A student’s desire to get out of a exam led to a compression algorithm

#87

Post by Timon3 »

ninkendo wrote: post #83
Timon3 wrote: post #81

Earlier quoted context omitted.

How so? Your ISP is still the channel over which the integrity information is sent. Why couldn't they just swap the SRI hashes as well?

You could sign the content with the same CA architecture we already use to encrypt it, but leave it plain text (just a thought.) A browser could render a similar security warning to what it already does, if the signature doesn’t match or if the hash is wrong.

Right, I didn't thank about that part. They could technically replace it with another valid certificate, but if you're looking for specific certificates you will notice immediately.

Re: A student’s desire to get out of a exam led to a compression algorithm

#88

Post by fguerraz »

crispyambulance wrote: post #72
fguerraz wrote: post #8

Quanta magazine and all these pop science websites need to be stopped. (Not because popular science is bad, but because they do it badly, and the clickbait is insufferable)

> but because they do it badly, and the clickbait is insufferable Well, then, who does pop science "right"? There's no shortage, of course, of 30+ page review articles on every scientific topic imaginable. And if someone has a few days, a freshly minted STEM degree or years of related experience, and a compelling interest in the topic, they can just pick up one of these review articles and go to town. But that isn't…

I must say, there is no written media that I know of that pleases me. Of course, it doesn't mean they don't exist.

I much prefer the treatment from some YouTubers, such as Sabine Hossenfelder (to name just one).

It's far from dry, she has a wicked sense of humour, she tries her best to be impartial (and sometimes fails), and makes it clear when something she says is her opinion.

PBS also has a ton of good content.

Of course, all these actually require the viewer to make an effort, but if you're not ready to do that, then a poorly written clickbait article is likely to do more harm than good anyways.

Re: A student’s desire to get out of a exam led to a compression algorithm

#89

Post by zulban »

ixwt wrote: post #75
jhallenworld wrote: post #74

I thought Huffman coding was obsolete, arithmetic coding replaced it. It allows for fractional number of bits per symbol, so is more efficient.

My understanding why arithmetic coding isn't used as much as Huffman coding is because of the patents that IBM had on arithmetic coding. I'm not familiar with any actively used compression algorithm that uses arithmetic coding. I have explored using arithmetic coding on some data at work (mainly large amounts of XYZ points). My attempts did not work better than standard zip compression.

Huh? I thought you cannot patent an algorithm, mathematical process, number, game design, etc.

Re: A student’s desire to get out of a exam led to a compression algorithm

#90

Post by cesarb »

ixwt wrote: post #75
jhallenworld wrote: post #74

I thought Huffman coding was obsolete, arithmetic coding replaced it. It allows for fractional number of bits per symbol, so is more efficient.

My understanding why arithmetic coding isn't used as much as Huffman coding is because of the patents that IBM had on arithmetic coding. I'm not familiar with any actively used compression algorithm that uses arithmetic coding. I have explored using arithmetic coding on some data at work (mainly large amounts of XYZ points). My attempts did not work better than standard zip compression.

> I'm not familiar with any actively used compression algorithm that uses arithmetic coding.

I believe some popular video codecs use arithmetic coding (and IIRC, some other video codecs use the related range coding to avoid these patents).

Post reply on HN