Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intern Impact: Brotli compression for Play Store app downloads

students.googleblog.com

Post reply on HN
Print view View original

171–180 of 184 posts

Re: Intern Impact: Brotli compression for Play Store app downloads

#171

Post by captn3m0 »

iamleppert wrote: post #143

This compression technique seems to be based on the fact they have previous installation of an app that can be diffed and patched, so it wouldn't receive any benefit from first installations, only updates. But still might be worth it for many applications. I remember I investigated a way to send and apply diffs of javascript code (using a js version of patch) and store in the browser using localstorage. However, at t…

What you want is http://www.daemonology.net/bsdiff/

All objects in git packfiles are already compressed, so you aren't gaining much by adding another layer of compression.

Re: Intern Impact: Brotli compression for Play Store app downloads

#172

Post by dang »

forgottenpass wrote: post #109

I bet your hacker excellent skills cannot save 1.5 PB traffic per day. Your own shitty[0] remark highlights why your post doesn't have much of a point. The reason the impact of this change is worth mentioning at all is because google already wields the centralized control needed to effect change to such a massive volume of internet traffic on it's whims. Edit: [0] "shitty" is not a quality rating, but my assessment o…

> Your own shitty[0] remark

You can't do this kind of name-calling on HN, so please don't. You've also been posting quite a few ranty comments lately. Please don't do that; we're trying to for a higher quality of discussion than that here. When hot under the collar, please cool down before posting.

We detached this subthread from https://news.ycombinator.com/item?id=13581770 and marked it off-topic.

Re: Intern Impact: Brotli compression for Play Store app downloads

#173

Post by dang »

therealmarv wrote: post #156
forgottenpass wrote: post #109

I bet your hacker excellent skills cannot save 1.5 PB traffic per day. Your own shitty[0] remark highlights why your post doesn't have much of a point. The reason the impact of this change is worth mentioning at all is because google already wields the centralized control needed to effect change to such a massive volume of internet traffic on it's whims. Edit: [0] "shitty" is not a quality rating, but my assessment o…

Although you don't deserve an reply for your rudeness. Let's say I can get into a position and make some good work and it counts (good, is not it?). Let's say you think you can also make something meaningful and even better and you are currently not in the position to currently make it count for anybody (why then disrespect others people work?). This is something you are lacking: respect.

> Although you don't deserve an reply for your rudeness

> something you are lacking: respect

Please don't break the HN guidelines by being uncivil and making this site worse, even if someone else has behaved badly. Reacting like this creates a downward spiral.

Re: Intern Impact: Brotli compression for Play Store app downloads

#174

Post by NetStrikeForce »

Klockan wrote: post #146
NetStrikeForce wrote: post #113

Earlier quoted context omitted.

> So she used a compression algorithm developed by other googlers? Yes. > So what? So she saved 1.5PB per day - which no other Googler did. Nor you or me. She did. > Don't get me wrong, I'm sure she did a lot of work for it, but looks like a lot of people would have been able to do that, there is nothing innovative in what she did, right? If it was that easy, why wasn't done already? Why did she have to come along to…

From the article: "Anamaria’s project was to add support for Brotli for both new app installs and app updates." So someone else told her to do it, if she wasn't there some other intern would have made the same savings.

I'm not sure how that says "someone else told her to do it", but even if it was true, the point still stands:

She did it, no one else did. You didn't, I didn't.

If other intern would have did it, then we would be having the same conversation about that other intern.

Re: Intern Impact: Brotli compression for Play Store app downloads

#175

Post by NetStrikeForce »

danellis wrote: post #125
NetStrikeForce wrote: post #113

Earlier quoted context omitted.

> So she used a compression algorithm developed by other googlers? Yes. > So what? So she saved 1.5PB per day - which no other Googler did. Nor you or me. She did. > Don't get me wrong, I'm sure she did a lot of work for it, but looks like a lot of people would have been able to do that, there is nothing innovative in what she did, right? If it was that easy, why wasn't done already? Why did she have to come along to…

They had to wait for someone to design and implement the compression algorithm before they could have an intern swap it out.

And they had to wait for people to invent computers so you could do your job; still you get praised for doing good stuff that no one else in your company does :)

Re: Intern Impact: Brotli compression for Play Store app downloads

#176

Post by NetStrikeForce »

ng12 wrote: post #120
NetStrikeForce wrote: post #113

Earlier quoted context omitted.

> So she used a compression algorithm developed by other googlers? Yes. > So what? So she saved 1.5PB per day - which no other Googler did. Nor you or me. She did. > Don't get me wrong, I'm sure she did a lot of work for it, but looks like a lot of people would have been able to do that, there is nothing innovative in what she did, right? If it was that easy, why wasn't done already? Why did she have to come along to…

What makes you think she came up with the idea? Read the article -- it's clear that it was her "project". She's an intern at Google, they're generally on a short leash.

> What makes you think she came up with the idea?

It might have not been her sole idea or even her idea, I'll give you that. However there's no evidence it was not her idea of partly her idea.

> Read the article

Please, this is the kind of comment not welcomed here. I did read the article before commenting. You are trying very hard to read between the lines, when Google is claiming once and again that her work saved 1.5PB/day - I'm not sure why will you try so hard to read a hidden message in Google's words and dismiss what's written in plain English.

> it's clear that it was her "project".

So what? See my previous paragraph.

> She's an intern at Google, they're generally on a short leash.

Generally or always? In the US or in Europe? It was the second time she was interning at Google (it's in the article).

Again, you are trying very hard to read between the lines and guess.

This engineer did something that saved users 1.5PB/day. She did, no one else did. Other people did other stuff. To each their own - I'm not sure why do we have to downplay her achievement though.

P.S.: This is the kind of shit women have to put up with constantly. It might look like it's not important, but when you have to go through this every day it takes a toll on you. The worst part is that we don't even want to acknowledge it.

Re: Intern Impact: Brotli compression for Play Store app downloads

#177

Post by bhouston »

Someone1234 wrote: post #77
bhouston wrote: post #52

I bet switching to LZMA would have saved even more. LZMA beats Brotli nearly every time. zStandard would likely have worked better as well. Brotli is very slow to compress.

That doesn't appear to be true: https://cran.r-project.org/web/packages/brotli/vignettes/bro... I'm sure you can use those results to argue that LZMA is superior in some way (e.g. compression speed) but it definitely isn't clear cut superior in other important ways (compressed size and decompression speed are inferior). I can see why, given those results, that they would use Brotli over LZMA.

The independent tests I did here: https://github.com/google/brotli/issues/165

And also those here: https://www.percona.com/blog/2016/03/09/evaluating-database-...

Suggest that LZMA compresses better than Brotli except in the case of text documents.

Re: Intern Impact: Brotli compression for Play Store app downloads

#178

Post by bhouston »

adtac wrote: post #108
gsylvie wrote: post #67

Earlier quoted context omitted.

LZMA is much slower at decompress compared to Brotli. https://www.opencpu.org/posts/brotli-benchmarks/

Apps are usually in the order of 40-50 MB. From a back of the hand calculation, the a 50 MB file when compressed with Brotli is 14.5 MB and LZMA is 17.5 MB. The difference of 3 MB translates to an additional 8 seconds of data transfer in a 3.1 Mbps 3G connection (an average one). This [1] states that a LZMA has a decompression speed of 70 MB/s, which is about 0.7 seconds. The 334 MB/s speed of Brotli does the same in…

In my tests and others LZMA seems to beat Brotli by quite a bit on non-text cases (DB, 3D meshes): https://news.ycombinator.com/item?id=13589048

I am concerned that the Brotli v LZMA v GZip paper you cite is not fully representative as it is written by the the Brotli team.

Re: Intern Impact: Brotli compression for Play Store app downloads

#179

Post by bhouston »

adtac wrote: post #108
gsylvie wrote: post #67

Earlier quoted context omitted.

LZMA is much slower at decompress compared to Brotli. https://www.opencpu.org/posts/brotli-benchmarks/

Apps are usually in the order of 40-50 MB. From a back of the hand calculation, the a 50 MB file when compressed with Brotli is 14.5 MB and LZMA is 17.5 MB. The difference of 3 MB translates to an additional 8 seconds of data transfer in a 3.1 Mbps 3G connection (an average one). This [1] states that a LZMA has a decompression speed of 70 MB/s, which is about 0.7 seconds. The 334 MB/s speed of Brotli does the same in…

I just installed brotli on my ubuntu machine and compared it compressing a executable.

Brotli: 4.77MB -> 1.21MB (8.5s) xz: 4.77MB -> 1.129MB (1.4s)

LZMA does better than Brotli in the case of binaries by a fair bit every time.

Re: Intern Impact: Brotli compression for Play Store app downloads

#180

Post by bhouston »

mturmon wrote: post #71
bhouston wrote: post #52

I bet switching to LZMA would have saved even more. LZMA beats Brotli nearly every time. zStandard would likely have worked better as well. Brotli is very slow to compress.

More on the (speed, compression) trade space here: http://www.gstatic.com/b/brotlidocs/brotli-2015-09-22.pdf

This is not correct for the case of binaries.
Post reply on HN