Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

brave.com

Post reply on HN
Print view View original

101–110 of 289 posts

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#101

Post by shatsky »

no_wizard wrote: post #99
shatsky wrote: post #96

Earlier quoted context omitted.

And yet Rust ecosystem practically killed runtime library sharing, didn't it? With this mentality that every program is not a building block of larger system to be used by maintainers but a final product, and is statically linked with concrete dependency versions specified at development time. And then even multiple worker processes of same app can't share common code in memory like this lib, or ui toolkit, multimedi…

if I recall correctly Rust does not support any form of dynamic linking or library loading. Most of the community I’ve interacted with are big on either embedding a scripting engine or WASM. Lots of momentum on WASM based plugins for stuff. It’s a weakness for both Rust and Go if I recall correctly

It does support dynamic libs, but virtually all important Rust software seems to be written without any consideration for it.

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#102

Post by nosman »

smt88 wrote: post #88
nosman wrote: post #16

I hope that this is the start of developers being conscious of using resources efficiently again, especially in the browser. The more rust gets written, the better AI will be able to write it for people... I like to be optimistic.

I think the Rust part of the headline is incidental in this case. The previous version that used more resources was also written in Rust.

This is true, but a team using rust will probably care more about resource use in general.

I do agree it's kind of a misleading headline, the real update is their use of Flatbuffers.

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#103

Post by shatsky »

terafo wrote: post #100
shatsky wrote: post #96

Earlier quoted context omitted.

And yet Rust ecosystem practically killed runtime library sharing, didn't it? With this mentality that every program is not a building block of larger system to be used by maintainers but a final product, and is statically linked with concrete dependency versions specified at development time. And then even multiple worker processes of same app can't share common code in memory like this lib, or ui toolkit, multimedi…

Dynamic libraries are a dumpster fire with how they are implemented right now, and I'd really prefer everything to be statically linked. But ideally, I'd like to see exploration of a hybrid solution, where library code is tagged inside a binary, so if the OS detects that multiple applications are using the same version of a library, it's not duplicated in RAM. Such a design would also allow for libraries to be update…

I believe NixOS-like "build time binding" is the answer. Especially with Rust "if it compiles, it works". Software shares code in form of libraries, but any set of installed software built against some concrete version of lib which it depends on will use this concrete version forever (until update replaces it with new builds which are built against different concrete version of lib).

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#104

Post by shatsky »

nineteen999 wrote: post #34
nicoburns wrote: post #14

Brave's adblocking engine is a neat example of open source and the ease of sharing lbraries in Rust. It uses Servo crates (also used by Firefox) to parse CSS and evaluate selectors, and is then itself published as a crate on crates.io where it can be pulled in by others who may want to use it.

At risk like node/npm with all the supply-chain attacks then? Or is there something that cargo does to manage it differently (due diligence?).

I don't know much about node but cargo has lock file with hashes which prevents dep substitution unless dev decide to update lock file. Updating lock file has same risks as initial decision to depend on deps.

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#105

Post by lmm »

shatsky wrote: post #96
nicoburns wrote: post #14

Brave's adblocking engine is a neat example of open source and the ease of sharing lbraries in Rust. It uses Servo crates (also used by Firefox) to parse CSS and evaluate selectors, and is then itself published as a crate on crates.io where it can be pulled in by others who may want to use it.

And yet Rust ecosystem practically killed runtime library sharing, didn't it? With this mentality that every program is not a building block of larger system to be used by maintainers but a final product, and is statically linked with concrete dependency versions specified at development time. And then even multiple worker processes of same app can't share common code in memory like this lib, or ui toolkit, multimedi…

> And yet Rust ecosystem practically killed runtime library sharing, didn't it?

Yes, it did. We have literally millions of times as much memory as in 1970 but far less than millions of times as many good library developers, so this is probably the right tradeoff.

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#106

Post by cindyllm »

timcobb wrote: post #95
ashishb wrote: post #66

Earlier quoted context omitted.

> You’re looking at the number of dependents. The React package has no dependencies. Indeed. My apologies for misinterpreting the link that I posted. Consider "devDependencies" here https://github.com/facebook/react/blob/main/package.json As far as I know, these 100+ dev dependencies are installed by default. Yes, you can probably avoid it, but it will likely break something during the build process, and most people…

> And do what? Keep on keepin on

[dead]

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#107

Post by antonok »

ThierryBuilds wrote: post #44

I am curious to know what was the contribution of switching to FlatBuffers in that improvement.

FlatBuffers was definitely the majority of the improvements here!

On 64-bit systems, pointers themselves can really start to take up a lot of memory (especially if you multiply them across 100k+ adblock filters). Switching to array indices instead of pointers saves a lot of memory that's otherwise wasted when you don't need to address the entire possible memory space.

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#108

Post by chii »

dfajgljsldkjag wrote: post #49
drnick1 wrote: post #45

Earlier quoted context omitted.

This is good news, but I am confused by the following: """ Brave Origin is: 1/ new, optional, separate build (stripped down, no telemetry/rewards/wallet/vpn/ai); 2/ free on Linux, one time buy elsewhere. """ So the stripped down version (at least the non-Linux one) will not be open source?

Open source software can be sold for money. For example redhat selling cds with rhel on them, for quite a big sticker price. Free if you build it yourself but you have to pay to get a ready to use version.

or figure out how to build it yourself from source. But you can count on the lazy tax - esp. for windows (as building from source on linux is likely to be much more convenient).

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#109

Post by imron »

quotemstr wrote: post #40
nosman wrote: post #16

I hope that this is the start of developers being conscious of using resources efficiently again, especially in the browser. The more rust gets written, the better AI will be able to write it for people... I like to be optimistic.

> I hope that this is the start of developers being conscious of using resources efficiently again, especially in the browser. Me too. > The more rust gets written, Rust seems neither necessary nor sufficient for getting developers to care about memory efficiency again though.

It is not necessary in a theoretical sense, but in a practical sense developers that care about memory efficiency only have a handful of options.

If you also care about memory safety it further limits options.

Re: Brave overhauled its Rust adblock engine with FlatBuffers, cutting memory 75%

#110

Post by eYrKEC2 »

VladVladikoff wrote: post #91

Is Brave actually a good browser now? Did they rewrite it or something? Last time I looked it was a janky JavaScript mess.

The most effective ad blocker I've ever used. With that, it becomes better than chrome, in my opinion. Give it a spin, it's my daily driver for web dev as well ( just chrome debugger)
Post reply on HN