Live data from Hacker News

Brave Leo now uses Mixtral 8x7B as default

brave.com

141–150 of 184 posts

Re: Brave Leo now uses Mixtral 8x7B as default

#141
post #110

Earlier quoted context omitted.

Not just because of the filter list cap. It also reduces ad blockers to static filter lists instead of powerful dynamic filters. MV3 makes it impossible for ad-blockers to inspect requests with code and then allow/deny dynamically.

>It also reduces ad blockers to static filter lists instead of powerful dynamic filters. This is very outdated information and borderline misinformation by representing it as how it currently works. It allows for 30,000 dynamic rules and 5,000 session rules (session rules only persist until the browser is closed). >MV3 makes it impossible for ad-blockers to inspect requests with code and then allow/deny dynamically.…

> It allows for 30,000 dynamic rules

That is not what we mean by dynamic filters. From https://developer.chrome.com/blog/improvements-to-content-fi...

> However, to support more frequent updates and user-defined rules, extensions can add rules dynamically too, without their developers having to upload a new version of the extension to the Chrome Web Store.

What Chrome is talking about is the ability to specify rules at runtime. What critics of Manifest V3 are talking about is not the ability to dynamically add rules (although that can be an issue), it is the ability to add dynamic rules -- ie rules that analyze and rewrite requests in the style of the blockingWebRequest permission.

It's a little deceptive to claim that the concerns here are outdated and to point to vague terminology that sounds like it's correcting the problem, but on actual inspection turns out to be entirely separate functionality from what the GP was talking about. It's almost like the Chrome team deliberately decided to call these "dynamic rules" so they could claim that Chrome supported them and muddy the issue, even though Chromes "dynamic rules" have nothing to do with support for a blockingWebRequest API. But I don't want to be conspiratorial.

> Giving this ability to extensions can slow down the browser for the user. These ads can still be blocked through other means.

This is the debate; most of the adblocking community disagrees with your assertion. Chrome has been saying this for ages, but saying it doesn't make it true.

uBO maintains a list of some common features that are already not possible to support in Chrome (https://github.com/gorhill/uBlock/wiki/uBlock-Origin-works-b...) and has written about features that are not able to be supported via Chrome's current V3 API (https://github.com/uBlockOrigin/uBOL-home/wiki/Frequently-as...). Of particular note are filtering for large media elements (I use this a lot on mobile Firefox, it's great for reducing page size), and top-level filtering of domains/fonts.

Chrome could of course add support for some of this -- Chrome could implement more filter controls for analyzing headers and request sizes, but as far as I know that support hasn't been added yet. And even if that support does eventually get added, this all glosses over the bigger issue that devs have been saying from the beginning, which is that it's ridiculous to make every innovation in adblocking dependent on Chrome explicitly adding new APIs to support each individual use-case. The model that Chrome is moving towards is one where every new kind of filtering metric that devs want to use requires asking Google for permission to use it and waiting for Chrome to implement it.

Re: Brave Leo now uses Mixtral 8x7B as default

#142

Earlier quoted context omitted.

Because the filter list is capped, right? Is there a reason the Brave team cannot just remove or increase the cap?

>Because the filter list is capped, right? The limits are 300,000 static rules [1] + 30,000 dynamic rules [2] + 5,000 session rules [3]. For reference easylist is about 35k rules. The Chrome team has been constantly tweaking these limits themselves and Brave could set their own limits if they wish. The API is designed such that extensions can query to see how many rules they can use. [1] https://source.chromium.org/c…

Note that "dynamic rules" as specified here are not the same as what GP was talking about. The API does not support dynamic rules in the way that Firefox/V2 users use that term.

Re: Brave Leo now uses Mixtral 8x7B as default

#143

Earlier quoted context omitted.

Did you do anything special to make that work? Is it useful? Or just a toy?

I have a 14" MBP with an M1 Max and 64GB. The M3 won't really make a difference, but the RAM, since unified, is huge. I can run most models on this machine with realtime performance compared to a Ryzen 7735HS and 64GB (DDR5). Now I'm not saying the Ryzen setup should be good, but the M1 architecture just makes it a much better option. I could add an eGPU to the Ryzen system and it could likely do better, but would al…

it's not just that it's huge and unified - ryzen APUs obviously can have 2x32GB SODIMMs put in them and they support unified memory too.

the difference is the bandwidth and the computational power of the APU. M1 Max is roughly similar to a PS5 in terms of overall system design (shader configuration and bandwidth) plus has dedicated AI inference units already (which won't be added to consoles until PS5 Pro launches with RDNA 3.5). It is far more bandwidth than you can get out of a socketed-memory laptop system.

https://twitter.com/Locuza_/status/1450271726827413508

To support that level of performance in a socketed-memory system you will need an extra layer of caching added to the processor to supplement the bandwidth - and maybe still need to go to quad-channel. Those products are Strix and Strix Halo and should be hitting the market over the next year or two but the reality is that the M1 Max was an absurdly powerful laptop, far more potent than even the first-gen 5nm laptops for x86 let alone the other junk you could buy in 2020.

This is the problem with the discourse around apple silicon for the last few years: yeah, they're expensive, but even a loaded-out x86 laptop doesn't get you the same capabilities. Even if the x86 is competitive in some particular benchmark on iso-node you are probably spending more power to do it, and the x86 product comes years after the apple product, and still has a much weaker gpu and less bandwidth (which doesn't just matter for GPU, it matters for compiling and JIT too).

It is incredibly silly to look back on the discourse in 2020-2023 around apple silicon, a lot of reviewers made extremely silly claims about how "even 7nm x86 processors were already competitive with apple silicon" and as the ecosystems have matured it is obvious that even 5nm processors are not quite competitive yet. And they dumped on the SPEC tests and Geekbench that measured this properly, in favor of dumb things like cinebench R23 and so on (it's always cinebench used for this dumb shit tbh, CB R13/R15 were hugely misleading at the zen1 launch too). Let alone things like, you know, compiling or JVM/node workloads...)

(similarly, gotta love the vibe a few years ago of: "threadripper vs mac pro" - did you know that a 64C threadripper with 256GB RAM is actually cheaper than a mac pro loaded out with 2TB!? waow, who knew systems with an order of magnitude less capacity would be cheaper!? https://youtu.be/BH291DQRIOg )

Re: Brave Leo now uses Mixtral 8x7B as default

#144

Earlier quoted context omitted.

>It also reduces ad blockers to static filter lists instead of powerful dynamic filters. This is very outdated information and borderline misinformation by representing it as how it currently works. It allows for 30,000 dynamic rules and 5,000 session rules (session rules only persist until the browser is closed). >MV3 makes it impossible for ad-blockers to inspect requests with code and then allow/deny dynamically.…

> It allows for 30,000 dynamic rules That is not what we mean by dynamic filters. From https://developer.chrome.com/blog/improvements-to-content-fi... > However, to support more frequent updates and user-defined rules, extensions can add rules dynamically too, without their developers having to upload a new version of the extension to the Chrome Web Store. What Chrome is talking about is the ability to specify rules…

>That is not what we mean by dynamic filters.

My point was that "reduces ad blockers to static filter lists" is not true as it is not a static list of rules.

>What critics of Manifest V3 are talking about is not the ability to dynamically add rules

Yes, they are. Due to early versions of declarativeNetRequest only having a static list of rules there are a large amount of people who believe this to still be true. On this site I have encountered many people who make a claim about it being impossible to update rules without an update to the extension through the store.

>It's a little deceptive to claim that the concerns here are outdated and to point to vague terminology that sounds like it's correcting the problem, but on actual inspection turns out to be entirely separate functionality from what the GP was talking about.

Dynamic rules do correct the problem of a static list. The second half of my comment points out that there was a trade off made between functionality and performance which is why "dynamic filters" are no longer possible.

>This is the debate; most of the adblocking community disagrees with this assertion.

Considering MV3 lets extensions turn every page into a blank document or inject random scripts I don't see how it can be possible for ads to somehow force themselves to be shown to the user.

>common features that are already not possible to support in Chrome

>CNAME-uncloaking

The relevant issue in the bug tracker is https://bugs.chromium.org/p/chromium/issues/detail?id=115104... and the engineer working on it does not have enough bandwidth.

>Browser launch

This is no longer a problem with declarativeNetRequest since the rules for blocking network requests is part of the browser itself.

>as written about features that are not able to be supported via Chrome's current V3 API

Some of these are self inflected by the author like only updating rules with store updates and, some of these are due to the lack of maturity of MV3, some of these are due to trade offs that are being made ecosystem wide.

Re: Brave Leo now uses Mixtral 8x7B as default

#145

Earlier quoted context omitted.

>Because the filter list is capped, right? The limits are 300,000 static rules [1] + 30,000 dynamic rules [2] + 5,000 session rules [3]. For reference easylist is about 35k rules. The Chrome team has been constantly tweaking these limits themselves and Brave could set their own limits if they wish. The API is designed such that extensions can query to see how many rules they can use. [1] https://source.chromium.org/c…

Note that "dynamic rules" as specified here are not the same as what GP was talking about. The API does not support dynamic rules in the way that Firefox/V2 users use that term.

GP did not mention dynamic rules.

Re: Brave Leo now uses Mixtral 8x7B as default

#146

Earlier quoted context omitted.

Have you tried this yourself? Curious to know how well this works for an LLM home lab.

I’ve worked with Jetson going back to the TK1 and I highly recommend you do not do this. Nvidia has significant dominance in the AI space because of their work on software and the overall platform. With the Jetson line being the sole exception. Use it for what it’s for - a targeted build for an embedded/specific application requiring small size and low power. The software is a mess. Support for Jetson (generally) is…

This is a bit surprising to hear. Current Jetpack 6 is Ubuntu 22.04 - this is the current Ubuntu LTS release. There's nothing ancient about it, no? I'm pretty sure, if I go and check versions of CUDA, PyTorch, Tensorflow - it'd be also relatively recent.

I'd suggest checking what examples are available, see what community is doing, see if what you need had already been tried - https://www.jetson-ai-lab.com

From what I've seen, mainstream LLM libraries like VLLM, llamacpp that use CUDA under the hood tend to work out-of-the-box. And there are tutorials available: https://www.jetson-ai-lab.com/tutorial_text-generation.html. I think that TensorFlow/Pytorch are also well maintained, although I've not checked recently.

Re: Brave Leo now uses Mixtral 8x7B as default

#147

Earlier quoted context omitted.

I use firefox because it has the most hassle-free hardware decoding in linux. However, everything basically feels better with Brave, even with the same amount of plug-ins.

I found the snap update notifications too annoying on Ubuntu, so I tried the ppa. But it the video plugin would crash. So back to Chrome for me.

Curious whether you've tried with the new (non-PPA) repo directly from Mozilla as of v122 [1]. I think the old PPA was also Mozilla, so I don't know what may have changed aside from being more publicly acknowledged. Might be worth a try?

I don't have an Ubuntu VM at-hand but on Debian bookworm it installed fine, and (after tweaking one line in profiles.ini to point to my old ESR profile) it loaded and played Widevine-protected videos without any issues.

[1]: https://support.mozilla.org/en-US/kb/install-firefox-linux#w...

Re: Brave Leo now uses Mixtral 8x7B as default

#148

Earlier quoted context omitted.

Note that "dynamic rules" as specified here are not the same as what GP was talking about. The API does not support dynamic rules in the way that Firefox/V2 users use that term.

GP did not mention dynamic rules.

> It also reduces ad blockers to static filter lists instead of powerful dynamic filters.

Just to make sure, did you actually not know what comment I was referring to, or are you just upset that I incorrectly referred to it as a GP comment?

In either case, it remains true that Chrome's current plans for V3 do limit the ability of extensions like uBO to dynamically filter requests.

Re: Brave Leo now uses Mixtral 8x7B as default

#149

Earlier quoted context omitted.

> It allows for 30,000 dynamic rules That is not what we mean by dynamic filters. From https://developer.chrome.com/blog/improvements-to-content-fi... > However, to support more frequent updates and user-defined rules, extensions can add rules dynamically too, without their developers having to upload a new version of the extension to the Chrome Web Store. What Chrome is talking about is the ability to specify rules…

>That is not what we mean by dynamic filters. My point was that "reduces ad blockers to static filter lists" is not true as it is not a static list of rules. >What critics of Manifest V3 are talking about is not the ability to dynamically add rules Yes, they are. Due to early versions of declarativeNetRequest only having a static list of rules there are a large amount of people who believe this to still be true. On t…

> My point was that "reduces ad blockers to static filter lists" is not true as it is not a static list of rules.

That's not what we mean by static either. What we mean by static is that extensions lose the ability to dynamically analyze requests and block them using on-the-fly logic.

> Yes, they are. Due to early versions of declarativeNetRequest only having a static list of rules there are a large amount of people who believe this to still be true.

Very obviously that is not what gkbrk meant by "powerful dynamic filters", considering that gkbrk clarified that they were referring to the ability to dynamically modify requests literally one sentence later. Also very obviously that is not what gkbrk meant because gkbrk literally said in the first sentence that they weren't talking about filter list caps.

When you replied to gkbrk saying that dynamic filters were supported you were not correcting misinformation, you were saying that gkbrk was wrong -- and they're not, they're correct -- dynamic filtering of the kind they are talking about is not supported and the lack of that API meaningfully constrains adblockers (a point which, quite frankly, you don't even disagree with given that you are characterizing this as a "tradeoff").

Did you mean to reply to a different person who wasn't present?

> Considering MV3 lets extensions turn every page into a blank document or inject random scripts I don't see how it can be possible for ads to somehow force themselves to be shown to the user.

"Just inject scripts into the webpage" is a wild take from someone who is trying to claim that Manifest V3 improves privacy, security, or performance. Nobody wants addons to be putting all of their functionality into the page itself, there are myriads of problems with that approach. If you're defending Manifest V3 and you're suggesting that addons should just completely destroy the separation between extension code and page code, then... I mean, that's just not a serious suggestion.

It's good that extensions can insert code into pages, that is sometimes necessary and extensions should have that ability. I think uBO full-on uses that ability for some features. But running code that way is not the preferred way to handle functionality or request blocking, and its something that should be done sparingly and carefully.

If the upshot of Manifest V3 in Chrome is to encourage developers to start pushing tons of extension code into the page itself when they could have before kept that code separate -- then that's a failure of Manifest V3; injecting tons of extra scripts into pages that wouldn't otherwise need to be there will make performance, security, and privacy worse for end users.

> The relevant issue in the bug tracker is https://bugs.chromium.org/p/chromium/issues/detail?id=115104... and the engineer working on it does not have enough bandwidth.

So it's not supported. I'm sorry, you want me to pretend that developer concerns have been addressed because there's an open issue where the developer is overworked and does not have the bandwidth to address it? This issue has been open for 3 years! You could not show a better example of the intrinsic problems with Manifest V3 and the intrinsic problems of extension devs needing to ask Chrome devs for permission to innovate on adblockers if you tried.

This is exactly the problem of Manifest V3. There's need for adblockers to be able to innovate, and when that innovation is dependent on Chrome adding specific APIs, then the issues sit open for 3 years. Chrome is proving before V3 even becomes a requirement that they are not capable of keeping pace with adblocker innovation and that this whack-a-mole approach to adding individual capabilities into declarativeNetRequests is unworkable and bad for addons.

If Google can't keep pace with requests during a period when extensions are still able to use V2, then they're not going to magically get better at handling feature requests when every extension is using V3. If Google's approach to addon APIs is to overwork developers and spread them thin between features, it is reasonable to conclude that either Google does not see addon support as a priority worth investing sufficient developer resources into, or that Google's model for browser development is just inherently not capable of handling API requests at a reasonable speed.

---

And I think that's the broader point. This is not a healthy relationship for Chrome to have with extension developers, and Chrome is proving in real time that it doesn't have the resources or ability to support developer requests.

But to address the other specific APIs you bring up:

> This is no longer a problem with declarativeNetRequest since the rules for blocking network requests is part of the browser itself.

I'm sorry, the inability of an extension to load before the browser starts sending requests is a non-issue to you? Sure, you can pull the blocklist out so the static rules still take effect before the extension loads, but that is far from the only issue in having extensions load asynchronously from their pages.

To be fair, you are right that declarativeNetRequest makes this slightly better, but only because prior to declarativeNetRequest Chrome already exhibited this behavior so at least now there's one way to make sure that requests get filtered from browser launch, as limited as it may be. As opposed to Firefox, where this has never been a problem for either MV2 or MV3 extensions, because Firefox doesn't make the wild decision to start loading pages before the user's addons have initialized.

> Some of these are self inflected by the author like only updating rules with store updates

I actually don't think that's the main problem with most of these. Dynamic user rules, fonts, and noscript rules are blocked by rule scoping around top-level document URLs, not because of store updates. uBO Lite does avoid making network requests to update lists, but that is one paragraph out of an entire page describing limitations that make useful features difficult or impossible to build under MV3.

Additionally, if you dig into the actual details of what Gorhill is saying about updating extension lists, you'll find that this is not just about whether you can load list updates, it's also about when it happens and the fact that (as mentioned above) uBO Lite can't guarantee that it will be running when a request is made, and can't guarantee that those rule updates will take effect before the request is sent.

> and, some of these are due to the lack of maturity of MV3

Well, it's a good thing that Google is waiting for MV3 to become mature and it's a good thing that MV2 won't stop working in June, right? /s

> some of these are due to trade offs that are being made ecosystem wide.

Sure, other apps other than adblockers will also be affected. I maintain hobby Firefox extensions that are impossible to ship on Chrome for Manifest V3. Do you expect "we didn't just break adblockers, we broke lots of other addons too" to be an argument that makes anyone more sympathetic to Google?

Re: Brave Leo now uses Mixtral 8x7B as default

#150
post #136
post #77

Earlier quoted context omitted.

IME ollama ran mixtral on a 1070 fast enough.

Though it most probably does not run in on the 1070 but rather on the cpu. It cannot fit on a 1070, it is not about speed, a 1070 cannot run it period.

In llama.cpp You can offload some of the layers to gpu with -ngl X. Where x is the number of layers
Post reply on HN