My post was an earlier post in this thread. I give figures for pure cpu, not using any GPU. The RTX 2070 figure is a separate figure (with rather insane speeds for that model!). You could contribute to the discussion by grabbing some numbers on lower power hardware if you have some available.

Though low memory devices like 4-8gb probably does need specialized inference to give fair numbers.

> you could disable it on those low end devices

it's not like it does anything if you don't ask for it-- and good thing, because the privacy invasion would be all the worse if it did!

> mistral was the safest

Sending the user's confidential information to third parties while falsely suggesting that it is private can cause them irrecoverable harm.

An alternative is that a feature is slow for some users on slow hardware, and there is a setting to make it faster at the expense of privacy and security that they can switch on. The large body of users that find the default config fast enough have no reason to flip the switch.

> But then people would complain about the size, like what happened with Google Chrome embedded llm.

People or astroturf accounts? :P but having read some of the commentary on it there was some pretty weird takes, like generalized complaints about AI and people thinking google was using their computer to serve other people. Google's business model generally prevents correctly marketing the functionality in any case: it's not like they're going to make a proper pitch for how important it is for your privacy when the rest of their business is centered on hoovering everything up. Mozilla is not so constrained :P