341–350 of 588 posts
This has to be one of the craziest uploads on the internet up until now. Raw fucking intelligence at your disposal, free to download. If you'd describe what's happening now to someone from five years ago they'd think you're hallucinating or mad.
Earlier quoted context omitted.
> Even if the output is like 5-6 tok/s, that might be usable for some purposes. You'll spend ~100x more on electricity than the API cost to have it run on someone else's GPU at several hundred tokens per second. I think some sort of extreme data privacy requirement is the only situation that justifies this, but the intersection of {needs absolute data privacy, needs to run SOTA model, cannot afford GPUs} is really re…
There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socke…
Getting 404 on the OP's link. Does it mean it got banned or self-censored in the meantime?
At the risk of sounding like a conspiracy theorist, this sounds like a great opportunity to make a statement . US or China, but likelier to be the former. Maybe Clem's on a call with the US government right now?
Getting 404 on the OP's link. Does it mean it got banned or self-censored in the meantime?
Until a few minutes ago there was a countdown page. (The weights haven't been released yet.) The countdown should be over in 19min, not sure why we're suddenly getting a 404.
Getting 404 on the OP's link. Does it mean it got banned or self-censored in the meantime?
I imagine it's just technical issues on the flip. It's also going to be interesting what happens to HF with loads of people downloading a many TB model. Even though almost no one has the capability to run it, it does seem like something to stash away in case it suddenly becomes unavailable due to government controls. FWIW, China is suddenly talking about model export controls. It was one thing to release also-ran mod…
Thanks for the hint. It makes sense, even if the main driver for releasing the models is to reduce the market size for the US companies.
Do you have any sources?
Edit: "market size"
Earlier quoted context omitted.
I have never seen anyone report "this produced really great results" from intentionally quantizing their context vs. leaving it at full precision which is the ordinary default.
Gemma's QAT is surprisingly good (although Gemma isn't that great to begin with).
Getting 404 on the OP's link. Does it mean it got banned or self-censored in the meantime?
I imagine it's just technical issues on the flip. It's also going to be interesting what happens to HF with loads of people downloading a many TB model. Even though almost no one has the capability to run it, it does seem like something to stash away in case it suddenly becomes unavailable due to government controls. FWIW, China is suddenly talking about model export controls. It was one thing to release also-ran mod…