331–340 of 360 posts
Earlier quoted context omitted.
You know she has advisors, right?
If someone "advises" you that a company is about to do something major, and this isn't public information, and you take action on the stock market accordingly, that's insider trading.
Earlier quoted context omitted.
Are you alleging that Nancy Pelosi invested in Databricks, a private company without a fluctuating share price, because she learned that they would soon release a small, fairly middling LLM that probably won't move the needle in any meaningful way?
Are you suggesting that Nancy Pelosi, who consistently beats the market through obvious insider trading for years in a row, bought a share in Databricks without any insider info? Possible, yet unlikely is my opinion. https://jacobin.com/2021/12/house-speaker-paul-stocks-inside... PS: "without a fluctuating share price" is non-sense. Just because the share is of a private company, doesn't mean its price can't fluctuat…
Earlier quoted context omitted.
If someone "advises" you that a company is about to do something major, and this isn't public information, and you take action on the stock market accordingly, that's insider trading.
US Congress members are generally immune from insider trading laws
https://en.wikipedia.org/wiki/STOCK_Act
For instance, Pelosi in the Databricks case gets to purchase significant shares at pre-IPO prices, which is a thing that shouldn't even exist.
I am planning to buy a new GPU. If the GPU has 16GB of VRAM, and the model is 70GB, can it still run well? Also, does it run considerably better than on a GPU with 12GB of VRAM? I run Ollama locally, mixtral works well (7B, 3.4GB) on a 1080ti, but the 24.6GB version is a bit slow (still usable, but has a noticeable start-up time).
I am planning to buy a new GPU. If the GPU has 16GB of VRAM, and the model is 70GB, can it still run well? Also, does it run considerably better than on a GPU with 12GB of VRAM? I run Ollama locally, mixtral works well (7B, 3.4GB) on a 1080ti, but the 24.6GB version is a bit slow (still usable, but has a noticeable start-up time).
I genuinely recommend considering AMD options. I went with a 7900 XTX because it has the most VRAM for any $1000 card (24 GB). NVIDIA cards at that price point are only 16 GB. Ollama and other inference software works on ROCm, generally with at most setting an environment variable now. I've even run Ollama on my Steam Deck with GPU inferencing :)
Funnily, I think the card is new (smells new) and unused, most likely a scalper bought it and couldn't sell it.
I would note the actual leading models right now (IMO) are: - Miqu 70B (General Chat) - Deepseed 33B (Coding) - Yi 34B (for chat over 32K context) And of course, there are finetunes of all these. And there are some others in the 34B-70B range I have not tried (and some I have tried, like Qwen, which I was not impressed with). Point being that Llama 70B, Mixtral and Grok as seen in the charts are not what I would call…
Qwen1.5-72B-Chat is dominant in the Chatbot Arena leaderboard, though. (Miqu isn't on there due to being bootleg, but Qwen outranks Mistral Medium.)
And I don't think I was using it wrong. I know, for instance, the Chinese language models are funny about sampling since I run Yi all the time.
Earlier quoted context omitted.
I didn’t suggest a free lunch, just that the 8x reduction in RAM (+ faster processing) does not result in an 8x growth in the error. Thus a quantized model will outperform a non-quantized one on a evaluation/RAM metric.
That's not a good metric.
The approval on the base model is not feeling very open. Plenty of people still waiting on a chance to download it, where as the instruct model was an instant approval. The base model is more interesting to me for finetuning.
Earlier quoted context omitted.
Many applications dont want to host inference on the cloud and would ideally run things locally. Hardware constraints is clearly important. Id actually say its the most important metric for most open models now, since the price per performance of closed cloud models is so competitive with open cloud models, so edge inference that is competitive is a clear value add
It's not that memory usage isn't important, it's that dividing error by memory gives you a useless number. The benefit from incremental error decrease is highly nonlinear, as with memory. Improving error by 1% matters a lot more starting from 10% error than 80%. Also a model that used no memory and got everything wrong would have the best score.
Earlier quoted context omitted.
> chart crime of truncating the y axis If you chart the temperature of the ocean do you keep the y-axis anchored at zero Kelvin?
If you chart the temperature of the ocean are you measuring it in Kelvin?