291–300 of 311 posts
Earlier quoted context omitted.
It's aimed at OpenAI's moat. Making sure they don't accumulate too much of one. No one actually has to use this, it just needs to be clear that LLM as a service won't be super high margin because competition can simply start building on Meta's open source releases.
So. Strange as it seems, is Meta being more 'Open', than OpenAI that was created to be the 'open' option to fight off Meta and Google?
Earlier quoted context omitted.
How do you test a 70B model locally? I've tried to query, but the response is super slow.
Personally I was testing with TogetherAI because I don't have the specs for a local 70b. Using quantized versions helps (Ollama's downloads 4-bit by default, you can get down to 2), but it would still require a higher-end Mac. Highly recommend Together, it runs quite quickly and is $0.9/million tokens
Curious what's the current SOTA local copilot model? Are there any extensions in vscode that give you a similar experience? I'd love something more powerful than copilot for local use (I have a 4090, so I should be able to run a decent number of models).
This is a completely fair, but open question. Not to be a typical HN user, but when you say SOTA local, the question is really what benchmarks do you really care about in order to evaluate. Size, operability, complexity, explainability etc. Working out what copilot models perform best has been a deep exercise for myself and has really made me evaluate my own coding style on what I find important and things I look out…
Wow, just realized, in the future employers will mostly interview LLMs instead of people.
Earlier quoted context omitted.
I hope someone will soon post a quantized version that I can run on my macbook pro.
Ollama has released the quantized version. https://ollama.ai/library/codellama:70b https://x.com/ollama/status/1752034686615048367?s=20 Just need to run `ollama run codellama:70b` - pretty fast on macbook.
Earlier quoted context omitted.
Ollama has released the quantized version. https://ollama.ai/library/codellama:70b https://x.com/ollama/status/1752034686615048367?s=20 Just need to run `ollama run codellama:70b` - pretty fast on macbook.
I'm trying to understand how this works.. does it actually run the model on the MacBook Pro? Sorry I am totally new to this...
If you want to try it out, this blog post[3] shows how to do it step by step - pretty straightforward.
[1] https://huggingface.co/docs/optimum/concept_guides/quantizat...
Earlier quoted context omitted.
I'm trying to understand how this works.. does it actually run the model on the MacBook Pro? Sorry I am totally new to this...
Yes, it runs a quantized [1] version of the model locally. This version uses low-precision data types to represent reduced weights and activations (8-bit integer instead of 32-bit). The specific model published by Ollama uses 4-bit quantization [2] and that's why it is able to run on MacBook pro. If you want to try it out, this blog post[3] shows how to do it step by step - pretty straightforward. [1] https://hugging…
Earlier quoted context omitted.
I hope someone will soon post a quantized version that I can run on my macbook pro.
Ollama has released the quantized version. https://ollama.ai/library/codellama:70b https://x.com/ollama/status/1752034686615048367?s=20 Just need to run `ollama run codellama:70b` - pretty fast on macbook.
Earlier quoted context omitted.
> The idea that the company has underperformed for the last 10 years ... is a joke Did you even read the Triad Partners quote from their letter? It's their words, not mine.
There is no quote that says that. "Earnings per share (“EPS”) in the most recent fiscal year were lower than the EPS generated by Disney a decade ago" is not the same as "underpeforming for a decade". all that says is that EPS is currently low, not that it has been low and reducing/stagnant for a decade.
> Disney shareholders were once over $200 billion wealthier than they are now
Is that an exaggeration?
Earlier quoted context omitted.
There is no quote that says that. "Earnings per share (“EPS”) in the most recent fiscal year were lower than the EPS generated by Disney a decade ago" is not the same as "underpeforming for a decade". all that says is that EPS is currently low, not that it has been low and reducing/stagnant for a decade.
Ok, it's quite possible I just didn't understand the writing. How about this part? > Disney shareholders were once over $200 billion wealthier than they are now Is that an exaggeration?
https://filmthreat.com/news/male-and-pale-is-stale-responses...
As a result, the money generated by the industry has seen a
remarkably noticeable drop. The rise of the term “flop buster”
(in response to so many films like Indiana Jones and the
Dial of Destiny underperforming at the box office) seems to
be directly related to mainstream audiences’ wholesale rejection
of the over-messaging they see hijacking entertainment.
Earlier quoted context omitted.
I'm trying to understand how this works.. does it actually run the model on the MacBook Pro? Sorry I am totally new to this...
Yes, it runs a quantized [1] version of the model locally. This version uses low-precision data types to represent reduced weights and activations (8-bit integer instead of 32-bit). The specific model published by Ollama uses 4-bit quantization [2] and that's why it is able to run on MacBook pro. If you want to try it out, this blog post[3] shows how to do it step by step - pretty straightforward. [1] https://hugging…