Codestral Mamba
mistral.ai
Codestral Mamba
1–10 of 148 posts
Re: Codestral Mamba
#2It's great to see a high-profile model using Mamba2!
Re: Codestral Mamba
#3Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally?
This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc.
I want to see if I have any hardware I can stress and run something locally, but don’t know where to start or even what are the available options.
Re: Codestral Mamba
#4Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally? This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc. I want to see if I have any hardware I can stress and run something locally, but d…
If I understand correctly what you are looking for, Ollama might be a solution (https://ollama.com/)?. I have no affiliation, but I lazily use this solution when I want to run a quick model locally.
Re: Codestral Mamba
#5Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally? This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc. I want to see if I have any hardware I can stress and run something locally, but d…
If you mean LLM in general, maybe try llamafile first
Re: Codestral Mamba
#6Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally? This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc. I want to see if I have any hardware I can stress and run something locally, but d…
Find good models here: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_...
Check hardware requirements here: https://rahulschand.github.io/gpu_poor/
Re: Codestral Mamba
#7Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally? This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc. I want to see if I have any hardware I can stress and run something locally, but d…
not sure about the history/progression part, but there's ollama which makes it possible to run models locally. The UX of ollama is similar to docker.
Re: Codestral Mamba
#8What are the steps required to get this running in VS Code?
If they had linked to the instructions in their post (or better yet a link to a one click install of a VS Code Extension), it would help a lot with adoption.
(BTW I consider it malpractice that they are at the top of hacker news with a model that is of great interest to a large portion of the users where and they do not have a monetizable call to action on the page featured.)
Re: Codestral Mamba
#9Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally? This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc. I want to see if I have any hardware I can stress and run something locally, but d…
Most the 7b instruct models are very bad outside very simple queries.
You can run a 7b on most modern hardware.How fast will vary.
To run 30-70b models you're getting in the realm of needing 24gb or more of vRAM.
Re: Codestral Mamba
#10Any recommended product primers to Mamba vs Transformers - pros/cons etc?