11–20 of 157 posts
M1 is only 3 years old and no one cares to support intel macs any more. There are surely a lot of them out there. Are they that much worse to run LLMs on?
The demo on this video is from Intel Mac https://youtu.be/C0GmAmyhVxM?si=puTCpGWButsNvKA5
It also supports openai compatible api and completely open-source unlike LM studio
M1 is only 3 years old and no one cares to support intel macs any more. There are surely a lot of them out there. Are they that much worse to run LLMs on?
Ollama works super fine on Intel Mac The demo on this video is from Intel Mac https://youtu.be/C0GmAmyhVxM?si=puTCpGWButsNvKA5 It also supports openai compatible api and completely open-source unlike LM studio
M1 is only 3 years old and no one cares to support intel macs any more. There are surely a lot of them out there. Are they that much worse to run LLMs on?
Ollama works super fine on Intel Mac The demo on this video is from Intel Mac https://youtu.be/C0GmAmyhVxM?si=puTCpGWButsNvKA5 It also supports openai compatible api and completely open-source unlike LM studio
This app could use some simple UI improvements: - The chatbox field has a normal "write here" state, when no chat is really selected. I thought my keyboard broke until I discovered that - I didn't find a way to set cuda acceleration before loading a model, only managed to set gpu offloaded layers and using "relaunch to apply" - Some HugginFace models are simply not listed and there's no indication about why. I guess…
This app could use some simple UI improvements: - The chatbox field has a normal "write here" state, when no chat is really selected. I thought my keyboard broke until I discovered that - I didn't find a way to set cuda acceleration before loading a model, only managed to set gpu offloaded layers and using "relaunch to apply" - Some HugginFace models are simply not listed and there's no indication about why. I guess…
So if you suspect I am using this for business related purposes you may take any action you want to spy on me? Such Terms. Very Use.
Thats why as a business i would rather use a trusted FOSS LLm interface like textgeneration-WEBUI https://github.com/oobabooga/text-generation-webui
I don't mean this as a criticism, I'm just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who can't run one of the many open source local llm software? It looks in the screenshot like it's exposing much of the complexity of configuration anyway. Is the value in the interface and management of conversation and models? It would be nice to see info or even…
1. People who understand LLMs and know how to run them and have access to run them on the cloud. 2. People who understand LLMs well enough but don’t have access to cloud resources - but still have a decent MacBook Pro. Or maybe access to cloud resources is done via overly tight pipelines. 3. People who are interested in LLMs but don’t have enough technical chops/time to get things going with Llama CPP. 4. People who are fans of LLMs but can’t even install stuff in their computer.
This is clearly for #3 and it works well for that group of people. It could also be for #2 when they don’t want to spin up their own front end.
I don't mean this as a criticism, I'm just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who can't run one of the many open source local llm software? It looks in the screenshot like it's exposing much of the complexity of configuration anyway. Is the value in the interface and management of conversation and models? It would be nice to see info or even…