221–230 of 252 posts
Earlier quoted context omitted.
You can't upgrade it? Edit: I haven't owned a laptop for years, probably could have surmised they'd be more user hostile nowadays.
You are getting downvoted because you vaguely suggested something negative about an Apple product, as is my comment below
It feels absolutely amazing to build an AI startup right now. It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. - We first struggled with limited context windows [solved] - We had issues with consistent JSON ouput [solved] - We had rate limiting and performance issues for the large 3rd party models [solved] - Hosting our own OSS models for small…
If you don't mind, I'm trying to experiment w/ local models more. Just now getting into messing w/ these but I'm struggling to come up w/ good use cases. Would you happen to know of any cool OSS model projects that might be good inspiration for a side project? Wondering what most people use these local models for
I think the real value in using local models is exposing them to personal/unique information that only you have, thus getting novel and unique outcomes that no public model could provide.
1. Project 1 - Self Knowledge - Download/extract all of my emails and populate into a vector database, like Chroma[0] - For each prompt do a search of the vector store and return N number of matches - Provide both prompt and search result to LLM, instructing it to use the search result as context or in the answer itself.
2. Project 2 - Chat with a Friend - I exported the chat and text history between me and a good friend that passed away - I created a vector store of our chat history in chunks, each consisting of 6 back-and-forth interactions - When I "chat" with the LLM the a search is first conducted for matching chunks from the vector store and then using those as "style" and knowledge context for a response. Optional: You can use SillyTavern[1] for a more "rich" chat experience
The above lets me chat, at least superficially, with my friend. It's nice for simple interactions and banter; I've found it to be a positive and reflective experience.
So this one is 3x the size but only 7% better on MMLU? Given Moores law is mostly dead, this trend is going to make for even more extremely expensive compute for next gen AI models.
That's 25% fewer errors.
Earlier quoted context omitted.
These days with Apple Silicon, RAM is a part of the SoC. It's not even soldered on, it's a part of the chip. Although TBF, they also offer insane memory bandwidths.
Yes it’s almost like we got some benefit from iterating on these aspects of hardware design, as opposed to the typical HN grump characterisation of unbridled evil whenever a laptop isn’t exactly like how they were in 2006.
Earlier quoted context omitted.
You are getting downvoted because you vaguely suggested something negative about an Apple product, as is my comment below
FWIW, I am downvoting this comment because you’re whinging, not because I have an issue with which companies you do or do not like.
Both of which seem to be exactly what GP took issue with, and was calling out.
First test I tried to run a random taxation question through it Output: https://gist.github.com/IAmStoxe/7fb224225ff13b1902b6d172467... Within the first paragraph, it outputs: > GET AN ESSAY WRITTEN FOR YOU FROM AS LOW AS $13/PAGE Thought that was hilarious.
It ranks between Mistral Small and Mistral Medium on my NYT Connections benchmark and is indeed better than Command R Plus and Qwen 1.5 Chat 72B, which were the top two open weights models. Grok 1.0 is not an instruct model, so it cannot be compared fairly.
If you are looking to play with the model without installing it locally, we've added it our playground at https://trypromptly.com/playground .
Earlier quoted context omitted.
Not sure why your comment was downvoted. ^ is absolutely the right answer. Open WebUI is functionally identical to the ChatGPT interface. You can even use it with the OpenAI APIs to have your own pay per use GPT 4. I did this.
Hey can you guys elaborate how this works? I'm looking at the Ollama section in their docs and it talks about load balancing? I don't understand what that means in this context.
https://docs.openwebui.com/getting-started/
IIUC the load balancing page is for people who want to run openwebui at a larger scale