151–160 of 208 posts
I see the efforts required to create the little app, but inference via llama.cpp or core Ml is trivial and the models are open weights, so it makes more sense to have a free app for this: most of the value is in the LLM which is free.
Earlier quoted context omitted.
No, every user input is surrounded by those tags. Scroll down from that link you posted and read the next two example prompts.
No this isn't the case. Here's the example from https://www.promptingguide.ai/models/mistral-7b#chat-templat... [INST] What is your favorite condiment? [/INST] "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!" [INST] The right amount of what? [/INST] Note that the sentence starting "Well, I'm quite partial isn't…
messages = [
{"role": "user", "content": "What is your favourite condiment?"},
{"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
{"role": "user", "content": "Do you have mayonnaise recipes?"}
]I made a free and open source Windows equivalent: https://github.com/Const-me/Cgml/releases/tag/1.1
I have a 3060 on a laptop and it is faster than gpt4. I use it all the time.
How can you possibly use this? I tried doing CPU based on a bleeding edge laptop and couldn't use it.
Some other local LLM iOS apps: - MLC Chat: https://llm.mlc.ai - LLM Farm: https://llmfarm.site - Enchanted (not local, just a frontend): https://github.com/AugustDev/enchanted But I don't think any of these support Mistral 0.2 which is a pretty big deal.
where can i find info about Mistral 0.2? why is it harder to support than Mistral 0.1?
Earlier quoted context omitted.
Kiwix supports any .zim file, there are plenty of different wikipedia versions available, check https://download.kiwix.org/zim/wikipedia/?C=S;O=D
Thanks, but wikipedia_en_all_nopic_2023-12.zim is still 56 GB, whereas the BZ2-compressed Wiki2Touch archives are only about 14 GB for the latest (and only 8 GB for an archive from 2012 which I'm using).
Earlier quoted context omitted.
Understood but at some point it becomes the responsibility of the user of the hammer if they use it in an attack or hurt someone else or themselves with it. LLMs are LLMs anyone who is using it who doesn’t understand it is language model and is a machine and how at the high level it works, probably shouldn’t use it, and it shouldn’t be Apple’s responsibility to keep hammers out of the hands of everyone due to the few…
Apple forcefully takes on that responsibility, and their customers love it (as clearly evidenced by their domination of the market). If you don't want a hammer that gets reviewed and screened by Apple before you can use it, then you're on the wrong platform.
Your average person doesn't spend a second thinking about Apple's app store policies. In fact most people don't even install new apps unless absolutely necessary.
Earlier quoted context omitted.
Understood but at some point it becomes the responsibility of the user of the hammer if they use it in an attack or hurt someone else or themselves with it. LLMs are LLMs anyone who is using it who doesn’t understand it is language model and is a machine and how at the high level it works, probably shouldn’t use it, and it shouldn’t be Apple’s responsibility to keep hammers out of the hands of everyone due to the few…
Apple forcefully takes on that responsibility, and their customers love it (as clearly evidenced by their domination of the market). If you don't want a hammer that gets reviewed and screened by Apple before you can use it, then you're on the wrong platform.
Probably an alternative version of this app or a similar app can provide an option to load your own models. Is that a problem for Apple to allow?
What I have found in my personal (and perhaps biased and anecdotal experience) is that there is a large cadre of LLM and AI hating people who inevitably start rambling about (a) safety and (b) copyright violations. Which I find reflect more about their mental model of the world and less about reality, in that they tends to be statists and collectivists that want a central authority to “protect” them. Which obviously get’s under my skin as that unfortunately mass instinct is what enabled big government, mass surveillance, and totalitarianism. Just my 2 cents, which undoubtedly many on HN may disagree with, and that’s fine, especially as I’m actually very interested in hearing more specifics from the AI safeguards crowd as admittedly perhaps I’m missing something terrible about this new hammer that has been invented, a hammer I find to be incredibly useful for nailing in all sorts of ways.
They shouldn’t be just chasing the api revenue stream like OpenAI
Earlier quoted context omitted.
would love for you to give cnvrs a shot! - save characters (system prompt + temperature, and a name & cosmetic color) - download & experiment with models from 1b, 3b, & 7b, and quant options q2k, q4km, q6k - save, search, continue, & export past chats along with smaller touches: - custom theme colors - haptics and more coming soon! https://testflight.apple.com/join/ERFxInZg
Do not download this. I downloaded this on my 14 Pro and it completely locked up the system to the point where even the power button wouldn’t work. I couldn’t use my phone for about 10 minutes.
So rather than reporting in the app you can report it in Feedback Assistant, if you want to.