Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros

apps.apple.com

Post reply on HN
Print view View original

21–30 of 208 posts

Re: I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros

#21

Post by nl »

YetAnotherNick wrote: post #9
hananova wrote: post #3

Why do none of these apps allow you to set the system prompt? I find these LLM apps kind of useless without being able to refine the way in which the model will respond to later questions.

Mistral instruct doesn't have system prompt AFAIK. Also llama chat system prompt is very useless in my testing.

Mistral Instruct does use a system prompt.

You can see the raw format here: https://www.promptingguide.ai/models/mistral-7b#chat-templat... and you can see how LllamaIndex uses it here (as an example): https://github.com/run-llama/llama_index/blob/1d861a9440cdc9...

Re: I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros

#22

Post by brittlewis12 »

simonw wrote: post #2

Does it save all conversations and let me revisit them later? I use MLC Chat to run Mistral 7B on my iPhone at the moment, but the lack of conversation history is a real nuisance: https://apps.apple.com/us/app/mlc-chat/id6448482937

you can absolutely access and continue all your past chats in cnvrs!

would love to hear what you think: https://testflight.apple.com/join/ERFxInZg

Re: I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros

#24

Post by LewisVerstappen »

Firmwarrior wrote: post #14

Is it weird if I carry a phone with this and a solar charger around at all times, in case I suddenly get hurled back in time?

I think it would be far better to just store a bunch of epubs on your phone in case you get hurled back.

Textbooks on physics, chem, etc.

Re: I made an app that runs Mistral 7B 0.2 LLM locally on iPhone Pros

#27

Post by geuis »

I have a 2020 16in MacBook Pro. I think it's the last generation of Intel chips. I've been struggling to get some of the LLM models like Mixtral to run on it.

I hate the idea of needing to buy another $3k laptop less than 4 years after spending that much on my current machine. But if I want to get serious about developing non-chatgpt services, do I need a new M2 or M3 chip to get this stuff running locally?

Post reply on HN