How was your experience in switching from GPT to a custom model for a production use case ?
Ask HN: People who switched from GPT to their own models. How was it?
1–10 of 87 posts
Re: Ask HN: People who switched from GPT to their own models. How was it?
#2For personal use I tried this.
I no longer use llms.
Re: Ask HN: People who switched from GPT to their own models. How was it?
#3I fine-tuned an LLM to do technical stuff. It works pretty darn good. What I actually discovered is that when evaluating LLMs, it is surprisingly difficult to evaluate them. And, also, that GPT 4 isn't that great, in general.
Re: Ask HN: People who switched from GPT to their own models. How was it?
#4I fine-tuned an LLM to do technical stuff. It works pretty darn good. What I actually discovered is that when evaluating LLMs, it is surprisingly difficult to evaluate them. And, also, that GPT 4 isn't that great, in general.
Which model did you start with?
Re: Ask HN: People who switched from GPT to their own models. How was it?
#5Running Mistral-Instruct-0.1 for call/email summarization, Mixtral for contract mining & OpenChat to augment agentic chatbot equipped with RAG tools(Instruct again).
Experience has been great, INT8 tradeoffs are acceptable until hardware FP8(FP4 anyone?) becomes more widely & cheaply available. On-prem costs have been absorbed already for few boxes of A100s & legacy V100s running millions of such interactions.
Re: Ask HN: People who switched from GPT to their own models. How was it?
#6I fine-tuned an LLM to do technical stuff. It works pretty darn good. What I actually discovered is that when evaluating LLMs, it is surprisingly difficult to evaluate them. And, also, that GPT 4 isn't that great, in general.
Which base model did you use and what methodologies you used?
Re: Ask HN: People who switched from GPT to their own models. How was it?
#7[dead]
Re: Ask HN: People who switched from GPT to their own models. How was it?
#8Anyone has a tutorial how to achieve it to own a self-hosted model?
Re: Ask HN: People who switched from GPT to their own models. How was it?
#9I fine-tuned an LLM to do technical stuff. It works pretty darn good. What I actually discovered is that when evaluating LLMs, it is surprisingly difficult to evaluate them. And, also, that GPT 4 isn't that great, in general.
Could you provide more details on this matter? Specifically, I'm interested in knowing which base model you've utilized and the approach you've taken to fine-tune it. Your insights would be greatly appreciated and highly beneficial.
Re: Ask HN: People who switched from GPT to their own models. How was it?
#10Anyone has a tutorial how to achieve it to own a self-hosted model?
you can use Ollama and download many models. Performance depends on your laptop's capacity.