Someone needs to make an LLM purpose-built for creating high-quality datasets for fine-tuning other LLMs.
This. The best use of the current llms is to create better Datasets.
Fine-tune your own Llama 2 to replace GPT-3.5/4
71–80 of 194 posts
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#72Is Llama 2 currently the way to go for fine-tuning your own models? Are there other open-source LLMs worth considering?
It's one of widely fine tuned model for now. Take a look at this colab for fine tuning on your dataset https://github.com/mlabonne/llm-course/blob/main/Fine_tune_L...
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#73For translation jobs, I've experimented with Llama 2 70B (running on Replicate) v/s GPT-3.5; For about 1000 input tokens (and resulting 1000 output tokens), to my surprise, GPT-3.5 turbo was 100x cheaper than Llama 2. Llama 7B wasn't up to the task fyi, producing very poor translations. I believe that OpenAI priced GPT-3.5 aggressively cheap in order to make it a non-brainer to rely on them rather than relying on oth…
Yes, openAI is dumping the market with chat-gpt 3.5. Vulture capital behaviour at its finest, and I'm sure government regulations will definitely catch on to this in 20 or 30 years... It's cheaper than the ELECTRICITY cost of running a llama-70 on your own M1.Max (very energy efficient chip) assuming free hardware. I guess they are also getting a pretty good cache hit rate - there are only so many questions people as…
turbo is likely nowhere near 70b.
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#74This post made me think of human hierarchies. Line level ICs are cheap because they are specialized and fine tuned. Leet code is a way to roughly measure degree of fine-tuning even though it doesn't accurately measure how well the fine tuning is for the job. As you go up the hierarchy what you want is higher quality answers to more and more abstract and general questions. AGI, God, CEOs, and figures like Paul Graham,…
> AGI, God, CEOs, and figures like Paul Graham, Elon Musk
hacker news pantheon just dropped
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#75For translation jobs, I've experimented with Llama 2 70B (running on Replicate) v/s GPT-3.5; For about 1000 input tokens (and resulting 1000 output tokens), to my surprise, GPT-3.5 turbo was 100x cheaper than Llama 2. Llama 7B wasn't up to the task fyi, producing very poor translations. I believe that OpenAI priced GPT-3.5 aggressively cheap in order to make it a non-brainer to rely on them rather than relying on oth…
Google Maps was also cheap. Initially.
So it is aggressively cheap now, but would aggressively change later.
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#76For translation jobs, I've experimented with Llama 2 70B (running on Replicate) v/s GPT-3.5; For about 1000 input tokens (and resulting 1000 output tokens), to my surprise, GPT-3.5 turbo was 100x cheaper than Llama 2. Llama 7B wasn't up to the task fyi, producing very poor translations. I believe that OpenAI priced GPT-3.5 aggressively cheap in order to make it a non-brainer to rely on them rather than relying on oth…
Yes, openAI is dumping the market with chat-gpt 3.5. Vulture capital behaviour at its finest, and I'm sure government regulations will definitely catch on to this in 20 or 30 years... It's cheaper than the ELECTRICITY cost of running a llama-70 on your own M1.Max (very energy efficient chip) assuming free hardware. I guess they are also getting a pretty good cache hit rate - there are only so many questions people as…
Based on my research, GPT-3.5 is likely significantly smaller than 70B parameters, so it would make sense that it's cheaper to run. My guess is that OpenAI significantly overtrained GPT-3.5 to get as small a model as possible to optimize for inference. Also, Nvidia chips are way more efficient at inference than M1 Max. OpenAI also has the advantage of batching API calls which leads to better hardware utilization. I don't have definitive proof that they're not dumping, but economies of scale and optimization seem like better explanations to me.
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#77For translation jobs, I've experimented with Llama 2 70B (running on Replicate) v/s GPT-3.5; For about 1000 input tokens (and resulting 1000 output tokens), to my surprise, GPT-3.5 turbo was 100x cheaper than Llama 2. Llama 7B wasn't up to the task fyi, producing very poor translations. I believe that OpenAI priced GPT-3.5 aggressively cheap in order to make it a non-brainer to rely on them rather than relying on oth…
Yes, openAI is dumping the market with chat-gpt 3.5. Vulture capital behaviour at its finest, and I'm sure government regulations will definitely catch on to this in 20 or 30 years... It's cheaper than the ELECTRICITY cost of running a llama-70 on your own M1.Max (very energy efficient chip) assuming free hardware. I guess they are also getting a pretty good cache hit rate - there are only so many questions people as…
gpt3.5 turbo is (mostly likely) Curie which is (most likely) 6.7b params. So, yeah, makes perfect sense that it can't compete with a 70b model on cost.
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#78Thanks! When it comes to choosing where to work with these models, which compute platform do you recommend (assuming locally doesn't really make sense with my resources)?
Colab?
AWS StudioLab?
Which is your go to?
Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#79Re: Fine-tune your own Llama 2 to replace GPT-3.5/4
#80Earlier quoted context omitted.
Yes, openAI is dumping the market with chat-gpt 3.5. Vulture capital behaviour at its finest, and I'm sure government regulations will definitely catch on to this in 20 or 30 years... It's cheaper than the ELECTRICITY cost of running a llama-70 on your own M1.Max (very energy efficient chip) assuming free hardware. I guess they are also getting a pretty good cache hit rate - there are only so many questions people as…
Based on my research, GPT-3.5 is likely significantly smaller than 70B parameters, so it would make sense that it's cheaper to run. My guess is that OpenAI significantly overtrained GPT-3.5 to get as small a model as possible to optimize for inference. Also, Nvidia chips are way more efficient at inference than M1 Max. OpenAI also has the advantage of batching API calls which leads to better hardware utilization. I d…
I also do not have proof of anything here, but can't it be both?
They have lots of money now and the market lead. They want to keep the lead and some extra electricity and hardware costs are surely worth it for them, if it keeps the competition from getting traction.