41–50 of 134 posts
Does anything currently beat GPT-4? I saw some comments here say to check out Claude. From what I can tell, Claude hasn't figure out yet how to do the whole "generate Python code and run it in a Juptyer notebook" for math yet.
We're not in the business of training models. We will never be as good as OpenAI / Anthropic etc.
Where the real value in applications is smarter prompting techniques and RAG. There is a lot of room at the bottom in doing "dumb" things and simply feeding models with the right context to deliver customer value.
Does anything currently beat GPT-4? I saw some comments here say to check out Claude. From what I can tell, Claude hasn't figure out yet how to do the whole "generate Python code and run it in a Juptyer notebook" for math yet.
Debugging a Python function this morning. Claude 3 Opus failed completely. GPT-4 found the bug, as well as two others I hadn’t even been looking for.
Earlier quoted context omitted.
A low-tech example to create a good blog post title for submission to Hacker News would be a system prompt like: You are an expert copywriter. Write five distinct blog post titles optimized for high clickthrough for Hacker News for the article the user provides. Your response must follow the style of these titles: - The ü/ü Conundrum - Why isn't preprint review being adopted? - Majority of web apps could just run on…
interesting thank you. intuitively, prompting like this to get an answer seems basically like the first part of a fine tuning process (more exemplars). what is your thought here behind why reinforcing good output via a loss optimization is worse than the one shot example? does the model start to over fit at some point towards some local minima? and this is avoided in this scenario?
On the other hand, GPT-4 actually did worse on the NER task - labelling and tagging terms used in the text - vs their finetuned model. I assume the finetuned model was better at using the specific labels they were targeting.
Does anything currently beat GPT-4? I saw some comments here say to check out Claude. From what I can tell, Claude hasn't figure out yet how to do the whole "generate Python code and run it in a Juptyer notebook" for math yet.
https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
I lead AI teams at my company. I've advised leadership against any kind of training / fine-tuning anything. We're not in the business of training models. We will never be as good as OpenAI / Anthropic etc. Where the real value in applications is smarter prompting techniques and RAG. There is a lot of room at the bottom in doing "dumb" things and simply feeding models with the right context to deliver customer value.
I lead AI teams at my company. I've advised leadership against any kind of training / fine-tuning anything. We're not in the business of training models. We will never be as good as OpenAI / Anthropic etc. Where the real value in applications is smarter prompting techniques and RAG. There is a lot of room at the bottom in doing "dumb" things and simply feeding models with the right context to deliver customer value.
Extremely hot LLM take: You will often get better results with few-shot prompting (with good examples) on a modern LLM than with a finetuned LLM. Finetuning was the best option for weaker LLMs with lower context windows (e.g. the original GPT-3): both problems have been solved nowadays. The cost economics are much better with few-shot prompting to modern LLMs too: input tokens are super cheap (especially with the rec…
That might be true for finetuning ChatGPT 3.5, but if you can finetune a small model (7B or less) to perform on par with GPT-4, while being faster and private, that’s a different story.
It can make sense if you have a particularly simple use case.