Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You probably don’t need to fine-tune an LLM

tidepool.so

Post reply on HN
Print view View original

51–60 of 75 posts

Re: You probably don’t need to fine-tune an LLM

#51

Post by adamgordonbell »

gk1 wrote: post #48
adamgordonbell wrote: post #45

Earlier quoted context omitted.

I am just using GPT4-8k and breaking tasks up. When I tested with Claude 1 ( 2 wasn't out yet) you could feed it lots of context, but it didn't always seem like it paid attention to it all. It would go off the rails. For example, if I put instructions first, then a lot of context, it would forget the task, so instead put a small explanation, then context, then question, but still it would seem to sometimes loose the…

Interesting, thanks for sharing. We also found[1] that context stuffing leads (generally) to worse quality results, as did a recent study.[2] But that's for question-answering use cases and not summarization. [1] https://www.pinecone.io/blog/why-use-retrieval-instead-of-la... [2] https://arxiv.org/pdf/2307.03172.pdf

Great work! Matches my findings but much more rigorous.

The tasks I'm doing might be uncommon. Here is a build script in some language you (the llm) understand, and I want to get it translated to a language you don't. And so the context is example conversions and documentation.

I've also had some luck with things that boil down to "Here is a very large style guide. now how would you improve this code?". Or "here is a number of examples of feedback on writing to conform to a style. now generate the same on this new input."

I found the large context windows and Claude to work quite well in those examples. But, if its possible, breaking it down into multiple steps with less context somehow and using GPT4 even better (though more work).

Re: You probably don’t need to fine-tune an LLM

#52

Post by treprinum »

binarymax wrote: post #7
norwalkbear wrote: post #2

RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches

RAG is the ONLY way to make sure you models are keeping true to facts and source material. Fine-tuning a model before using RAG helps with shaping the style of the summary, and gravitating towards more important facts presented.

There are other methods than RAG over vector database; you can use basic TF-IDF or even full-text search to find candidate paragraphs and put them into the context.

Re: You probably don’t need to fine-tune an LLM

#53

Post by ttul »

gk1 wrote: post #25
adamgordonbell wrote: post #4

I agree with summary. When I first wanted to tackle a hard problem I thought to reach for fine-tuning with lots of input and output pairs, but it wasn't needed. Past few shot and RAG, you can overcome context window limits if you find ways to break a single request into many, each with specific context and then roll them up somehow. This can help get past context window limits. Claude 2 has a large context window, bu…

How much do you make use of Claude's large context window, and has it helped? Or did you basically revert back to small context size + multiple steps?

In my experience, Claude 2 is marvellously good at sucking in massive documents and accurately generating responses to questions. I gave it the entire Georgia indictment (minus a few pages of crap at the start that was irrelevant to reduce token count) and it wrote a NYTimes article based on the indictment that could be favourably compared to the actual NYTimes piece summarizing the same. There were no factual errors in its output.

I imagine OpenAI is not far behind in expanding the context window of its models. The LLM companies have access to the same techniques and - in my estimation - are just choosing to focus on one aspect or another to address different market needs. For instance, Claude 2 clearly focuses on maximum context window size at the cost of speedy inference and, presumably, inference cost. By contrast, OpenAI seems to be focused on speed and low cost (GPT-3.5) and accuracy (GPT-4) rather than maximum token length.

Re: You probably don’t need to fine-tune an LLM

#54

Post by gsuuon »

herdcall wrote: post #15

We're using LLMs (OpenAI) to generate SQL queries to search customer data, and the current approach using chat API frequently generates queries using the wrong record/column names. I'm exploring use of fine tuning to improve accuracy on a customer/customer basis to train on their set of data, isn't that a good use case?

I think this is something that sample biasing would work better for, which you could do with local LLM's. For example with ad-llama[1] you would just have a sampler bias like so:

  const knownColumns = ['name', 'email', 'id']

  template`
    SELECT "${a('column name', {
      sampler: bias.accept(oneOf(knownColumns))
    })}"
    FROM "table"
  `
You're able to enforce, at the sampler level, that the output is one of the expected choices.

[1] https://ad-llama.vercel.app/playground/

Re: You probably don’t need to fine-tune an LLM

#55

Post by mark_l_watson »

For most use cases this article is right on target.

I have been self hosting a 16K context size model and there is a lot you can do with 16K or larger context.

There are also great use cases for fine tuning. For example, if you are writing a chatbot for your company’s products, it might make sense to fine tune on product data and then RAG it with specific customer data when setting up a chat session.

Re: You probably don’t need to fine-tune an LLM

#56

Post by 011021 »

gk1 wrote: post #43
gdiamos wrote: post #42

How can you edit or remove guardrails without fine tuning?

How would guardrails be applied through fine-tuning? I haven't heard of that, so genuinely curious. What I'm familiar with is guardrails applied on top of the LLM. For instance through prompting, managing the available data inside the vector database that could be used for context (if using RAG), or through something like NeMo[1]. https://www.pinecone.io/learn/nemo-guardrails-intro/

for example by using a full fine-tuning it's possible to censor a LLama model. then people use the same dataset, but filter out guardrails to create an uncensored type of this model. just google for "llama wizard vicuna uncensored".

Re: You probably don’t need to fine-tune an LLM

#58

Post by jdmccarty »

Is there a method for this to be "Augmented" and not "Replacement" eg in the example from the blog post, "retriever=vectorstore.as_retriever()" which I believe would return something like "I don't know" if the content is not in the vectorstore.

In humans, a person might say something like, "I'm not an expert, but X" and I think being able to default back to the underlying LLM would be useful.

Re: You probably don’t need to fine-tune an LLM

#59

Post by JamesBarney »

valgaze wrote: post #39

Fine-tuning is such a dangerous phrased bc it sounds perfect. “We’ll just fine-tune based on our (we think valuable + special) data” Fine-tuning doesn’t enhance the model w/ new “knowledge” but a new narrowly-defined task One other “cost” to consider is fine-tuning a 3rd-party model means if that foundation model changes or goes away that effort/cost needs to be repeated

My understanding of fine-tuning is 95% of the work could be re-used between different foundation models.

Re: You probably don’t need to fine-tune an LLM

#60

Post by binarymax »

treprinum wrote: post #52
binarymax wrote: post #7

Earlier quoted context omitted.

RAG is the ONLY way to make sure you models are keeping true to facts and source material. Fine-tuning a model before using RAG helps with shaping the style of the summary, and gravitating towards more important facts presented.

There are other methods than RAG over vector database; you can use basic TF-IDF or even full-text search to find candidate paragraphs and put them into the context.

Of course. “Retrieval” in RAG doesn’t require a special kind of retriever. As long as the relevance is tuned for the top documents to seed the prompt context, it doesn’t matter what kind of search backend you use.
Post reply on HN