Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You probably don’t need to fine-tune an LLM

tidepool.so

Post reply on HN
Print view View original

71–75 of 75 posts

Re: You probably don’t need to fine-tune an LLM

#71

Post by fnordpiglet »

gsuuon wrote: post #63
fnordpiglet wrote: post #61

Earlier quoted context omitted.

I fine tune smaller LLMs with fanfic to generate stories in a genre. For instance, most models have no meaningful Minecraft content. By feeding lots of fanfic into a Lora and supplementing with RAG, I get pretty good results. RAG alone is garbage as there’s too much broad context on say Herobrine spanning many many stories, but the base models know almost nothing. Few shot doesn’t help because there’s not enough sema…

That's really interesting! I've wanted to do something similar but assumed that the token count required to ingrain a meaningful amount of information would make it very difficult. How many tokens do you typically need to feed into these base models to get consistent content out? Does generated text ever end up leaving the specialized domain? (like League of Legends characters mixed into a Minecraft story)

I’ve not done anything particularly scientific to measure but my observation is if symbols are fairly rare it doesn’t require a huge amount. Herobrine for instance as a string isn’t very common, so it seems to be pretty good at keeping in the semantic region. Steve and Alex are popular Minecraft characters but are not very unique so you see more drift. But the more you mix in specific semantics, like creepers, or other tokens that are pretty specific to the genre, things improve. You see similar behavior with stable diffusion Lora.

I’ve never seen it wander between genres, but it can tell pretty generic stories particularly if you’re pretty generic in your promoting.

Re: You probably don’t need to fine-tune an LLM

#73

Post by a_ml »

simonw wrote: post #21
norwalkbear wrote: post #2

RAG sucks. Microsoft is the force behind it because they don't allow training on their chatgpt models. Fine-tuning even Lora on the open source models is nearly always better than these other approaches

"Fine-tuning even Lora on the open source models is nearly always better than these other approaches" Can you expand on that? I've not seen evidence of that myself yet, but maybe I haven't looked in the right places.

From Aug 2022, but still worth a look

"Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning" https://arxiv.org/abs/2205.05638

Re: You probably don’t need to fine-tune an LLM

#74

Post by binarymax »

treprinum wrote: post #64
binarymax wrote: post #60

Earlier quoted context omitted.

Of course. “Retrieval” in RAG doesn’t require a special kind of retriever. As long as the relevance is tuned for the top documents to seed the prompt context, it doesn’t matter what kind of search backend you use.

Agreed though R in RAG typically means specifically vector search. Not sure how this was called before vector DBs were popularized by LLaMAIndex, probably just retrieval systems with LLMs.

That's not true. Information Retrieval needs much more than vector search. I've contributed to a book on the subject: https://aipoweredsearch.com

Re: You probably don’t need to fine-tune an LLM

#75

Post by QuantumGood »

binarymax wrote: post #74
treprinum wrote: post #64

Earlier quoted context omitted.

Agreed though R in RAG typically means specifically vector search. Not sure how this was called before vector DBs were popularized by LLaMAIndex, probably just retrieval systems with LLMs.

That's not true. Information Retrieval needs much more than vector search. I've contributed to a book on the subject: https://aipoweredsearch.com

Any more info? The book's not published yet, I see.
Post reply on HN