Live data from Hacker News

Correlation between the model scale and the quality of long text generation?

old.reddit.com

1–2 of 2 posts

Re: Correlation between the model scale and the quality of long text generation?

#2
Recently, I conducted a few fine-tuning experiments on a multitask instruction dataset on 1b7 LLM, primarily related to open-ended long story generation. However, it was observed that after generating nearly 300 tokens, the quality of the generated text started to decline, becoming less fluent.

I am curious to know if there exists a relationship between the scale of the model and the quality of long text generation. For example, is it possible to achieve fluent long text after generating a certain number of tokens for models like 1b7?