Longwriter – Increase llama3.1 output to 10k words
1–10 of 30 posts
Re: Longwriter – Increase llama3.1 output to 10k words
#2How to use this with the local ollma setup
Re: Longwriter – Increase llama3.1 output to 10k words
#3Interesting project!
Does anyone know how LongWriter handles maintaining coherence and structure in longer outputs? Also, are there specific strategies or parameters recommended for fine-tuning LLaMA 3.1 with this setup to maximize the quality of generated text?
Re: Longwriter – Increase llama3.1 output to 10k words
#4Paper:
https://arxiv.org/abs/2408.07055
The model is stock llama, fine tuned with a set of long documents to encourage longer outputs.
Most of the action seems to happen in an agent.
Re: Longwriter – Increase llama3.1 output to 10k words
#5Interesting project! Does anyone know how LongWriter handles maintaining coherence and structure in longer outputs? Also, are there specific strategies or parameters recommended for fine-tuning LLaMA 3.1 with this setup to maximize the quality of generated text?
How do people eval these very long outputs?
I've never figured that out (and no I can't just... read all of them)
Re: Longwriter – Increase llama3.1 output to 10k words
#6The sample output is interesting - it has highly suggestive chapter titles which read like pretty normal story beats. It seems like it's guiding itself on these, then able to chunk out longer form writing per chapter.
For what it's worth, the writing is .. bland. In the way that only an LLMs writing can be -- relatively grammatically sound, and totally soulless. I will never think of the love story of Elizabeth and Thomas again, despite having read the entire thing.
In early days of GPT-3, I experimented a lot with getting it respond as certain authors, and it was really quite excellent at that. This is one of the many things that seem likely to have been nerfed over time, I'd guess partly because human preference training just asks for bland responses, and partly because the injected prompts from OpenAI strongly discourage doing things related to real people, and those preferences are carried through, subtlely or not, into the augmented training data most open models tune on.
Re: Longwriter – Increase llama3.1 output to 10k words
#7What the difference between this and using chat history to concatenate outputs and prompting with something like “Now write the next section” repeatedly? I’ve done that with NotebookLM and it’ll write a complete fictional story based on sources, for example.
Re: Longwriter – Increase llama3.1 output to 10k words
#8Really interesting. I wonder if you can do this on ollama too.
Re: Longwriter – Increase llama3.1 output to 10k words
#9What the difference between this and using chat history to concatenate outputs and prompting with something like “Now write the next section” repeatedly? I’ve done that with NotebookLM and it’ll write a complete fictional story based on sources, for example.
Most LLMs are trained to write "complete" outputs. So each section will end up being like a tiny self-contained short book. Without manual editing they will not create long narratives.
Re: Longwriter – Increase llama3.1 output to 10k words
#10What the difference between this and using chat history to concatenate outputs and prompting with something like “Now write the next section” repeatedly? I’ve done that with NotebookLM and it’ll write a complete fictional story based on sources, for example.
In my testing, that often causes the model to 'drift' and ramble wildly compared to just getting one long output from the very start.
The issue is probably that when you split it by just asking for the next section, you're asking it to figure out how to continue from a block that wasn't written with the awareness that it'd have to add on to it.
From the diagram on the repo, I guess this first plans out the structure for each block, and generates the blocks based on the plan.