61–64 of 64 posts
Earlier quoted context omitted.
It's not clear from the link if it's possible to fine tune Flan-T5 on consumer hardware?
They released different sizes from "small" to "xxl", and at least "base" should be small enough to fine-tune virtually anywhere.
Clue on conversation "history"-- "While ChatGPT is able to remember what the user has said earlier in the conversation, there is a limit to how much information it can retain. The model is able to reference up to approximately 3000 words (or 4000 tokens) from the current conversation - any information beyond that is not stored. Please note that ChatGPT is not able to access past conversations to inform its responses.…
Although people unfamiliar with "active listening" often find it annoying, this is something I attempt to implement within my conversations (quickly summarizing what I understand after somebody explains something technical). Oftentimes people feel like I'm `interrupting them`, but it is really just a way to stay engaged in two-way conversation.
Eerie how LLM's benefit from this "focus" as similarly as this autistic commenter.
Earlier quoted context omitted.
They released different sizes from "small" to "xxl", and at least "base" should be small enough to fine-tune virtually anywhere.
I spent several hours on this today. Currently stuck at "out of memory" error trying to fine-tune the base model.
This runs fine in RAM constrained ( https://huggingface.co/EleutherAI/gpt-neo-125M It's bigger brother, 1.3b, uses ~5.5gb of memory but yields slightly more GPT-like answers. Both take ~5-20 seconds to generate a response though, so take that into account when building with it.
Hi from OctoML -- our team just put out some benchmarking results using GPT-J with a few model config/hw combinations that have relatively low latency: /octoml.ai/blog/evaluating-gpt-j-deployment-options/