Fantastic article if you are quick to just go to the comments like I usually do, don't. Read it. One of my favorites: LoRA works by representing model updates as low-rank factorizations, which reduces the size of the update matrices by a factor of up to several thousand. This allows model fine-tuning at a fraction of the cost and time. Being able to personalize a language model in a few hours on consumer hardware is…
If you use the web interface (oobabooga), then training a LoRa is as easy as clicking the "training" tab, keeping all the defaults, and giving it a flat text file of your data. The defaults are sane enough to not begin undermining any instruction tuning too much. Takes 3-5 hours on a 3080 for 7B, 4bit model (and ~1KWh). So far I've trained 3: 2 on the entire text of ASOIAF (converted from e-books) and 1 on the Harry…
Re: Google “We have no moat, and neither does OpenAI”
#1001Which model did you use? I read your comment when this HN thread was alive and decided to try to feed it a few websites (such as PEP8[0] for Python), and it only took a few minutes for LoRa training to complete, and the answers were not good. I also have a 3080.