41–50 of 90 posts
I got this model working on a GPU instance, notes here: https://til.simonwillison.net/llms/dolly-2 Anyone managed to run it on an M1/M2 Mac yet?
I’m not seeing how 15k q/a training can get you much other than the simplest things. Maybe that’s the point, get the ball rolling for people to add more training data?
What reasons do you have for believing that is true? It seems plausible to me that a general autoregressive LLM that is capable of completing text wouldn't take that much fine-tuning to shift it from "text completion" to "instruction following". After all, the raw GPT3 model can be made to follow instructions with just a few examples. Consider the prompt: What is the capital of France? Raw GPT3, not the newer instruc…
I got this model working on a GPU instance, notes here: https://til.simonwillison.net/llms/dolly-2 Anyone managed to run it on an M1/M2 Mac yet?
How much ram is likely needed on an apple arm for models like this? And for general use, 64, 96, 128? Trying to decide how large I should go for a new laptop.
Earlier quoted context omitted.
Thanks for the feedback. The potential edge with Dolly is huge. Building a firewalled model with custom corpus is a big deal. I have been experimenting with openai and even with public data (but really limiting to the domain), yields great improvements (openai may be stale because of cut off data). I am excited to see where Dolly goes.
Dolly appears to fundamentally be a tech demo advertising how you can use Databricks for compute. I honestly wouldn't expect them to take it that much further, particularly in the context of larger models that would be significantly more expensive to fine-tune. But I'm happy to be proven wrong.
Earlier quoted context omitted.
What reasons do you have for believing that is true? It seems plausible to me that a general autoregressive LLM that is capable of completing text wouldn't take that much fine-tuning to shift it from "text completion" to "instruction following". After all, the raw GPT3 model can be made to follow instructions with just a few examples. Consider the prompt: What is the capital of France? Raw GPT3, not the newer instruc…
Just saying if you ask for capital of an obscure country that it hasn’t been trained on, you will not get the answer, so 15k will get you come general stuff only within the confines. Also, to code you will need pretty complete documentation for it to ingest and then enough examples on how the code is done
With the raw LLM, you can get the capital of Mongolia with the prompt "The capital of Mongolia is", i.e. text completion. The fine-tuning allows you to get at that information by asking questions or giving commands, e.g. "Tell me the capital of Mongolia"
Amazing. Love databricks.
It's certainly better than what we did prior to Databricks, which was roll our own in-house provisioning and notebook solution. I won't/can't go into too many details, but not only was it cumbersome and very buggy, but it was as if they designed it to encourage data scientists to spend as much money on compute as possible (only to panic at the millions they were spending). They dropped it for cost reasons, which is hilarious given how expensive Databricks is.
I do appreciate the work Databricks have done improving Spark. Capabilities like adaptive query execution have made optimization significantly easier.
I got this model working on a GPU instance, notes here: https://til.simonwillison.net/llms/dolly-2 Anyone managed to run it on an M1/M2 Mac yet?
How much ram is likely needed on an apple arm for models like this? And for general use, 64, 96, 128? Trying to decide how large I should go for a new laptop.
I got this model working on a GPU instance, notes here: https://til.simonwillison.net/llms/dolly-2 Anyone managed to run it on an M1/M2 Mac yet?
I got this model working on a GPU instance, notes here: https://til.simonwillison.net/llms/dolly-2 Anyone managed to run it on an M1/M2 Mac yet?
Most docs Ive read on setting up finetuners and inference require some extra stuff. Taking some LORA fine tuners, they include instructions like this:
conda create -n llm-finetuner python=3.10
conda activate llm-finetuner
conda install -y cuda -c nvidia/label/cuda-11.7.0
conda install -y pytorch=2 pytorch-cuda=11.7 -c pytorch