Thanks for sharing this, I found it helpful as an addition to my homebrew curriculum for learning how to fine-tune open source LLMs.
Can you please point me to good resources on fine tuning? Thanks.
Fine-tuning Mistral 7B on Magic the Gathering Draft
51–60 of 101 posts
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#52I like that this shows how hard even conceptually simple ideas are to achieve in fine-tuning LLMs. Even given a pretty good starting dataset, a decent starting model, etc. this appears to have been a challenge. One thing it did make me think about was that these models are suitable for things that don't have a natural definitive answer. That is, picking the perfect card given a set of picks is probably combinatoriall…
I wonder if you could define a specific complexity class of problems that LLMs are good at
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#53Thanks for sharing this, I found it helpful as an addition to my homebrew curriculum for learning how to fine-tune open source LLMs.
Can you please point me to good resources on fine tuning? Thanks.
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#54I like that this shows how hard even conceptually simple ideas are to achieve in fine-tuning LLMs. Even given a pretty good starting dataset, a decent starting model, etc. this appears to have been a challenge. One thing it did make me think about was that these models are suitable for things that don't have a natural definitive answer. That is, picking the perfect card given a set of picks is probably combinatoriall…
That lines up with my experience- for high-stakes decisions, they rarely give me a great answer. But for low stakes decisions, they do well at giving me a good enough answer. For example, I've been using them to help find gifts for friends and children this month. I don't need the best choice to solve the problem, just a good one.
How much additional calculation occurs in high-stakes decisions by individuals. Also what is the variability in quality of high stakes decisions in humans?
I'm guessing LLM decision is rather average, but that the LLM has no easy way of spending the extra time to gather information around said high stakes decisions like a human would.
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#55In case you didn't see it, https://news.ycombinator.com/item?id=38525978 (I hacked Magic the Gathering: Arena for a 100% win rate) may interest this audience if for no other reason that the investigator discovered that Sparky, the pseudo-AI in MTGA, doesn't appear to be as stupid complicated as one may have suspected from the outside
Sparky is the Arena AI, but no one ever accused it of being a good Arena AI - it is very much only there for the new player experience of playing against a dumb computer when you're first exposed to the game and don't know the rules, or for the computer equivalent of "playing against a goldfish" a deck you made to see how it draws or combos. It's not a Chess CPU.
I hope I also did not accuse it of being good, but the observation I was trying to make is that -- according to the article, I have not myself confirmed the claim -- they run the card evaluation logic and gameplanning locally, not in a data center full of H100s, which I consider to be quite a feat given the free-text-y self-modifying rules of M:TG
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#56For some reason I thought fine tuning is not possible without specialized hardware (A100 / H100). Where can I learn more about hardware requirements for fine tuning on consumer GPUs?
Definitely possible on even older off-the-shelf hardware. I use 24GB 4090s for 13b-sized models and have even used 12GB Titans for 7b models, admittedly at much slower rates.
You can also use Apple silicon for this: https://www.reddit.com/r/LocalLLaMA/comments/15y9m64/fine_tu...
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#57Thanks for sharing this, I found it helpful as an addition to my homebrew curriculum for learning how to fine-tune open source LLMs.
Can you please point me to good resources on fine tuning? Thanks.
Search for articles showing you code for fine-tuning Llama 2, ideally including a colab notebook that you can run and modify yourself so that you have real code to work with. You can try to modify their working example to suit your own toy project as a first step.
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#58I wonder if you could use a smaller model or get better results if you treated each card as a token, gave the state of the draft as an input and the predicted token would be the card to pick. You woukd have to train from scratch with a custom tokenizer.
I tried adding special tokens for a reddit-style dataset once. The format was: ` username title here...` The resulting model was so much worse than just formatting everything plaintext. This was with MPT-30B, 15 special tokens, 300M training tokens, and a full finetune. I may have made a mistake, but I haven't seen any open source finetunes successfully add a large number of tokens yet either.
Try doing the same thing in your dataset, but don't actually add them as "special tokens", and just let them just be multiple tokens.
Adding new tokens needs a ton of data to train what the token means. Reusing existing tokens, will allow you to easily teach that a sequence of tokens now has a new meaning after fine tuning.
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#59> I ended up renting an hourly GPU from Runpod (an RTX 4090 w/ 24GB of VRAM) for ~$0.7/hr.
Sorry if I missed this, but how much did it cost total to do the fine-tune? Is that the 40 hour number (~$27)?
Also, very cool writeup. Thanks for sharing!
Re: Fine-tuning Mistral 7B on Magic the Gathering Draft
#60For some reason I thought fine tuning is not possible without specialized hardware (A100 / H100). Where can I learn more about hardware requirements for fine tuning on consumer GPUs?
Definitely possible on even older off-the-shelf hardware. I use 24GB 4090s for 13b-sized models and have even used 12GB Titans for 7b models, admittedly at much slower rates.
I have a 3080Ti with 12Gb VRAM and would like to try fine tuning the same Mistral 7B model (which I found incredibly potent). Any tips on how to get started?