61–70 of 101 posts
> I ended up renting an hourly GPU from Runpod (an RTX 4090 w/ 24GB of VRAM) for ~$0.7/hr. Sorry if I missed this, but how much did it cost total to do the fine-tune? Is that the 40 hour number (~$27)? Also, very cool writeup. Thanks for sharing!
I think if you add up all of the learning and testing I did, probably closer to ~$50 total
I wonder if you could use a smaller model or get better results if you treated each card as a token, gave the state of the draft as an input and the predicted token would be the card to pick. You woukd have to train from scratch with a custom tokenizer.
I tried adding special tokens for a reddit-style dataset once. The format was: ` username title here...` The resulting model was so much worse than just formatting everything plaintext. This was with MPT-30B, 15 special tokens, 300M training tokens, and a full finetune. I may have made a mistake, but I haven't seen any open source finetunes successfully add a large number of tokens yet either.
a simple input might be 1 14 56 5 64 2 -> predicted token is the draft pick.
Then train a transformer based network from scratch.
Earlier quoted context omitted.
I tried adding special tokens for a reddit-style dataset once. The format was: ` username title here...` The resulting model was so much worse than just formatting everything plaintext. This was with MPT-30B, 15 special tokens, 300M training tokens, and a full finetune. I may have made a mistake, but I haven't seen any open source finetunes successfully add a large number of tokens yet either.
Try doing the same thing in your dataset, but don't actually add them as "special tokens", and just let them just be multiple tokens. Adding new tokens needs a ton of data to train what the token means. Reusing existing tokens, will allow you to easily teach that a sequence of tokens now has a new meaning after fine tuning.
> Adding new tokens needs a ton of data to train what the token means.
But how much? 300M tokens is fine for a simple version of ChatML with ~4 tokens. Not for 15, at least in my case. How's this relationship scale?
Just trying to offer one datapoint for what doesn't work, with the hedge that I might have just had a bug
Earlier quoted context omitted.
Can you please point me to good resources on fine tuning? Thanks.
Search for articles showing you code for fine-tuning Llama 2, ideally including a colab notebook that you can run and modify yourself so that you have real code to work with. You can try to modify their working example to suit your own toy project as a first step.
I like that this shows how hard even conceptually simple ideas are to achieve in fine-tuning LLMs. Even given a pretty good starting dataset, a decent starting model, etc. this appears to have been a challenge. One thing it did make me think about was that these models are suitable for things that don't have a natural definitive answer. That is, picking the perfect card given a set of picks is probably combinatoriall…
That lines up with my experience- for high-stakes decisions, they rarely give me a great answer. But for low stakes decisions, they do well at giving me a good enough answer. For example, I've been using them to help find gifts for friends and children this month. I don't need the best choice to solve the problem, just a good one.
I wonder if you could use a smaller model or get better results if you treated each card as a token, gave the state of the draft as an input and the predicted token would be the card to pick. You woukd have to train from scratch with a custom tokenizer.
Earlier quoted context omitted.
Sparky is the Arena AI, but no one ever accused it of being a good Arena AI - it is very much only there for the new player experience of playing against a dumb computer when you're first exposed to the game and don't know the rules, or for the computer equivalent of "playing against a goldfish" a deck you made to see how it draws or combos. It's not a Chess CPU.
I hope I also did not accuse it of being good, but the observation I was trying to make is that -- according to the article, I have not myself confirmed the claim -- they run the card evaluation logic and gameplanning locally, not in a data center full of H100s, which I consider to be quite a feat given the free-text-y self-modifying rules of M:TG