Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fine-tuning Mistral 7B on Magic the Gathering Draft

generallyintelligent.substack.com

Post reply on HN
Print view View original

11–20 of 101 posts

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#11

Post by skerit »

rkwz wrote: post #4

> I was particularly interested in testing models’ ability to reason (i.e., perform a somewhat complex task that requires high context understanding) about out-of-distribution (i.e., unseen) data. I was under the assumption that finetuneing LLMs was useful only when you need to change the model's tone (speak like a pirate, voldemort etc). Are there other examples where LLMs were trained to reason a particular way?

Aren't a lot of base models fine-tuned with (Q)Lora on instruct-based datasets with good results? I thought this was a very common practice?

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#12

Post by selfhoster11 »

rkwz wrote: post #4

> I was particularly interested in testing models’ ability to reason (i.e., perform a somewhat complex task that requires high context understanding) about out-of-distribution (i.e., unseen) data. I was under the assumption that finetuneing LLMs was useful only when you need to change the model's tone (speak like a pirate, voldemort etc). Are there other examples where LLMs were trained to reason a particular way?

Check our Orca. IIRC, it's a technique that aims to encode additional logical capabilities into smaller models by having larger models generate step-by-step solutions to various problems. This doesn't just make them speak more like GPT-4/3.5, but is supposedly making them think more like it as well.

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#13

Post by dmakian »

rkwz wrote: post #4

> I was particularly interested in testing models’ ability to reason (i.e., perform a somewhat complex task that requires high context understanding) about out-of-distribution (i.e., unseen) data. I was under the assumption that finetuneing LLMs was useful only when you need to change the model's tone (speak like a pirate, voldemort etc). Are there other examples where LLMs were trained to reason a particular way?

> I was under the assumption that finetuneing LLMs was useful only when you need to change the model's tone (speak like a pirate, voldemort etc).

A lot of why I tried this out was to test the limits of this belief, you see a lot of talk like this out there and it sounded like nonsense to me.

Finetuning is fundamentally not much different than continued pretraining; if you feed the model high-quality and high-volume data I think it's reasonable to expect it to acquire new skills

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#14

Post by oceanplexian »

rkwz wrote: post #4

> I was particularly interested in testing models’ ability to reason (i.e., perform a somewhat complex task that requires high context understanding) about out-of-distribution (i.e., unseen) data. I was under the assumption that finetuneing LLMs was useful only when you need to change the model's tone (speak like a pirate, voldemort etc). Are there other examples where LLMs were trained to reason a particular way?

In order to speak like a pirate, it has to be able to reason :) I've done some fine tunes as well similar to the MTG example, in mine I was fine tuning it to speak JSON and reason about some input- and yes, you can indeed get these models to perform on novel tasks.

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#15

Post by freediver »

Super interesting work. Do you have thoughts how to leverage this to create a deck builder AI that would also simulate games? The major problem here is that the search space for MTG is amazingly vast.

I've seen this effort previously, pretty exciting stuff:

https://www.youtube.com/watch?v=Xq4T44EvPvo

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#16

Post by samus »

rkwz wrote: post #4

> I was particularly interested in testing models’ ability to reason (i.e., perform a somewhat complex task that requires high context understanding) about out-of-distribution (i.e., unseen) data. I was under the assumption that finetuneing LLMs was useful only when you need to change the model's tone (speak like a pirate, voldemort etc). Are there other examples where LLMs were trained to reason a particular way?

Finetuning is a useful workaround for cases when the context size is unsuitable for the task at hand. Anybody knows whether it was ever considered to finetune an LLM on the Linux kernel sources' history and its associated mailing lists?

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#19

Post by mdaniel »

In case you didn't see it, https://news.ycombinator.com/item?id=38525978 (I hacked Magic the Gathering: Arena for a 100% win rate) may interest this audience if for no other reason that the investigator discovered that Sparky, the pseudo-AI in MTGA, doesn't appear to be as stupid complicated as one may have suspected from the outside

Re: Fine-tuning Mistral 7B on Magic the Gathering Draft

#20

Post by dmakian »

matsemann wrote: post #18

How is the fine tuning actually performed? They have the data of drafts, and a prompt. But what does one do with it, more concretely?

High level it's basically: 1. Generate a lot of text examples that look like this: https://gist.githubusercontent.com/davidhershey/f57d0b19563f...

2. The model is effectively trained to predict the next token based on the previous tokens in each of these examples, which has the side effect here of teaching it to make a draft pick based on the contents of a pack.

Nothing too fancy, just next word prediction more or less

Post reply on HN