Ask HN: Does it make sense to do research in fine tuning?

#1

Post by tokyowiz » Fri, Oct 04, 2024, 10:49 AM UTC

Hi, I'm studying machine learning and I'm interested in starting research in the field of fine-tuning. Do you think it makes sense to pursue this? Or is it a domain where it's difficult to publish papers and conduct research? I like the fact that fine-tuning is something that can be easily applied and used in companies.

Re: Ask HN: Does it make sense to do research in fine tuning?

#2

Post by malomn » Fri, Oct 04, 2024, 11:48 AM UTC

Your question is very broad. Being a PhD student, I can only say that fine-tuning happens all the time. Research in particular areas require fine-tuning pre-trained models on specific tasks, taking advantage of the amount of data seen during pre-training. It's unlikely that your thesis will be solely about fine-tuning. However, chances are you will fine-tune many models during your thesis, for various tasks depending on your field of research.

Re: Ask HN: Does it make sense to do research in fine tuning?

#3

Post by tokyowiz » Fri, Oct 04, 2024, 12:48 PM UTC

malomn wrote: post #2

Your question is very broad. Being a PhD student, I can only say that fine-tuning happens all the time. Research in particular areas require fine-tuning pre-trained models on specific tasks, taking advantage of the amount of data seen during pre-training. It's unlikely that your thesis will be solely about fine-tuning. However, chances are you will fine-tune many models during your thesis, for various tasks depending…

thanks for your reply. and sorry for my bad question. i like ai/ml, but is very huge. and i'm not a master in math... but i want to research in this field and i don't know in which domain. or how can i start, i don't have mentors and my uni is not very good, can i ask some question to you?

Re: Ask HN: Does it make sense to do research in fine tuning?

#4

Post by malomn » Fri, Oct 04, 2024, 2:48 PM UTC

tokyowiz wrote: post #3

malomn wrote: post #2

Your question is very broad. Being a PhD student, I can only say that fine-tuning happens all the time. Research in particular areas require fine-tuning pre-trained models on specific tasks, taking advantage of the amount of data seen during pre-training. It's unlikely that your thesis will be solely about fine-tuning. However, chances are you will fine-tune many models during your thesis, for various tasks depending…

thanks for your reply. and sorry for my bad question. i like ai/ml, but is very huge. and i'm not a master in math... but i want to research in this field and i don't know in which domain. or how can i start, i don't have mentors and my uni is not very good, can i ask some question to you?

Yes, of course :-) I can be reached at malo (dot) maisonneuve (at) gmail (dot) com You can also ask your questions here so that others can benefit from the answers.