Viewing profile — w4nderlust

We got very similar findings: we published a paper that show that smaller LLMs (3-7b) when finetuned with LoRA can match or outperform GPT-4 on a variety of tasks (29 out of 31) in…

Predibase ( http://predibase.com ), also referred in the article, is a platform specifically designed for exactly that. It also has "repos" for finetuning multiple models and comap…

Here is an example of the Predibase platform, referred in the article for the Solar model, but that can train also Llama-3, Phi-3 and Mistral. https://www.youtube.com/watch?v=R2JQh…

Speech is coming to Ludwig soon, we are currently working on that.

You may be right about that, but also it depends on the requirements that you have. Ludwig gives you a lot of options for those tricks, like for instance gradient clipping or regul…

Text is just one of the possible types of features supported in Ludwig. For those features, no, you can train models on any language, with a couple little caveats: you can train bo…

That experience you talked about, being able to really quickly come up with solutions starting from an already established and working configuration, was one of the main motivating…

Your configuration file looks really good! Why did you stop pursuing it? To your points: * That may be true, but I think it also depends on the level of abstraction. Fon instance i…

You have a good point about debugging. So far my approach has been to write quite long and explanatory error messages (like this https://github.com/uber/ludwig/blob/9c9e5de56dcad89…

We are considering a bunch of options re feature store. I personally like EuclidDB, but I'm not sure it fits the notion of feature store that we need (while it's great for ANN infe…

I had the chance to read a preprint, it's a great survey, it will be published soon.

I edited the page. Now there's a download button and a preview button. If you click the preview button, the javascript document viewer embedding the pdf will be loaded, otherwise i…