101–110 of 148 posts
Earlier quoted context omitted.
Looks like they made a mistake in the article metadata - they definitely just released this article.
OK I let them know, and they've fixed it now.
Earlier quoted context omitted.
It is not unfeasible. It is absolutely realistic to do distributed finetuning of an 8B text model on previous generation hardware. You can add finetuning to your set of options for about the cost of one FTE - up to you whether that tradeoff is worth it, but in many places it is. The expertise to pull it off is expensive, but to get a mid-level AI SME capable of helping a company adopt finetuning, you are only going t…
I don't consider a small 8B model to be worth fine-tuning. Fine-tuning is worthwhile when you have a larger model with capacity to add data, perhaps one that can even grow its layers with the data. In contrast, fine-tuning a small saturated model will easily cause it to forget older information. All things considered, in relative terms, as much as I think fine-tuning would be nice, it will remain significantly more e…
> Note that in recent times, some doubt has been cast on if this technique is as powerful as believed. Additionally, there’s significant debate as to exactly what is going on during inference when Chain-of-Thought is being used... I love this new era of computing we're in where rumors, second-guessing and something akin to voodoo have entered into working with LLMs.
That's the thing, it's a novel form of computing that's increasingly moving away from computer science. It deserves to be treated as a discipline of its own, with lots of words of caution and danger stickers slapped over it.
That’s why LLMs are good at translating and spellchecking. We’ve been describing the same world and almost all texts respect grammar. That’s the first things that surface. But you can extract the same rules in other way and create a program that does it without the waste of computing power.
If we describe computing as solving problems, then it’s not computing because if your solution was not part of the training data, you won’t solve anything. If we describe computing as symbol manipulation, then it’s not doing a good job because the rules changes with every model and they are probabilistic. No way to get a reliable answer. It’s divination without the divine (no hint from an omniscient entity).
Earlier quoted context omitted.
I disagree. No amount of fine tuning will ever give the LLM the relevant context with which to answer my question. Maybe if your context is a static Wikipedia or something that will never change, you can fine tune it. But if your data and docs keep changing, how is fine tuning going to be better than RAG?
Continuous retraining and deployment maybe? But I'm actually not anti-RAG (although I think it is overrated because the retrieval problem is still handled extremely naively), I just think that fine-tuning should also be in your toolkit.
Earlier quoted context omitted.
If the LLM doesn't output data that conforms to a schema, you can't reliably parse it, so you're back to square one.
It’s significantly easier to output an integer than a JSON with a key value structure where the value is an integer and everything else is exactly as desired
This is Hamel, one of the authors of the article. We published the article with OReilly here: Part 1: https://www.oreilly.com/radar/what-we-learned-from-a-year-of... Part 2: https://www.oreilly.com/radar/what-we-learned-from-a-year-of... We were working on this webpage to collect the entire three part article in one place (the third part isn't published yet). We didn't expect anyone to notice the site! Either way, pa…
Earlier quoted context omitted.
It’s significantly easier to output an integer than a JSON with a key value structure where the value is an integer and everything else is exactly as desired
That's because you've dumbed down the problem. If it was just about outputting one integer, there would be nothing to discuss. Now add a bunch more fields, add some nesting and other constraints into it...
> Note that in recent times, some doubt has been cast on if this technique is as powerful as believed. Additionally, there’s significant debate as to exactly what is going on during inference when Chain-of-Thought is being used... I love this new era of computing we're in where rumors, second-guessing and something akin to voodoo have entered into working with LLMs.
That's the thing, it's a novel form of computing that's increasingly moving away from computer science. It deserves to be treated as a discipline of its own, with lots of words of caution and danger stickers slapped over it.
Imagine if physics literature was filled with stuff about psychology and how that would drive physicists nuts. That's how I feel right now ;)