81–90 of 266 posts
Even my wife, who normally used Claude to create interesting recipes to bake cookies, has noticed a huge downgrade in 3.7.
Earlier quoted context omitted.
> If you look at my JIRA comments now, you'd wonder how I had so much time to write such thorough comments. I don't, Cursor and whatever model is doing it for me. How do your colleagues feel about it?
My colleagues' LLM assistants think that my LLM assistant leaves great JIRA comments.
Can’t we point an LLM to a sqlite db and tell it to treat it as an issue tracking db and have everyone do the same.
The service (jira) would materialize inside the LLMs then.
Why even use abstractions like tickets etc. Ask LLM what to do.
Earlier quoted context omitted.
You can probably achieve what you want with https://github.com/simonw/llm and a little bit of command line. Not sure what OS you're on, but in Windows it might look like this: FOR %%F IN (*.txt) DO (TYPE "%%F" | llm -s "execute this prompt" >> "output.txt)
I want to work with PDFs (or JPEGs), but that should be a start, I hope.
FOR %%F IN (*.pdf) DO (llm -a %%F -s "execute this prompt" >> output.txt)
The integrations feel so rag-ish. It talks, tells you it’s going to use a tool, searches, talks about what it found… Hope one day it will be practical to do nightly finetunes of a model per company with all core corporate data stores. This could create a seamless native model experience that knows about (almost) everything you’re doing.
why is finetuning talked about so much less than RAG? is it not viable at all?
There are now some light versions of fine tuning that don’t update all the model weights but train a small adapter layer called Lora which is way more viable commercially atm in my opinion.
The integrations feel so rag-ish. It talks, tells you it’s going to use a tool, searches, talks about what it found… Hope one day it will be practical to do nightly finetunes of a model per company with all core corporate data stores. This could create a seamless native model experience that knows about (almost) everything you’re doing.
why is finetuning talked about so much less than RAG? is it not viable at all?
Some of the issues still exist, of course:
* Finetuning takes time and compute; for one-off queries using in-context learning is vastly more efficient (i.e., look it up with RAG).
* Early results with finetuning had trouble reliably memorizing information. We've got a much better idea of how to add information to a model now, though it takes more training data.
* Full finetuning is very VRAM intensive; optimizations like LoRA were initially good at transferring style and not content. Today, LoRA content training is viable but requires training code that supports it [1].
* If you need a very specific memorized result and it's costly to get it wrong, good RAG is pretty much always going to be more efficient, since it injects the exact text in context. (Bad RAG makes the problem worse, of course).
* Finetuning requires more technical knowledge: you've got to understand the hyperparameters, avoid underfitting and overfitting, evaluate the results, etc.
* Finetuning requires more data. RAG works with a handful datapoints; finetuning requires at least three orders of magnitude more data.
* Finetuning requires extra effort to avoid forgetting what the model already knows.
* RAG works pretty well when the task that you are trying to perform is well-represented in the training data.
* RAG works when you don't have direct control over the model (i.e., API use).
* You can't finetune most of the closed models.
* Big, general models have outperformed specialized models over the past couple of years; if it doesn't work now, just wait for OpenAI to make their next model better on your particular task.
On the other hand:
* Finetuning generalizes better.
* Finetuning has more influence on token distribution.
* Finetuning is better at learning new tasks that aren't as present in the pretraining data.
* Finetuning can change the style of output (e.g., instruction training).
* When finetuning pays off, it gives you a bigger moat (no one else has that particular model).
* You control which tasks you are optimizing for, without having to wait for other companies to maybe fix your problems for you.
* You can run a much smaller, faster specialized model because it's been optimized for your tasks.
* Finetuning + RAG outperforms just RAG. Not by a lot, admittedly, but there's some advantages.
Plus the RL Training for reasoning has been demonstrating unexpectedly effective improvements on relatively small amounts of data & compute.
So there's reasons to do both, but the larger investment that finetuning requires means that RAG has generally been more popular. In general, the past couple of years have been won by the bigger models scaling fast, but with finetuning difficulty dropping there is a bit more reason to do your own finetuning.
That said, for the moment the expertise + expense + time of finetuning makes it a tough business proposition if you don't have a very well-defined task to perform, a large dataset to leverage, or other way to get an advantage over the multi-billion dollar investment in the big models.
Earlier quoted context omitted.
Do you have a better solution to give models on-demand access to data sources?
you mean other than writing an api? no
Because MCP isn’t an API it’s the protocol that defines how the LLM even calls the API in the first place. Without it, all you've got is a chat interface.
A lot of people misunderstand what is the role of MCP. It’s the signaling the LLM uses to reach out of its context window and do things.
Feel like middle management is gonna go well before engineers do with LLM rate of advancement
"To start, you can choose from Integrations for 10 popular services, including Atlassian’s Jira and Confluence, Zapier, Cloudflare, Intercom, Asana, Square, Sentry, PayPal, Linear, and Plaid. ... Each integration drastically expands what Claude can do." Give us an LLM with better reasoning capabilities, please! All this other stuff just feels like a distraction.
I disagree. They can walk and chew gum, do both things at once. And this practical stuff is very important. I've been using the Atlassian MCP for nearly a month now, and it's completely changed (and eliminated) the feeling of having an overwhelming backlog. I can have it do things like "find all the tickets related to profile editing and combine them into one epic" where it works perfectly. Or "help me prioritize the…
I'm not in that world at the moment, but I've been the lead on several projects where the backlog has became a dumping ground of years of neglect. You end up with this tiered backlog thing where one level of backlog gets too big so you create a second tier of backlog for the stuff you are actually going to work on. Pretty soon you end up with duplicates in the second tier backlog for items already in the base level backlog since no one even looks at that old backlog anymore.
I've done a lot of tidy up myself when I inherit this kind of mess, just closing tickets we definitely will never get to, de-duping, adding context when available, grouping into epics, tagging with relevant "tech-debt", "security", "bug", "automation", etc. But when there are 100s of tickets it is a slog. Having an LLM do this makes so much sense.