1–10 of 10 posts
What types of problems have you successfully solved with LLMs? What are some common pitfalls or areas where they tend to underperform?
for classification, clustering, and both text and image retrieval. It is often a drop-in replacement for other ways of doing things and most of their models are not crazy large so you can run them on an ordinary computer.
As for chatbots you should note they have superhuman recall in some sense but a limited ability to generalize or "reason". I have been asking Microsoft's Copilot for help with a maintenance programming project and I am amazed it it's ability to explain unusual but highly repetitive code fragments like the ones generated by the Babel compiler. Explaining what a program does by looking at the code is a difficult problem that LLMs cannot do reliably if they haven't seen very similar code before but there are many idioms that are used in application code that it has seen before and for those it is helpful.
Proofreading. You have to be careful though, sometimes the model just bullshits.
I am a huge fan of this kind of model https://sbert.net/ for classification, clustering, and both text and image retrieval. It is often a drop-in replacement for other ways of doing things and most of their models are not crazy large so you can run them on an ordinary computer. As for chatbots you should note they have superhuman recall in some sense but a limited ability to generalize or "reason". I have been asking…
So, reading through logs, deciphering vague error messages, navigating an overcomplicated screen for a specific thing. Especially for something like Android dev, where about 90% of the tracestack is just garbage and error messages don't say anything like the real problem.
They're very good at stochastic searches. So drafting outlines for a paper, logos, creative brainstorming.
They're bad with numbers.
They hallucinate, so you don't really want them in a situation where you can't prove whether they're correct. You can use them for medical diagnosis, but only if you double check what it gives. They use the average of what they're given, so if you're trying to code a thing, it gives you old tech stacks.
Basically you don't want it for things that you have no experience with and can't verify.
They excel at paying attention. So, reading through logs, deciphering vague error messages, navigating an overcomplicated screen for a specific thing. Especially for something like Android dev, where about 90% of the tracestack is just garbage and error messages don't say anything like the real problem. They're very good at stochastic searches. So drafting outlines for a paper, logos, creative brainstorming. They're…
They underperform at anything that requires reasoning.