131–140 of 422 posts
> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!
No, typically __you__ are mad when you fire them ...
> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!
Yes, that's why you should add "... and answer in the style of a drunkard" to every prompt. Makes it easier to not forget what you are dealing with.
Earlier quoted context omitted.
I just spent a good 2 hours trying to debug a SM6 Vulkan issue with unreal engine using an LLM, it had got me to good state but UE kept falling to load a project, it transpired that the specific error message would provide a fix as the top Google result, which I found when I eventually decided to look for myself. LLM did help a lot to get some busy work out of the way, but it's difficult to know when you need to jump…
Fwiw I think the ratio of times I needed to go to google for a solution instead of an LLM is like 20:1 for me so your mileage may vary. Depends a lot on the specific niche you're working in. Unrelated to software but recently I wanted to revive an old dumbphone I haven't used since 2014 and apparently I had it password protected and forgot the password and wanted to factory reset it. I found the exact model of the ph…
> hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. Nice.
In that framing, you can look at an agent as simply a filter on those hallucinations.
So the only real difference between "perception" and a "hallucination" is whether it is supported by physical reality.
> We should ask the LLM the question more than once For any macOS users, I highly recommend an Alfred workflow so you just press command + space then type 'llm ' and it opens tabs with the prompt in perplexity, (locally running) deepseek, chatgpt, claude and grok, or whatever other LLMs you want to add. This approach satisfies Fowler's recommendation of cross referencing LLM responses, but is also very efficient and…
Are you just opening a browser tab?
> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!
Individual sentences and paragraphs may mostly work, but it's an edifice built on sand out of poorly constructed bricks plus mortar with the wrong proportions (or entirely wrong ingredients).
LLM output is "truthy" - it looks like it might be true, and sometimes it will even be accurate (see also, "stopped clock") but depending on it is foolish because what's generating the output doesn't actually understand what it's putting out - it's just generating output that looks like the kind of thing you've requested.
Earlier quoted context omitted.
Part of what got me into software was this: no matter how complex or impressive the operation, with enough time and determination, you could trace each step and learn how a tap on a joystick lead to the specific pixels on a screen changing. There’s a beautiful invitation to learn and contribute baked into a world where each command is fully deterministic and spec-ed out. Yes, there have always been poorly documented…
"Computers are deterministic!" If I wanted to plumb together badly documented black boxes, I'd have become an EE.
Earlier quoted context omitted.
These metaphors all suck. Well, ok, yours is funny. But anyway, LLMs are just very different from any human. They are extremely shallow, even compared to a junior developer. But extremely broad, even compared to the most experienced developer. They type real fuckin fast compared to anyone on earth, but they need to be told what to do much more carefully than anyone on earth.
I've gotten Claude Code to make CUDA kernels and all kinds of advanced stuff that there's zero percent chance a junior would pull off. AI is like a super advanced senior wearing a blindfold. It knows almost everything, it's super fast, and it gets confused pretty quickly about things after you tell it.
> Certainly if we ever ask a hallucination engine for a numeric answer, we should ask it at least three times, so we get some sense of the variation. This works on people as well! Cops do this when interrogating. You tell the same story three times, sometimes backwards. It's hard to keep track of everything if you're lying or you don't recall clearly so you can get a sense of confidence. Also works on interviews, ask…