161–170 of 378 posts
Earlier quoted context omitted.
Fundamentally there's no way to deterministically guarantee anything about the output.
That is "fundamentally" not true, you can use a preset seed and temperature and get a deterministic output.
Earlier quoted context omitted.
Actually at a hardware level floating point operations are not associative. So even with temperature of 0 you’re not mathematically guaranteed the same response. Hence, not deterministic.
You are right that as commonly implemented, the evaluation of an LLM may be non deterministic even when explicit randomization is eliminated, due to various race conditions in a concurrent evaluation. However, if you evaluate carefully the LLM core function, i.e. in a fixed order, you will obtain perfectly deterministic results (except on some consumer GPUs, where, due to memory overclocking, memory errors are freque…
In chats that run long enough on ChatGPT, you'll see it begin to confuse prompts and responses, and eventually even confuse both for its system prompt . I suspect this sort of problem exists widely in AI.
Gemini seems to be an expert in mistaking its own terrible suggestions as written by you, if you keep going instead of pruning the context
Earlier quoted context omitted.
I have been saying this for a while, the issue is there's no good way to do LLM structured queries yet. There was an attempt to make a separate system prompt buffer, but it didn't work out and people want longer general contexts but I suspect we will end up back at something like this soon.
Fundamentally there's no way to deterministically guarantee anything about the output.
Earlier quoted context omitted.
I'll grant that you can guarantee the length of the output and, being a computer program, it's possible (though not always in practice) to rerun and get the same result each time, but that's not guaranteeing anything about said output.
You can guarantee what you have test coverage for :)
Earlier quoted context omitted.
The principal security problem of LLMs is that there is no architectural boundary between data and control paths. But this combination of data and control into a single, flexible data stream is also the defining strength of a LLM, so it can’t be taken away without also taking away the benefits.
It’s easier not to have that separation, just like it was easier not to separate them before LLMs. This is architectural stuff that just hasn’t been figured out yet.
With databases there exists a clear boundary, the query planner, which accepts well defined input: the SQL-grammar that separates data (fields, literals) from control (keywords).
There is no such boundary within an LLM.
There might even be, since LLMs seem to form adhoc-programs, but we have no way of proving or seeing it.
Earlier quoted context omitted.
You still can’t deterministically guarantee anything about the output based on the input, other than repeatability for the exact same input.
What does deterministic mean to you?
> This class of bug seems to be in the harness, not in the model itself. It’s somehow labelling internal reasoning messages as coming from the user, which is why the model is so confident that “No, you said that.” Are we sure about this? Accidentally mis-routing a message is one thing, but those messages also distinctly "sound" like user messages, and not something you'd read in a reasoning trace. I'd like to know if…
These tokens are almost universally used as stop tokens which causes generation to stop and return control to the user.
If you didn't do this, the model would happily continue generating user + assistant pairs w/o any human input.
Earlier quoted context omitted.
The principal security problem of LLMs is that there is no architectural boundary between data and control paths. But this combination of data and control into a single, flexible data stream is also the defining strength of a LLM, so it can’t be taken away without also taking away the benefits.
Exactly like human input to output.