261–270 of 291 posts
Earlier quoted context omitted.
...as an ambiguous and inadequately-specified specification language.
In the end, every specification is specified via natural language, this is just where the buck stops. All math books are written in natural language, even the ones about specification languages.
Earlier quoted context omitted.
> When you say "the LLM can easily hallucinate code that will satisfy the compiler but still fail the actual intent of the user", all you are saying is that the code will have bugs. My code has bugs. So does yours. You don't get to use the fancy word "hallucination" for reasonable-looking, readable code that compiles and lints but has bugs. There is an obvious and categorical difference between the "bugs" that an LLM…
They obviously are.
Earlier quoted context omitted.
Nobody has claimed to be getting deterministic outputs from LLMs.
> My prompts specify very precisely what should be implemented. I specified the public API and high-level design upfront. I let the AI come up with its own storage schema initially but then I prompted it very specifically through several improvements (e.g. "denormalize this table into this other table to eliminate a lookup"). I designed the end-to-end encryption scheme and told it in detail how to implement it. I poi…
Earlier quoted context omitted.
From the link: > You can use the -frandom-seed option to produce reproducibly identical object files. Deterministic. Also, with regard to __DATE__/__TIME__ macros, those are deterministic, because the current date and time are part of the inputs.
Determinism is predicated on what you consider to be the relevant inputs. Many compilers are not deterministic when only considering the source files or even the current time. For example, any output produced by iterating over a hash table with pointer keys is likely to depend on ASLR and thus be nondetermistic unless you consider the ASLR randomization to be one of the inputs. Any output that depends on directory it…
> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…
Idk kinda different tho.
Earlier quoted context omitted.
Determinism is predicated on what you consider to be the relevant inputs. Many compilers are not deterministic when only considering the source files or even the current time. For example, any output produced by iterating over a hash table with pointer keys is likely to depend on ASLR and thus be nondetermistic unless you consider the ASLR randomization to be one of the inputs. Any output that depends on directory it…
LLMs are 100% absolutely not deterministic even if you constrain all of their inputs. This is obviously the case, apparent from any even cursory experimentation with any LLM available today. Equivocating the determinism of a compiler given some source code as input, with the determinism of an LLM given some user prompt as input, is disingenuous to the extreme!
Earlier quoted context omitted.
> My prompts specify very precisely what should be implemented. I specified the public API and high-level design upfront. I let the AI come up with its own storage schema initially but then I prompted it very specifically through several improvements (e.g. "denormalize this table into this other table to eliminate a lookup"). I designed the end-to-end encryption scheme and told it in detail how to implement it. I poi…
I don't even think anybody expects reliably-correct code. They expect code that can be made as reliably as they themselves could make code, with some minimal amount of effort. Which clearly is the case.
Earlier quoted context omitted.
Also: " This OAuth library represents something larger than a technical milestone—it's evidence of a new creative dynamic emerging " Em-dash baby.
Can we please stop using the em-dash as a metric to “detect” LLM writing? It’s lazy and wrong. Plenty of people use em-dashes, it’s a useful punctuation mark . If humans didn’t use them, they wouldn’t be in the LLM training data. There are better clues, like the kind of vague pretentious babble bad marketers use to make their products and ideas seem more profound than they are. It’s a type of bad writing which looks…
Earlier quoted context omitted.
In the end, every specification is specified via natural language, this is just where the buck stops. All math books are written in natural language, even the ones about specification languages.
Huh? Is ABNF a "natural language"? Is the Go language spec a "natural language"?
Earlier quoted context omitted.
LLMs are 100% absolutely not deterministic even if you constrain all of their inputs. This is obviously the case, apparent from any even cursory experimentation with any LLM available today. Equivocating the determinism of a compiler given some source code as input, with the determinism of an LLM given some user prompt as input, is disingenuous to the extreme!
Most LLM software isn’t deterministic, sure. But LLMs are just doing a bunch of arithmetic. They can be 100% deterministic if you want them to be.