Human Judgment as a Specification
blog.brownplt.org
Human Judgment as a Specification
1–10 of 24 posts
Re: Human Judgment as a Specification
#2> This is also why PICK can usefully fail. Sometimes none of the model’s candidates is right, and PICK ends with zero survivors. Under the spec-elucidation reading, that outcome means: the commitments you made through classification could not be satisfied by anything the model produced. Better to know than to ship the regex anyway.
Zooming out (but only a little) from the impetus to formalize a commitment to a particular class of result candidate (what the author here is calling "spec elucidation"), we can also imagine this same evolution of concerns being applied in order to cause what we currently term "AI safety" into something more like "AI ethics".
For example, if we can elucidate the specifications for things like peace and justice to ensure that the class of results is formally verified as non-participation in war (or perhaps, further in the future, non-participation in state activities whatsoever), we may be able to throw cold water on all the vitriolic arguments about model capabilities and which need to be banned or delayed lest we accelerate the apocalypse (or whatever is actually on the mind of the ban-this-model constituency).
I like how the author ends tersely with:
> If you have a formal language with the closure properties above — we suspect you would be surprised how many do — we would very much like to hear from you.
That's certainly not me, but I bet it's true that it's somebody.
Re: Human Judgment as a Specification
#3> This is also why PICK can usefully fail. Sometimes none of the model’s candidates is right, and PICK ends with zero survivors. Under the spec-elucidation reading, that outcome means: the commitments you made through classification could not be satisfied by anything the model produced. Better to know than to ship the regex anyway. Zooming out (but only a little) from the impetus to formalize a commitment to a partic…
> ensure that the class of results is formally verified as non-participation in war
There are very few things that cannot be stated as dual use, with one totally benign and one totally screwed up. It's like wanting a hammer to distinguish if it's striking a nail for a roof vs. a nail for an illegal animal pen. That's the wrong application of constraints. The hammer shouldn't care.
Re: Human Judgment as a Specification
#4this is the type of thing you need to build a foundation sturdy enough to let you operate higher up the stack and ratchet to design-by-metaphor and then design-by-philosophy. those design skills are taught in humanities departments, not engineering departments, so this is a weird feeling place for those of us that wandered over from a technical field.
Re: Human Judgment as a Specification
#5> Telling people “you must read all the code generated by an LLM” is definitely meaningful—but it is not at all moderate (so most people won’t do it)
But they should! The code is the best source of truth on what the software is doing after all.
Instead of giving up on that, we should make it easier to read generated code, e.g. by generating less code in a higher level language.
On the flip side, forcing myself to read all the code also resulted in a smaller, higher quality code base.
Re: Human Judgment as a Specification
#6> This is also why PICK can usefully fail. Sometimes none of the model’s candidates is right, and PICK ends with zero survivors. Under the spec-elucidation reading, that outcome means: the commitments you made through classification could not be satisfied by anything the model produced. Better to know than to ship the regex anyway. Zooming out (but only a little) from the impetus to formalize a commitment to a partic…
> ensure that the class of results is formally verified as non-participation in war There are very few things that cannot be stated as dual use, with one totally benign and one totally screwed up. It's like wanting a hammer to distinguish if it's striking a nail for a roof vs. a nail for an illegal animal pen. That's the wrong application of constraints. The hammer shouldn't care.
The author addresses this point as well:
> This is also why we do not believe PICK becomes less useful as models improve. Better models do not make user intent more articulate — asked for “a regex matching countries of North America”, a more capable model still cannot tell you whether you want the Caribbean included, or where you want to stop heading south. Better models produce better candidates, faster — which shifts user effort precisely toward the work PICK is built to support.
Re: Human Judgment as a Specification
#7> Telling people “you must read all the code generated by an LLM” is definitely meaningful—but it is not at all moderate (so most people won’t do it).
I am honestly heartbroken to live in a world where reading the code is seen as an unreasonable ask by either students or by professional working programmers.
Re: Human Judgment as a Specification
#8> Telling people “you must read all the code generated by an LLM” is definitely meaningful—but it is not at all moderate (so most people won’t do it). I am honestly heartbroken to live in a world where reading the code is seen as an unreasonable ask by either students or by professional working programmers.
Don't tell me you're reading all assembly generated by your local golang or javac compiler? And that you've read every line of code down the dependency tree for your node_modules?
I'm just upset that we are throwing away the original prompts for generated code in such a cavalier fashion.
Re: Human Judgment as a Specification
#9> Telling people “you must read all the code generated by an LLM” is definitely meaningful—but it is not at all moderate (so most people won’t do it). I am honestly heartbroken to live in a world where reading the code is seen as an unreasonable ask by either students or by professional working programmers.
[dead]
Re: Human Judgment as a Specification
#10> Telling people “you must read all the code generated by an LLM” is definitely meaningful—but it is not at all moderate (so most people won’t do it). I am honestly heartbroken to live in a world where reading the code is seen as an unreasonable ask by either students or by professional working programmers.
Don't tell me you're reading all assembly generated by your local golang or javac compiler? And that you've read every line of code down the dependency tree for your node_modules? I'm just upset that we are throwing away the original prompts for generated code in such a cavalier fashion.
`npm install` is the OG vibecoding