Earlier quoted context omitted.
You can't train a language model capable of helping a high school student pass an introductory physics or chemistry class if the training data was filtered to the extent that the same language model can't explain how to build a pipe bomb. If you don't want it explaining how to build pipe bombs, either the filtering has to come later or you have to accept that the model will be useless for tons of innocent things that…
> You can't train a language model capable of helping a high school student pass an introductory physics or chemistry class if the training data was filtered to the extent that the same language model can't explain how to build a pipe bomb. Why not? I'm certain that "introductory physics or chemistry class" doesn't cover pipe bomb construction. There's literally no reason the model needs to even have the vocabulary t…
I think if you go into class and ask the teacher how to build a bomb, they will refuse to answer, but not because they were never allowed to learn enough about the world to figure it out for themselves.
Just about anyone with a basic education can figure it out if they're put to the task, but that doesn't mean the parents and school administrators want it taught in the classroom. The teacher also had to learn the concept of "a device that explodes with great force and is typically used to harm other people" somewhere in order to recognize that they're not supposed to present those kinds of instructions in general.
And this is just an example. The same goes for thousands of other topics in hundreds of other fields that would be inappropriate for teachers to teach. If you shelter them all from the world to the point that they're completely ignorant of the forbidden topics, they would be ineffective as teachers. A school full of those teachers would be ineffective as a school. A society filled with such schools would be stunted as a society.