> Not long ago, for example, while sitting with me in a cafe, my 3-year-old daughter spontaneously realized that she could climb out of her chair in a new way: backward, by sliding through the gap between the back and the seat of the chair. My daughter had never seen anyone else disembark in quite this way; she invented it on her own — and without the benefit of trial and error, or the need for terabytes of labeled d…
Instead of an anti fragile model that can continuously be run, current models need to be scrapped at error states. Hyperparameter tuning is just random guessing. Getting good data also doesn't work because of the high dimensionality of it for non-trivial tasks.
The idea that ctrl+z is good should be re-examined. Perfect memory like block chains have isn't the answer either though. Perhaps something similar to the non forgetting yet imperfect human mind.
Most data is garbage and even more eventually becomes garbage. (unless you exist in a finite defined world like Go) Is there any sort of neural net that that find or creates "core" memories with weaker supplementary memories?
The quintessential experiences would only be dislodged with an influx of contradictory data. Initial cores could be initialized via mother-child like training. The training data would be tiered and weighted. There would be an internal system that passed judgement on new ingestion sources. New data would be a necessity. Old data passed in as new would be like a monotonous life digging in cores preventing them from having meaningful change. Almost all data would be labelled as garbage initially unless vouched for somehow. Pure good data would be bad as well because there isn't enough quality differentiation to see what is core and what isn't.