Deep forgetting
Machine Unlearning Challenge
11–20 of 33 posts
Re: Machine Unlearning Challenge
#12Earlier quoted context omitted.
I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things. Certainly normal human existence involves preferential remembering and forgetting. For example not dwelling on pain or failure etc. Gradient descent obviously has none of that, it has not feelings or goals, and so there would be no preferential remembering or forgetting other than to do better ne…
> I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things. My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers. Actually, most of what we do with brains is pattern recognition, and if there isn't enough good input they will make shit up.
> My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers.
Supposedly OCD is your brain’s cause-effect loop being too potent. As in, you have a random fear I.e. “stove is on, fire will burn down house,” you go to check the stove, and the act of checking (regardless of its being on) creates the feeling that you saved your house from burning down, so now you feel compelled to check every time. Paraphrased from the Huberman podcast.
Re: Machine Unlearning Challenge
#13This is an interesting concept. Is there something like an MNIST for this, to make it more concrete?
Yes. Like all NeurIPS competition challenges, there is a starting kit which provides a dataset and metrics, there is also a grader on Kaggle.
Re: Machine Unlearning Challenge
#14Joel: "Is there any risk of brain damage?" Dr. Mierzwiak: "Well, technically speaking, the operation is brain damage, but it's on a par with a night of heavy drinking. Nothing you'll miss." - Eternal Sunshine of the Spotless Mind (2004)
Lord Mayor of Cologne: "You've damaged your brain, Universe, but no more than a week of binge drinking or five minutes on a cell phone."
- Futurama, Parasites Lost (2001)
Re: Machine Unlearning Challenge
#15They say privacy, but this sounds like a request to make censorship more censor friendly.
Edit: People seem to disagree, but is this not a tool for wiping ideas from your training data? Meaning they want to train on Wikipedia and remove concepts (per locale probably) easy as pie. This is Google outsourcing compliance with jurisdictions that require thought policing.
Re: Machine Unlearning Challenge
#16Joel: "Is there any risk of brain damage?" Dr. Mierzwiak: "Well, technically speaking, the operation is brain damage, but it's on a par with a night of heavy drinking. Nothing you'll miss." - Eternal Sunshine of the Spotless Mind (2004)
Lord Mayor of Cologne: "You've damaged your brain, Universe, but no more than a week of binge drinking or five minutes on a cell phone." - Futurama, Parasites Lost (2001)
Homer Simpson: Alright brain, you don't like me and I don't like you. But let's just do this and I can get back to killing you with beer.
- Simpsons, The Front (1992)
Re: Machine Unlearning Challenge
#17Joel: "Is there any risk of brain damage?" Dr. Mierzwiak: "Well, technically speaking, the operation is brain damage, but it's on a par with a night of heavy drinking. Nothing you'll miss." - Eternal Sunshine of the Spotless Mind (2004)
None of this is true for those who abstain from Alcohol :)
Re: Machine Unlearning Challenge
#18As with many ML-phrases, the use of 'unlearning' frames data modelling as a false dichotomy between 'learnable' and 'forgettable' data. Whereas humans are able to forget over time, it would be quite disturbing to block some memories from access entirely, how traumatic they might be (and quite apparent among people suffering memory loss).
While from a privacy perspective, not all 'data points' (i.e. memories) need be shared with the populace at large, to the individual they do constitute affective buildings blocks to deal with previous experiences. Similarly, the threshold for sharing (private) experiences depends on the communicative context one's in and how comfortable one feels sharing them.
Might similar mechanisms provide large models the ability to retain an internal recollection of 'traumatic' or 'problematic' data and become more attuned to the context its communicating in instead? Leaving blank spots or blocking out experience completely sets the stage for 'disturbing' a model's memory after the fact. While I don't want to draw a comparison between current NN architectures and organic ones, it is worth questioning our framing of currently emerging methods, as the incipient terminology can imbue how later practioners implement and think about these mechanisms.
Re: Machine Unlearning Challenge
#19As with many ML-phrases, the use of 'unlearning' frames data modelling as a false dichotomy between 'learnable' and 'forgettable' data. Whereas humans are able to forget over time, it would be quite disturbing to block some memories from access entirely, how traumatic they might be (and quite apparent among people suffering memory loss). While from a privacy perspective, not all 'data points' (i.e. memories) need be…
It's just statistics guys. You either put the data in the training set or you don't.
Re: Machine Unlearning Challenge
#20[deleted]