T0* – Series of encoder-decoder models trained on a large set of different tasks

It somehow picked up Modesty.

(author here) That's an interesting take (which I agree with).

Providing a quick way to stress test the model is definitely a double edge sword. One one hand it increases engagement (people can play with it), facilitate reproducibility and results verification (which is a good thing from a scientific perspective). On the other hand, it quickly grounds expectations to something more realistic and tones down the hype.

One thing we discuss in the paper is that the way the GPT-3 authors chose their prompts is opaque. Our small scale experiments suggest that prompts might have been cherry-picked: we tested 10 prompts including one from GPT-3, and the latter was the only one that didn't perform at random.

Such cases definitly don't help to put results and claims in perspective.

Given that you can effectively identify and reformulate biased content, the most low effort method being the use of multiple updated prompts, I count it a feature that the model contains a sub model of racist perspectives. If I were to ask you to compose a horribly offensive racist sentence, I am all but certain you could construct something that would be utterly shocking. You yourself have a model of biased, sexist, racist perspectives, and part of being a good human is recognizing and using that as proxy for what not to think or do or say.

If you're at all self aware, you can compare your thoughts and say "oh, that sounds like something a racist might say, let's reconsider whatever knowledge that led me to think that way. " We all do - and these models are trained on more literary content than any dozen humans have ever consumed in a lifetime, or even a dozen lifetimes each.

Removing the cruft, the chaos, and noise might be valuable, but if you want a generally capable model that can parse a huge spectrum of the human experience, that means taking the bad with the good. It's far more likely than not that the current state of the pile is not ideally curated, even with the intent of keeping the "bad"stuff, but i hope that becomes a consideration for them as they develop it further. There's a nietszchean abyss aspect to these giant models - you don't want them trained significantly on the horrible and evil, but enough of it to foster a nuanced and deep understanding of human nature can't help but be valuable. A naive, ignorant, childlike model is less valuable than one aware of the darker nature of humans at their worst.

Fine tuning on a 4chan dump might be highly effective for identifying trolling. Scraping neonazi and extremist and cult sites and forums could also allow these models to easily identify the content and contexts of such worldviews.

Prompt engineering is in its infancy, but will allow developers to engineer semantically in a way that's explicable through each prompt down to the level of the model.

You could encounter a Twitter user attempting to post an argument, and they encounter a popup that says sobering like: "you're not allowed to post this. Unfortunately, it makes a straw man argument that leads to the impression that it could be reasonable to be biased against a person based on their race. Here is the exact reasoning: because blah, blah, blah. "

If a user challenges it, kick it to a human, and they can recommend improving the prompts by pointing out where it went wrong, or validate the preemptive moderation.

Nothing like that would be possible without training on biased and racist content. That does mean the tool must be used responsibly, in full awareness that you need to be vigilant against bias, intentional or not. I'd rather it be like that than a naive, faux innocent model, unaware and less capable.

>would have been racist if there was anything causing the association other than bad modeling

Yes, I know how machine learning works.

In the future if "sets stochastic associations" end up putting a few more of one kind of people behind bars, or automated security sentries misidentify a certain group of people's behavior as threatening more often than others, do we just say "oh, it's not actually a racist robot that subdued your teenager because he was X, you see it's just a result of poor modeling and a set of stochastic processes ended up determining that he was displaying aggression"?

The machine is not intending to do anything, least of all "be" racist, but it is a racist invention if a product makes it into the wild labeling Black people gorillas.

*Edit because this AI answered it pretty well itself:

> Can AI be racist?

> if it is programmed to discriminate

I'm struggling to figure out a plausible scenario in which a model like this would be rendering judgment on employment suitability based on gender. Such a scenario would have to be deliberately created - models like these give you the answers to questions you ask. If you choose questions that lead to gender bias, it's not a problem with the model.

The whole scenario is contrived and not relevant to the functionality of these language models. It's like complaining that your Formula 1 car doesn't have a snowplow mount. Even if you add one, that's not how you should be using the tool.

The models use human generated text. They model human biases, like preferences for well being, humor, racism, sexism, and intelligence or ignorance. The ability to generate biased output is also the ability to recognize bias. It's up to the prompt engineer to develop a methodology that selects against bias.

You can use prompts to review the output - is this answer biased? Sexist? Racist? Hurtful? Shallow? Create a set of 100 questions that methodically seek potential bias and negative affect, and you could well arrive at output that is more rigorously fair and explained than most humans could accomplish in the casual execution of whatever task you're automating.

Zero-shot inference is a starting point - much the same way people shouldn't blurt out whatever first leaps to mind, meaningful output will require multiple passes.