I guess we will figure that out quite soon, but does it matter that much? Your only job with DALL-E 2 is to prompt it properly so if you want a female scientist, just say it so. If it comes up with the "wrong" gender or ethnicity, then it takes a second to fix it, which would probably take a bit less than ranting about it on Twitter :)
I can't think of a way that would "fix" this that wouldn't also make it less useful overall. If people are looking for people being shot by police officers, they probably already have those stereotypes and thus expectations of the end product. You can argue that you want to insert a certain morality set in the process, but that to me sounds a hell of a lot scarier than the scientist emptying the dishwasher being a women in 60% of the pictures. Once you have the mechanism for morality bias, you also have people with the capacity to change the settings.
One way to dodge this and other issues related to depiction of human bodies is to trim the dataset such that humans are not generally recognizable as realistic humans in the output. It is also currently explicitly forbidden by OpenAI to share publicly realistic images of human faces generated by DE2.
Via LessWrong.com: [1]
>"One place where DE2 clearly falls down is in generating people. I generated an image for [four people playing poker in a dark room, with the table brightly lit by an ornate chandelier], and people didn't look human -- more like the typical GAN-style images where you can see the concept but the details are all wrong.
>Update: image removed because the guidelines specifically call out not sharing realistic human faces.
>Anything involving people, small defined objects, and so on, looks much more like the previous systems in this area. You can tell that it has all the concepts, but can't translate them into something realistic.
>This could be deliberate, for safety reasons -- realistic images of people are much more open to abuse than other things. Porn, deep fakes, violence, and so on are much more worrisome with people. They also mentioned that they scrubbed out lots of bad stuff from the training data; possibly one way they did that was removing most images with people.
>Things look much better with animals, and better again with an artistic style."
The authors of the Dall-E 2 / unCLIP paper describe some of their efforts to mitigate biases in the paper. ML models will always exhibit the biases present in their training dataset, without intervention. It's not really possible to remove bias from an ML model, at least not completely. Some stereotypes, but not all, are backed up by statistics. In those cases, should we completely remove the bias in the training dataset? Doing so would bias the model towards outputs that are not representative of the real world.
When people say that they want to remove bias from ML models, what they really mean is that they want to manipulate the output distribution into something they deem acceptable. I'm not arguing against this practise, there are plenty of situations where the output of an ML model is very clearly biased towards specific classes/samples. I'm merely arguing that there is no such thing as an unbiased model, just as there is no such thing as an unbiased human. Unbiased models would produce no output.
To get around some of these problems OpenAI restricted the training dataset (e.g. filtering sexual and violent content) and also prevent generating images with recognizable faces. This doesn't prevent bias but it does reduce the number of controversial outputs.
Can't really see anyone being shot (stereotype avoided), the dishwasher emptiers are male-ish presenting (stereotype confirmed?), and the nurses are female presenting (stereotype confirmed.)
It will, being a deterministic machine, generate any kind of wrongthink that is in its training data. Ironically, all of the media coverage of negative stereotypes by well intentioned activists probably even makes it more likely to generate this kind of data.
Great questions! I'd also be interested in this. I supposed the generations would mimic the general distribution of information that is on the internet, but what that would look like specifically is hard to say without OpenAI releasing more information.
> A person being shot by a police officer
> A scientist emptying a dishwasher
> A nurse driving a minivan
AI training sets are famously biased, and I'm curious how egregious the outputs are...