Apologies i realize this is a bit of a tangent but what i would like and can't find is an understandable description of how Generative neural networks work. My very basic understanding of neural networks is as a classifier. Cat and dog pictures in, likelihood out. I can't get any intuition about how this can generate new cat pictures.. is it a search problem? How does that search work?
Classifiers are just one example of output, one where the final layer of output is interpreted to be probabilities of different items. Using a different loss function (or reward function), you can train for the output to be different things.
But in fact text generators work basically identically to classifiers. You train the model to classify texts according to which single word comes next. Then you append a word to the text according to that output, and repeat.
