297 — Dank Learning: Generating Memes Using Deep Neural Networks

This work automates the generation of internet memes.

It’s hard to really explain it more succinctly than that.

The process of generating memes is, as the authors point out, not easy to automate. In particular, the evaluation of generated memes for… um… goodness..? (what’s a metric of memeliness?) is also really hard. And so as a proxy metric, the authors use the BLEU score of the generated text — a metric I’ve talked about before ([#254] [#79] [#43])… though admittedly not in this context.

The BLEU score tells us how fluent a caption is: After all, that’s one way of interpreting what a meme and its text are.

Conveniently, there’s a lot of training data available. The authors use this huge dataset to embed the captions using a 300D word2vec style process, and likewise embed the image “content” as it is interpreted by a net into a 2048D space. The images are then projected into the word embedding space with another net in order to allow the captions and their images to exist in the same space.

The net then tries to find good correlations between these sets, where high-BLEU-score sentences are rewarded and low BLEU scores are punished.

The results are… fine. Not particularly dank, but probably of higher quality than most of the internet.