posts

This will be a two-part blog series, where the first part reproduces the experiments of the paper “Discovering neural nets with low Kolmogorov complexity” and the second part expands on the results of the paper.

In this part, we spotlight the paper and give an introduction to algorithmic complexity theory. The code used to reproduce the paper is available on GitHub. This blog post was produced in collaboration with Simon Brugman, whose blog can be found here. Both blogs contain the same version of this post.

In the previous post I laid out the basic idea behind independent component analysis through predictability minimization. In this post, I will supply a basic PyTorch implementation of the idea. The main goal is to get a taste for how adversarial gradient descent works (or often, does not work). In a follow-up post, I will go into the convergence properties of these kinds of adversarial optimization systems.

This will be a small blog post explaining the setup of Jürgen Schmidhubers 1992 paper on ‘Predictability Minimization’. The full title of the paper is ‘Learning Factorial Codes by Predictability Minimization’. Recently, there has been a lot of literature focusing on ‘disentangled representations’ in variational autoencoders and other neural network architectures. In a follow-up post, I will give a simple Python implementation of the paper’s central idea.