nice project, but the model that was being studied is really just a toy-model (both in size and training data). as such, this model can indeed be approximated by simpler models (I would suspect even n-gram LMs), but it might not be representative of how the larger LMs work.