As I understand it: this treats image generation as constraint satisfaction. The constraints are that each NxN patch appears in the source image. The satisfaction method is arc consistency https://en.wikipedia.org/wiki/AC-3_algorithm, except, when that settles down prematurely, pick the least-constrained patch and make a random valid choice, then continue. (If this leads to getting stuck, then give up instead of backtracking.)
Is that the idea? The description wasn't completely clear to me.
Yes, (C1) is a constraint problem. But we also want to satisfy (C2) as close as possible, otherwise we could have just colored some outputs in a single color.
Yes, I took that to mean, when you're making a random valid choice, the weights come from the input distribution.
I don't understand about "a single color" unless you mean setting all output pixels to the same color, which would only satisfy the constraints if the input has an NxN patch all one color.
I hope my comment didn't seem to imply that the work seemed unoriginal. I don't think that; I wanted to check my reading.
Thank you for writing this (and your last comment) out explicitly. The quantum mechanics description confused me (and possibly other from reading the thread here). Although I can understand how that inspired this work in the first place.
The algorithm used is exactly what you described here. It wasn't obvious to me that probability density functions were not tracked (the algorithm only tracks which NxN patch are allowed at each location) and randomness only come into play when a random valid choice is made, and there each valid patch is chosen probability proportional to its number of occurrence in the input.
Most of the examples in the repo have those NxN all one color patches. Or, without (C2) the algorithm would have generated completely empty integrated circuits, or completely grass terrain, which is really boring.
You understood right, it's constraints + probabilities.
I might try this approach to generate formal poetry -- it's something I've done by backtracking before, and I'd considered doing something like your ConvChain.
At first I thought that my methods don't offer anything new to text generation besides the Markov chain, but several people already proposed ideas that sound sensible, so let me know if you make anything!
Is that the idea? The description wasn't completely clear to me.