There was a cookie set for CSRF protection and the headers specify that content should not be cached if there is a cookie (or more precisely - the cached content includes the cookie as a cache key, so each request with a different cookie gets a cache-miss).
"Note that in 0.8.44 behaviour was changed to something considered
more natural. As of 0.8.44 nginx no longer caches responses with
Set-Cookie header and doesn't strip this header with cache turned
on (unless you instruct it to do so with proxy_hide_header and
proxy_ignore_headers). "
Developer here, since we had a click to open form at the time, we loaded the CSRF via AJAX. However that does not seem to be a good idea if we need it to work asap (and without javascript). I would look at something like SSI to put in the CSRF token to a cached page.
