They could provide access to the training code. It's useful for training smaller models or distilling larger ones. They don't need to release every details involved in tuning the optimization parameters during the pre-training stage.