From what I remember, larger LLMs like PaLM don't use biases for training stability, but smaller ones tend to still use them.