Neural networks are Turing complete, i.e. there is a universal neural network that can compute any effectively computable function¹. Incidentally, when this is combined with Rice's theorem² it means that safety research is essentially an unsolvable problem because any non-trivial property of a sufficiently complex neural network, e.g. one that can simulate a Turing machine, will have properties which can not be predicted with finite computation.

1: https://www.sciencedirect.com/science/article/pii/0893965991...

2: https://en.wikipedia.org/wiki/Rice%27s_theorem?useskin=vecto...