IIRC you are allowed to get the Signal client from the git master branch and install it yourself, but not sure if that extends to local modifications of the client. They don't want you to distribute binaries however that are connecting to the official Signal network, even if those binaries are the official ones. You are not supposed to find Signal anywhere else than on Google play and the app store.

The server is open source technically, but it's not federated. They have also not published updates in the past for months while deploying them on the server (probably to prevent people from finding out that they were testing some feature).