Yes having bitcoins is just having a private key. In 12 or 24 words format a wallet can even be memorized if needed. The blockchain is the global ledger of how much money there is on corresponding public keys.