> I'm not aware of a way to do e2ee voice chat with many participants without linearly increasing bandwidth requirements for users
This can be almost trivially extended from one-to-one end-to-end encrypted text chats: One peer picks a random symmetric key and forwards it to all participants over the existing encrypted and authenticated channel.
Afterwards, you can just use the most appropriate way of relaying encrypted voice data: Full mesh peer to peer, one party acting as relay for everybody else, using an SFU that relays incoming data from one peer to all others (without being able to decrypt it)...
This can be almost trivially extended from one-to-one end-to-end encrypted text chats: One peer picks a random symmetric key and forwards it to all participants over the existing encrypted and authenticated channel.
Afterwards, you can just use the most appropriate way of relaying encrypted voice data: Full mesh peer to peer, one party acting as relay for everybody else, using an SFU that relays incoming data from one peer to all others (without being able to decrypt it)...