I also just found something that sounds genuinely realistic: Piper (https://github.com/OHF-Voice/piper1-gpl/tree/main). It's slow but apparently you can run it as a daemon to be faster, and it integrates with Home Assistant and Speech Dispatcher.
$ sudo apt update
$ sudo apt install -y python3 python3-pip libsndfile1 ffmpeg
$ python -m venv piper-tts
$ ./venv/piper-tts/bin/pip install piper-tts
$ ./venv/piper-tts/bin/python3 -m piper.download_voices en_US-lessac-medium
$ ./venv/piper-tts/bin/piper -m en_US-lessac-medium -- 'This will play on your speakers.'
To manage the install graphically, you can use Pied (https://pied.mikeasoft.com/), which has a snap and a flatpak. That one's really cool because you can choose the voice graphically which makes it easy to try them out or switch voices. To play sound you just use "spd-say 'Hello, world!'"
It has some mispronounciations in the texts I tried to listen to, besides using so much RAM that it kills basically all other apps to make space for this. Not really worth it when espeak is already understandable
Festival I've tried in the meantime and doesn't support Dutch or German, two of the three languages I use regularly. I keep coming back to espeak at the only option that will simply always work xD
More crazy: Home Assistant did a "Year of Voice" project (https://www.home-assistant.io/blog/2022/12/20/year-of-voice/) that culminated in a real open-source voice assistant product (https://www.home-assistant.io/voice-pe/) !!! And it's only $60??