I was using travis-ci for a long time. One day my credit card died during their billing cycle. They emailed overdue receipts to colaborators on all my travis-ci repos. All clients, all devs, everyone. Thats breach of information, I don't want client1 to know who all are employed at client2's firm.
Ok, fine, bugs exists in software. I emailed them saying that 'Guys, you should email only repo owner when payment fails, no everyone on the team". No response. I considered that lack of support and moved to circleci.
Admittedly, when self-hosted some of these may go away, but here are a few that come to mind.
* General awfulness when it comes to packages. Outdated packages. Nonstandard packages. Ubuntu, instead of debian + testing, is a terrible choice for a CI.
* Horrible support for multilingual builds. God forbid if you have Python 3 scripts in your go project.
* With github, force-pushing during a build causes the build to error and travis can't detect that
* Lots of papercut issues in the UI
* There's always something when setting it up on a project that makes me end up fighting with it and figuring out crazy workarounds for it for several hours.
There's a lot more, though most of them have to do with point #1 and their choice of Ubuntu as a platform. I don't remember the rest.
I just went through the same process for my team, settled on CircleCI because the pricing structure fit our needs. We're still newish to CI in general so we might be wearing blinders but it's fit our needs so far.