Alas, you’ve found the reason this won’t scale. Doing customer support as the CTO is a superpower (up to a point!) for both user growth and product design. But there’s going be to come a point where we have to hand some portion of support over to a dedicated support team, and no matter how well we train those folks they’re just not going to be quite as empowered and effective. The longer I can kick that can down the road, though, the better!
(At least for the business. My sleep schedule would improve amazingly!)
You want Support Engineers who are up to speed on your code review processes and standards and given access to commit "straightforward" bug fixes. They exist :)
From there, you need to build a culture that is welcoming of "strangers" contributing code. You get these two things, you get nits and gotchas fixed directly from pain points customers are having, while product engineering is (mostly) focusing on feature dev.
Tier 0 are effectively secretaries who can file structured info around issue.
Tier 1 are dedicated support people who can follow scenarios and guide customers through the product.
Tier 2 are what you call "support engineers". They know the product, features, code, upcoming features and so on. For an in house product they are capable of making straightforward bugfixes.
Tier 3 is sometimes called "vendor support". For an in-house product this is effectively product development team.
As you can see, good supports bleeds into or blends with product development. This is how you get support answers like "this feature is planned to go live Y24Q1, but you can sign up to beta in exchange for feedback" or at least "This is not supported, but you can use features x and y to achieve similar result", instead of "Sorry, such workflow is not supported"
(At least for the business. My sleep schedule would improve amazingly!)