In my reality I am the "sysadmin". I am also the "developer". I am the guy who wrote tests (unit and integration). I also configured the CI/CD pipeline.
I am also the guy who puts metrics in place to monitor the health of the system and if any metrics breach alarming thresholds the owner of the service (me!) is automatically paged.
How does a sysadmin know that something is broken anyway, and how does a sysadmin know who needs to get paged? If a sysadmin can make this decision - so can an algorithm.
Much of this is in Google's SRE handbook[1]. The entire notion behind the DevOps concept was to make sure that you don't segregate development and operations.
The people who write crappy code must be the people who wake up at 3am when the crappy code breaks.
>It's too late for that when you're in production.
Bugs will always slip through testing, and things will break even in production. Nobody is perfect - not even Google.
So what do you do when rollback doesn't work, you've tried everything in the playbook but the system/service is still down? Whose job is it to understand how to recover your service? Surely you don't expect the sysadmins to be doing that? They don't understand the system. How could they? They didn't build it.
I am also the guy who puts metrics in place to monitor the health of the system and if any metrics breach alarming thresholds the owner of the service (me!) is automatically paged.
How does a sysadmin know that something is broken anyway, and how does a sysadmin know who needs to get paged? If a sysadmin can make this decision - so can an algorithm.
Much of this is in Google's SRE handbook[1]. The entire notion behind the DevOps concept was to make sure that you don't segregate development and operations.
The people who write crappy code must be the people who wake up at 3am when the crappy code breaks.
>It's too late for that when you're in production.
Bugs will always slip through testing, and things will break even in production. Nobody is perfect - not even Google.
So what do you do when rollback doesn't work, you've tried everything in the playbook but the system/service is still down? Whose job is it to understand how to recover your service? Surely you don't expect the sysadmins to be doing that? They don't understand the system. How could they? They didn't build it.
[1] https://landing.google.com/sre/books/