The target audience of this book isn't the layman, it's people studying mathematics. So a phrase like "relatively little mathematical background" is meant in the context of academia.
I don't think there's a basic and gentle approach to learning category theory that doesn't remove the rigor. You can learn at a high level what some of the stuff in category theory and get a basic feel and intuition for it, if that's what you want. But, if you want to learn category with the rigor, you're going to have to first learn how to write a proof in mathematics. Really though, you're going to want to have a decent grasp on some basic abstract algebra. Otherwise you're never going to be able to understand the examples for category theory.
I don't think there's a basic and gentle approach to learning category theory that doesn't remove the rigor. You can learn at a high level what some of the stuff in category theory and get a basic feel and intuition for it, if that's what you want. But, if you want to learn category with the rigor, you're going to have to first learn how to write a proof in mathematics. Really though, you're going to want to have a decent grasp on some basic abstract algebra. Otherwise you're never going to be able to understand the examples for category theory.