For years I've tried to use Redis as a persistent data store. I've only been dissatisfied, having bad experiences with both sentinel and cluster. Most of my service outages were linked to Redis replication getting broken.
Then I decided to give up and use it only as an empehemral cache. I have a large number of standalone Redis instances (actually, now they are Valkey), no storage, only memory, and have Envoy proxy on top of them for monitoring and sharding. And I'm really happy with it, storing hundreds of GBs of data there, if one goes down, only a small part of the data needs to be reloaded from the primary source, and with the Envoy proxy, applications see it as a single Redis server. I was considering just replacing it with memcached, but the redis data model is more rich, so I kept using it, just not expecting anything put there to be actually stored forever.
The problematic use case was using Redis as a counter for various app related metrics, which I want to make visible to the customer later on. App was writing to Redis, background job was reading these partially aggregated metrics from Redis and writing them to another database. So you could say it was a remote buffer. On way too many occasions, after short failure of the host where the current master instance was running, the replica was not propagated to master. That was using redis sentinel. I guess my main issue was always redis sentinel.
Then I decided to give up and use it only as an empehemral cache. I have a large number of standalone Redis instances (actually, now they are Valkey), no storage, only memory, and have Envoy proxy on top of them for monitoring and sharding. And I'm really happy with it, storing hundreds of GBs of data there, if one goes down, only a small part of the data needs to be reloaded from the primary source, and with the Envoy proxy, applications see it as a single Redis server. I was considering just replacing it with memcached, but the redis data model is more rich, so I kept using it, just not expecting anything put there to be actually stored forever.