Redis is a very bad store for a distributed lock but Postgres is only slightly better.
What you truly need is something like ZooKeeper and etcd that are designed to achieve distributed consensus using algorithms like Paxos or Raft.
This ensures strong consistency and reliability in a distributed system, making them ideal for tasks like leader election, configuration management, and lease management where consistency across nodes is critical.
Paxos and Raft are consensus algorithms that provide certain guarantees and capabilities that a master-slave system with synchronous replication, such as PostgreSQL, cannot offer.
These algorithms ensure that a majority of nodes (a quorum) must agree on any proposed chAnge. This agreement guarantees that once a decision is made (e.g., to commit a transaction), it is final and consistent across all nodes. This strong consistency is critical in distributed systems to avoid split-brain scenarios.
This is easily caused by :
1-network partition
2-latency issues.
3-Async failover (2 nodes think they are the master)
4-replica lag (some but not all replica acknowledged the write) while master send confirmation to client
Redis Sentinel provides high availability and monitoring for Redis, but it does not guarantee strong consistency.
Linearizability requires that once a write is acknowledged, all subsequent reads should reflect that write.
if min-replicas-to-write is set to the number of Redis replica then if a single node goes down you won't be able to do any write (take lock or release lock).
if min-replicas-to-write is set to any number smaller than the total number or Redis replica some replica could still be lagging because of Asynchronous replication.
Also when a replica acknowledges a write in Redis, it means that the write has been received and logged by the replica, but it doesn’t necessarily mean that the write has been fully processed and applied to the data set.
This mean reading from replica that acknowledges a write from master might still return the Old value for the Key.
inspiring + slightly terrifying that rather than a single server-side implementation, every client is responsible for its own implementation
if postgres provided fast kv cache and a lock primitive it would own