Google's Zanzibar actually does both: for the vast majority of queries, it uses significant levels of caching and a permitted amount of staleness [1], allowing Spanner to return a (somewhat stale) copy of the relationship data from local nodes, rather than having to wait or coordinate with the other nodes.
However, some deeply recursive or wide relations can still be slow, so Zanzibar also has a pre-computation cache called Leopard that is used for a very specific subset of these relations [2]. For SpiceDB, we called our version of this cache Materialize and it is designed expressly for handling "Enterprise" levels of scale in a similar fashion, as sometimes it is simply too slow to walk these deep graphs in real-time.
Ooh, and back when that was not a thing (iirc a few years back) me and a friend of mine had built a spiritually similar index for spicedb for our final year project at uni. We had a mini WAL and the ability to safely reject queries that specified a minimum update requirement after the index updation.
We actually have users that synchronize their resources from various sources (AWS, Kubernetes, etc) into SpiceDB, explicitly so they can perform these kinds of queries!
One of the major benefits of a centralized authorization system is allowing for permissions queries across resources and subjects from multiple different services/sources (of course, with the need to synchronize the data in)
Happy to expand on how some users do so, if you're curious.
In SpiceDB, this is known as the LookupResources [1] API, which returns all resources (of a particular type) that a particular subject (user in this case) has a particular permission on.
We have a guide on doing ACL-aware filtering and listing [2] with this API and describing other approaches for larger Enterprise scales
Disclaimer: I'm the co-founder and CTO of AuthZed, we develop SpiceDB, and I wrote our most recent implementation of LookupResources
It is actually slightly worse than even that: while New Enemy [1] is the primary concern, caching like this can also introduce a staleness issue from the other direction: let's say a user adds a new row or document, and immediately sends the link to their coworker... who tries to load that piece of data, but the (stale) access control dataset is cached and they are not in it... they get a "no access" error. While certainly fail safe (vs fail dangerous for New Enemy), it can be a fairly important UX concern as well.
Generally, the solution is to keep a timestamp of when the data changed (Zookies as you mentioned) or you can proactively reload or recompute the cache when the underlying data changes (sometimes in very smart ways), but yeah: it adds significant complications over a "simplified" approach to Zanzibar.
Disclaimer: I'm the cofounder and CTO of AuthZed and we develop the SpiceDB [2] and Materialize [3], which have quite a bit of logic around these exact problems
I'd start with reading the Zanzibar Paper. We built an annotated version [1] that provides additional guidance on some of the denser sections and how we interpreted them.
Then, I'd take a look at the history of SpiceDB [2] for how we developed the system over time.
Finally, if you have any questions, feel free to jump into our Discord [3] and ask: we're happy to answer!
> I remember building a project where we kept a mapping table of users to permissions for quick lookups, but man, it got messy with data updates.
Yep, as I mentioned above, its not an easy problem but once it is solved for you, it becomes "just" watching the events and performing the JOINs.
> especially if we could create an extension for other databases
See my video I linked above about the Postgres FDW: It does exactly this for SpiceDB and works seamlessly as-if there is a denormalized permissions table sitting in your Postgres, while still supporting the full array of complex authorization rules found in ReBAC.
Reconciling externalized authz with search is actually quite a challenging problem. For standard externalized authz, the recommendation is some form of pre-filtering or post-filtering [1], for which we actually built LookupResources (pre-filtering) and CheckBulkPermission (post-filtering) into SpiceDB.
However, as you mentioned, life is easier when the main database can handle everything, so we actually built a solution in that space called Materialize [2], which heavily denormalizes the authorization data and allows for joining within application databases such as Postgres. My colleague Evan actually put together a really cool video about using it with Gitea [3].
Recognizing that even with Materialize, however, the need to consume events can be a bit annoying, I've been doing some work to allow Postgres itself to do native JOINs against SpiceDB (and other operations). I demo it briefly in our recent announcements video [4] and I think it effectively solves this problem within Postgres, while still allowing for all the benefits (scale, performance, redundancy, distribution) of externalized authz.
We actually originally pronounced it as "kway" (the American pronunciation we had heard) but then had a saying we'd tell customers (when asked) of "pronounce it however you please, so long as you're happy using it!" :)
Hi HN, I'm happy to announce the open sourcing of the SpiceDB Playground [1], our in-browser playground for developing and testing authorization systems for SpiceDB, our open source implementation of Google Zanzibar.
In a previous HN post [2], we discussed how we moved the playground to use WASM for running SpiceDB, massively improving performance. Today, we're happy to open source the frontend side of the playground, so that everyone can see how it works and make improvements!
However, some deeply recursive or wide relations can still be slow, so Zanzibar also has a pre-computation cache called Leopard that is used for a very specific subset of these relations [2]. For SpiceDB, we called our version of this cache Materialize and it is designed expressly for handling "Enterprise" levels of scale in a similar fashion, as sometimes it is simply too slow to walk these deep graphs in real-time.
[1]: https://zanzibar.tech/24uQOiQnVi:1T:4S [2]: https://zanzibar.tech/21tieegnDR:0.H1AowI3SG:2O