The diagrams are indeed hypergraphs. They are taken from Chapter 12 of the book, and are simplifications of Figures 12.44 and 12.45. The solid lines enclose relation schemes, and the dotted line represent "objects", which here signify groups of relations it makes sense to join. There's a scan of the text at http://web.cecs.pdx.edu/~maier/TheoryBook/TRD.html if you want to look at the figure originals.
Yup, I'm the author. As I recall, I didn't have much input on the cover. Whoever did the design must have liked the look of the figures. However, as you see, they're pretty hard to interpret with the attribute names removed.
I'm going to guess that the pictures are of hypergraphs. Hypergraphs are good representations of mathematical relations. Each vertex of the hypergraph is a domain, while a hyperedge would be a "relation", the subject of the book.
For those unfamiliar with hypergraphs, they are very similar to classical graphs. Except hyperedges may connect more than two vertices together. (While normal graph edges normally only connect two vertices together) Therefore, you have to draw regions to represent hyperedges.
Does anyone know what the hyper- prefix here signifies? These diagrams remind me of microphone response graphs, and there's a type of microphone called hypercardioid that resembles this shape.
I don't know for sure, but I generally expect "hyper" in mathematics to mean "extended into an extra dimension".
A Hypercube is a 4-dimensional cube. A hypergraph is a graph with 3 (or more) vertices per edge, instead 2-vertices. Finally, a "Hypercardioid" is a cardioid, except in 3 dimensions (instead of the typical 2-dimensions).
https://i.imgur.com/24LEJyf.png