Then, could we say that the general or abstract form of the ellipse is a set of vectors? And with that set, there is an identity element, namely the unit ball. And that the identity is what produces the uninteresting basis vectors. Maybe what makes the vector space (that is, the set of vectors) interesting is equipped linear operators on all the vector space's elements?
Maybe this whole structure, a set equipped with some operations, has distinguishable features. Or maybe it can perform as underlying machinery for practical endeavors.
So in that sense an ellipse is a function `ellipse(p_1, ..., p_n, d)`. But given the foci you can get the vectors and vice versa. So one representation isn't necessarily 'more fundamental' than another. They just have different (equivalent) perspectives that lend more easily/naturally to certain problem contexts.
(for example, the set of points definition is a lot less directly useful than the matrix A if you're trying to do linear algebra)
The unit ball isn't really an inherent "identity element" for ellipses. It's more of a special case where the foci are equally spaced (eccentricity is zero).
If you want to go that route, the eccentricity is a more fundamental descriptor of all conic sections, of which ellipses and circles are just two. In fact, all conic sections can be described as `conic_section(p_1, ..., p_n, eccentricity)`. So maybe you'd argue that's more "fundamental" because it's more general.
The ellipse can be also encoded by just the lengths along each axis and then by a rotation in R^n (which is just a unitary matrix multiplication). So in essence, for the problem in the original post there are three pieces of information needed: a translation, a rotation, and the deformations of the unit ball along the axes.
Maybe this whole structure, a set equipped with some operations, has distinguishable features. Or maybe it can perform as underlying machinery for practical endeavors.