Just how would you make this case defined? The alternatives to undefined behavior tend to be (1) being silent about the issue and letting users and implementers find out themselves (2) defining the behavior in a way that makes it difficult to implement on some architectures or (3) defining the behavior in a way that imposes costs on all architectures. None of these is particularly attractive.
I would go with option 1 because it does not suggest the problem is solved. As I write this, I realize that there is a counter-argument in that introducing ptrdiff_t gives you a place to issue a warning.
