I think it’s more complex than that. If your program B is, as an example, a formatting-in-HTML wrapper around program A (say gnu grep), then you can ship B in binary-only, non-GPL form that calls A, so long as the end user can replace A with a later version or their own patched version of A and B continue to work. (This preserves the freedom of the user with respect to A.)

In that case, B specifically depends on A (for filtering functionality), yet is still (likely, IANAL) considered a mere aggregate as I read things.