Canvas is a pixel buffer...and a set of drawing rudiments to imperatively actually make that pixel buffer useful. If you were actually just using canvas as a pixel buffer it would be catastrophically slow.
SVG is a pixel buffer and a set of drawing rudiments to imperatively or declaratively actually make that pixel buffer useful.
The distinction you are drawing between these is sophistry.
No the distinction is important and directly related to performance. Supporting the canvas API requires fewer CPU instructions to get to pixels on the screen. The browser has to do a lot more work to turn SVG into pixels