The most important thing is that graph algorithms and concepts have a strong relation to numerical aspects of the linear algebra and can be used to accelerate computation.

(You could of course argue that the act that graphs can be represented as a diagram helps humans come up with such algorithms, but that's basically equivalent to saying that you can represent a matrix as a block of numbers and that helps humans look at it).

Diagrams are how you encode categorical models of semantics, which naturally can be represented as graphs. Those graphs can in turn be encoded as matrices. So you have a way to encode semantic foundations as matrices — which you can then use graph algorithms to analyze.

Being able to move your semantic models (eg, diagrams) into a computational framework (eg, linear algebra) is neat.

Please show us the graph that "encodes" the fundamental theorem of algebra.

The fundamental theorem of algebra is a fact about the diagram which relates polynomials via division by monomials.

Every polynomial of degree n is n divisions of a monomial away from the empty product.

- - - -

This is probably easier to see the other direction:

Polynomials of degree n are isomorphic to n-products of monomials, and you can build a graph of the assembly where each arrow represents a multiplication by a particular monomial. (Then reverse the arrows, to get my original diagram.)