Modems are part of the ISP's network. Where does the ISP's network end, and your network begin? You might be tempted to say it's at the wall jack, but your ISP is tempted to say it doesn't, and your computer is theirs too.
Note that it might not be the same at L1 and L2, e.g. your ISP may own the physical lines up to the side of your building, and the data link layer up to your modem.
In the past (and again on some fiber networks), there was https://en.wikipedia.org/wiki/Network_interface_device and the separation occurred there.
Note that it might not be the same at L1 and L2, e.g. your ISP may own the physical lines up to the side of your building, and the data link layer up to your modem.