I see why you would say that: these neural networks probably have thousands or millions of weights while the equations of motion can probably be written on an index card.
But I would argue that this parsimony is illusory. There's a lot of implicit knowledge needed for the interpretation of physical laws. The laws are written using specialized mathematical notation such as special functions, partial differential equations, in a certain conceptual framework such as Lagrangian mechanics. You need to understand the concept of abstracting and quantifying a dynamic system (most people wouldn't imagine you can do this) and then you have to learn all the tips and tricks of how to reformulate and solve systems.
For example, I could write a mathematical representation of quantum electrodynamics (the theory of how electrons and photons interact) on a single index card. However, I would need to dig into my two shelves of QFT textbooks to actually make any quantitative experimental predictions, on top of my degree, PhD and post doc experience, which I need to even be able to read the textbooks (and I would still mess up the minus signs).
I think it's important to remember that these neural networks are doing all of that - not just finding the physics, but also all the abstraction, calculation and interpretation that is usually taken for granted but actually very non-trivial.
I sort of agree with both myself and yourself. This point of mine technically must be qualified when interpreted like this but I actually mean something slightly more subtle than just parameterization.
The tools of physics have a lot of implicit assumptions that guide the end result in ways that I would describe as parsimonious in terms of how much the output state space must be reduced. They are much more free, which is why they can be amazing for some very hard shit, but proving they're behaving exactly in "physical" way is very hard.
"Time is defined so that motion looks simple" is my favourite quote from MTQ for this reason. It's intuitive and yet also very physically "rigorous" in a way that people don't necessarily realize is a thing in physics beyond just using mathematics.
Maybe we can just train the AI to do the maths for us, dunno, but I think currently this tabula Rasa approach will inform the physics-y-ness. I still call it physics personally, but I don't really think it's interesting from a purely physical perspective.
There have been some works deriving conservation's laws and so on from empirical motion, which I think is very impressive at scale, but I don't know what that does for physics as opposed to the applications of said physics.
