For reference, buildkite-agent [0] is about 40k lines of go. Running `go build` including dependencies took 40 seconds, and running `go clean && go build` took 2 seconds. I know Go and Rust aren't comparable, but Rust's attitude appears to be "we don't really care" when you compare it to Go, considering they both started at _roughly_ the same time and Rust's first stable release came long after Go was in use.
Note that `go clean` doesn't actually clear dependencies, which are stored globally. It's possible that your initial compile also wasn't a true clean compile, because some modules were already cached globally.
[0] https://github.com/buildkite/agent