It's certainly possible to generate the HTML up-front. Tooling like Next.js even sets things up so it's easier to render the HTML for the first page load on the server than to push it to the client.

I have a website. It's not great, it doesn't get much traffic, but it's mine :). If you disable JS, it works: links are links, HTML is loaded for each page view. If you enable JS, it still works: links will trigger a re-render, the address bar updates, all the nice "website" stuff.

If I write enough (and people read enough) then enabling JS also brings performance benefits: yes, you have to load ~100kB of JS. But each page load is 4kB smaller because it doesn't contain any boilerplate.

Obviously I could generate the HTML any way I choose, but doing it in React is nice and composable.

If you really want to, you can have a react app that is just static templates with no interactivity with a simple Node server that just called renderToString and all of a sudden react is just a backend templating framework. If you want to get really fancy you can then re-render specific components on the client side without re-rendering the entire page. You don't need NextJS to do this either, its very simple and straightforward and lets you use an entirely frontend toolchain to do everything.