"but generally I just get very dirty .JS files with mixed HTML/CSS in them"

Just put the css/sass on .css/.sass files and import the classes from the react jsx file. No need to mix css and html.