That's not universal. I have Brazilian family members of Portuguese descent who think of themselves as non-hispanic "white" and have Brazilian Portuguese as their first language. And a few others in the same side of the family with darker skin who I believe identify as black.
Yet I would probably have made the same classification you did without knowing them.
Brazilians are Latin and definitely not Hispanic. Latin is not a racial term (and that's why Brazilians tend to find the racialized term "Latino" as used in the US repellent), and has nothing to do with your appearance or genes: it means you speak a language that was derived from Latin as a first language. Angolans are Latin, the French are Latin, Romanians are Latin, etc. Some Swiss are Latin, some are not. So, Brazilians can be Latin, White and non-Hispanic all at once.
Hispanic is something related to Spain. Spaniards are definitely Hispanic, Colombians, Argentines, etc. are (in a more distant sense) Hispanic.
"Hispanic" is not considered a racial term by the US government. It's an ethnicity that can be applied to any race. It is, however, considered a minority group for various purposes.
I know many South Americans, Brazilians in particular, don't like the label as used by the US government. I also know native Spaniards who have embraced it.
Latino/Latina/Latinx is it's own brand of controversial in the communities it is applied to. Regardless, in the US it has never been applied to people unless their heritage traces back to the Iberian peninsula.
Yet I would probably have made the same classification you did without knowing them.