And if this is extended to sales taxes: how companies actually pay taxes worldwide? I mean, a SaaS would require the company to be registered anywhere it gets a new customer? In real life how it is handled?

Also, I have read that some services like 2checkout offer "name of record" where they seems to pay taxes on behalf of a company globally. But I haven't seen any other company offering that, so I'm not sure if it it's standard or not...

Sales taxes are a US-centric concept. Most of the world uses VAT, though Australia uses a variant known as the GST. (And not to be outdone, Hawaii has a variant of VAT based on gross income.)

That being said, for SaaS, it is still currently generally the prevailing rule that sales/VAT/GST/whatever compliance isn't required in the customer's country unless you have a physical nexus to that country (i.e., office, employee, etc.)

However, the US tech dominance has resulted in many countries proposing or even adopting rules that would subject SaaS transactions to VAT/GST/whatever compliance regardless of the location of the vendor. Most intl tax experts agree that this will become the standard within a decade, though there is substantial disagreement as to how soon within the next decade the transition will occur.