But large numbers of citizens who have always lived in the U.S. don't file/pay taxes they legally owe. California even publishes a list of the top 500 tax delinquents in the state, which shames some but nowhere near all of them into paying.
Few tax cases rise to the level of criminal offense, rather they are civil actions where liens and levies are placed against your income and property. I have never heard of someone being extradited by a foreign country because of anything the IRS did. Worst case is you probably cannot (re-)enter the country and your U.S. passport if you have one is suspended or revoked.
As a further note, often these expats only have an information filing requirement regarding their foreign financial accounts (FBAR and FATCA), not an actual tax bill (and if they had double-taxed income, U.S. gives a foreign tax credit for that).
