You have rights, yes, but the exact set of rights is different than most non-flight crew people think.
a very TL;DR summary is that they can throw you out, and you can sue them for it later if it was for invalid reason, but you still get thrown out. Similar rules happen on the sea, which results in hilarious summaries like "yes, a Disney Princess can sentence you to death and do the execution, if specific conditions are met" (condition being "evacuated on escape boat/device of a Disney cruise ship"). The summary is hilarious, but the same applies to plane's captain so long as flight operations are ongoing, including pre-flight preparation after embarking.
a very TL;DR summary is that they can throw you out, and you can sue them for it later if it was for invalid reason, but you still get thrown out. Similar rules happen on the sea, which results in hilarious summaries like "yes, a Disney Princess can sentence you to death and do the execution, if specific conditions are met" (condition being "evacuated on escape boat/device of a Disney cruise ship"). The summary is hilarious, but the same applies to plane's captain so long as flight operations are ongoing, including pre-flight preparation after embarking.