To be fair, you don't have complete control over struct memory layout in standard C, either, and C is certainly a systems language. The compiler can choose to add padding where it likes. Pragma pack, attribute(packed) etc are compiler extensions.
Edit: You can also link against .o files with Swift, which is the only sane way to integrate assembly language anyway (i.e. in a separate compilation unit).
I don't know if you are talking about swift only, but with GCC or clang working in C, inline asm is a lot more pleasant than another object file. More efficient too. (Unless you like that your code makes a pointless jump to run a single instruction and jump back.) You can contrast this with the Microsoft compiler which won't do inline asm on ARM.
Edit: You can also link against .o files with Swift, which is the only sane way to integrate assembly language anyway (i.e. in a separate compilation unit).