It comes from the Latin saeculum which is a period of time longer than a person's life. Rome used to hold the secular games, which was an event so big, it'd never be seen again in a person's lifetime.
1250-1300; < Medieval Latin sēculāris, Late Latin saeculāris worldly, temporal (opposed to eternal), Latin: of an age, equivalent to Latin saecul (um) long period of time + -āris -ar1; replacing Middle English seculer < Old French < Latin, as above
A saeculum is a length of time roughly equal to the potential lifetime of a person or the equivalent of the complete renewal of a human population. The word has evolved within Romance languages (and Swedish) to mean "century".