I switched to Math from CS at my university specifically because of this. Although the CS field does emphasize a lot of theory, having a more general theoretical background greatly helps with picking up new engineering fields as a hobby.

IMHO the main reason people feel stupid about algorithms and the like is the progress isn't as clear and as linear. I wasn't good starting out. Coursera is great, and probably the most flexible and least frustrating way to expand your skill set.