I suspect they do understand that "for fun" is a reasonable answer. The disconnect is probably that they don't see how your hobby could be fun.