Whilst a mask seems a good idea, I presume that the organic material that built up in your lungs would be broken down and washed out by the lungs natural processes within a few days or weeks?
Not a doctor but.. You're falling for a bit of a naturalistic fallacy here. Just as Asbestos is a naturally occurring mineral however inhaling its dust can come with very serious health problems, so can hay contain mould which can also cause serious lung disease. http://www.ccohs.ca/oshanswers/diseases/farmers_lung.html
Chris is absolutely right, and in addition to mo(u)ld, there's also tons of antigens to provoke inflammatory responses in the lungs, which can lead to asthma and chronic bronchitis.
Perhaps a doctor could clarify?