The cost is very low (it's everywhere), which drags down the price to zero. People do value oxygen a lot though, it's essential to survival. If someone was to take away all the oxygen,we would see how much people value it: they would go to great lengths to obtain some. This is not usually put to the test, of course.

Edit: this question was raised by Adam Smith, known as the diamond-water paradox, or paradox of value:

https://en.m.wikipedia.org/wiki/Paradox_of_value