The reason why people don't want to take this line literally is very simple: they don't want to believe that they can't live a life of luxury and still go into heaven. If you're both a believer and rich, you can either live with the cognitive dissonance, renounce your faith, or just reinterpret the precepts you don't like to suit you.
I know - the meaning is actually indisputable from all of the context, regardless of the precise phrase.
But, even so, it is also indisputable that many who consider themselves devout Christians, rich and poor alike, reach a very different conclusion about the meaning of this whole exchange. I can only imagine this must be motivated reasoning.
I found a thread showing some such rationalizations:
Interestingly, it seems modern day prosperity gospel sorts don't go for a reinterpretation of the camel, but for a reinterpretation of the word "rich".
