There are a few abstract, mathematical proofs that a theoretical quantum computer could be faster at certain specific specialized tasks (such as prime number factoring, which has important implications for cryptography). No one has actually really built such a quantum computer yet, it remains a theoretical idea. The theory of quantum physics itself is also incomplete. So our knowledge of the universe and the possibilities it contains is very limited. Someone who says "never" is operating from a mindset where they equate the whole of reality with their current, limited knowledge of the universe. I don't think it's far fetched to imagine that our understanding of quantum physics might one day lead to new breakthroughs in general computation. Given that silicon processors are already near the quantum limits, there would be a huge incentive to find a new breakthrough. That said, this is in the realm of pure science fiction and not something that people are actually working on today or something we are likely to see in our lifetimes.
>No one has actually really built such a quantum computer yet
What are you talking about? We've had very small quantum universal computers for some time, and quantum annealers for two decades. How did you miss that?
I regret to inform you that low bit strength RSA was even broken with a quantum annealer, not even a universal quantum machine. the NSA is a customer of D-Wave.
IBM's quantum roadmap has them shipping 1121 qubit machine next year, this year they've shipped a 400+ qubit machine. So far they've been able to keep doubling the number of qubits every year. These are universal quantum computers.
If they can keep scaling at that rate, both RSA and ECDSA are toast within a decade.