They've stopped exporting to western countries.

After the US (which I’m assuming imports from Canada, the largest exporter), the next 6 largest importers aren’t western countries (if you exclude Brazil in that):

https://www.nationmaster.com/nmx/ranking/potash-fertilizer-i...

https://www.nrcan.gc.ca/our-natural-resources/minerals-minin...

The biggest issue is that Belarus has restricted access to euro seaports, but Belarus and Russia have extensive rail networks.

Reduced supply to Western countries would still push prices way up.