That has been tried to great success with the rpi3, so I suspect it will at least make it less prone to throttling.
Edit: I just stacked a bunch of 10sec coins on the processor (without any thermal compound) brought the idle temperature down by 3 degrees C.
Edit2: Put an carbon tablet pipe on it, filled with coins. It is not up to heat yet, but right now it is idling at about 45C. It would probably be better with some fluid in it. I will probably keep this as my cooler.
Edit3: Seriously: I have wonderful thermal numbers. sysbench for 240 seconds, temperature 62C.
This is a rpi4 with a poe hat, so my passive cooling options are limited.