This shouldn't be a performance issue. It's a storage issue. The phones from early 2010 made by HTC had very low storage, like 512 MB, where 300-350 was occupied by the OS itself, and the rest was free for the user.
HTC Desire which basically is a Nexus One clone, couldn't even upgrade to Gingerbread+Sense, because there wouldn't be enough free storage left, so they had the choice of either using only stock Gingerbread to upgrade it, or leave it to Froyo+Sense.
Now I assume ICS is even bigger than Gingerbread, so I'm pretty sure it wouldn't fit in there. Now, will we see CM9 with ICS on Nexus One? It's possible, but maybe the CM team will be able to cut what the Android team can't cut from it, and offer a more stripped down version of it.
HTC Desire which basically is a Nexus One clone, couldn't even upgrade to Gingerbread+Sense, because there wouldn't be enough free storage left, so they had the choice of either using only stock Gingerbread to upgrade it, or leave it to Froyo+Sense.
Now I assume ICS is even bigger than Gingerbread, so I'm pretty sure it wouldn't fit in there. Now, will we see CM9 with ICS on Nexus One? It's possible, but maybe the CM team will be able to cut what the Android team can't cut from it, and offer a more stripped down version of it.