Everything seems so much snappier and more responsive (and no, not just as a result of a fresh install - I did that semi-regularly on Gingerbread and didn't end up with anything like this). How much is a result of ICS itself versus the specific ROM I've chosen I'm not entirely sure, but it's a significant jump in performance.
Was previously running DarkyRom v10.2.2, and am now on Slim ICS 4.2 with Semaphore ICS 1.3.0s kernel.
Under Darky I typically had on the order of 150mb RAM free, now I'm up around 210mb. Previously I had little annoyances like after waking my phone and quickly typing my PIN unlock the phone wouldn't always pick up every digit (usually registering only three key presses instead of four). This happened at least 50%+ of the time. Haven't had a single recurrence of this (and I've tried really hard to break it!).
So if anyone's on the fence I can at least say that in my case it was well worth the upgrade. Still have some questions over battery usage but it's hard to say if it's over and above what I was seeing on GB or just the result of excessive use thanks to it feeling like a new phone! Either way, I'm usually in close proximity to a charger, etc so it's not of major concern.