The only change I've noticed in the last 5-6 firmware updates is that Singstar is now built into the system. If they ever mentioned it, I didn't notice, just discovered that it was there when browsing through menus the other day.
They seem to have stopped publishing the changelogs on the update screens, irritating.
I don't think they've changed anything genuinely useful\interesting in a _long_ time.
The PSN store redesign is not what I'd call an improvement. There's an extra 3 "clicks" involved whenever you want to download anything now, total jumble. At least it's much easier to find things than on the Xbox though.
Edit: Just clicked, "CFW" - please disregard the above :)
I had mine CFW'd, got a bit worried about it after having the BluRay issues, managed to resurrect it and don't want to risk stuffing it up again, so it's all official, all the time for me now. (On the PS3 anyway)