I would swap based on the offer, not on who the bank is.
I have bank accounts with all the major banks (long story) and had a friend recently ask me who the best is.
My reply: none. They are all bad, just bad in different ways.
The experience you have with a bank is often determined to a large extent by the skills, personality and experience of the person you deal with, and the personal relationship you have with them - rather than with the bank as a whole.