My 2c, coming in at the end of the conversation...

As Timmmay and kiwijunglist have highlighted, there are two different levels/types of 'travel' camera.
One is where it's small and pocketable, and you take it everywhere with you, so it's always on hand to take a shot, and it doesn't require a separate bag to house it.  (Sony RX100 looks to be the goto for this)

And the other side is gear that can take quality photos, (equivalent sensors to larger DSLR gear with interchangeable lenses etc), in a small form factor that packs down to a small size.  That's where the Olympus, Sony A6000 and Fuiji cameras come into it.

My take is that whilst mirrorless cameras are smaller, they are extremely similar to DSLR's.  The only difference is a bit of size around the optical viewfinders and autofocus style.  The issue I have is that you are only saving the bulk of the body in size, as the lenses are fundamentally the same.  You can go pancake in lenses, but then you're back to being fixed, and you're potentially better off with a fixed lens camera.
If you are carting a twin lens kit and a prime around, then the body size is hardly the deciding factor in how portable it all is.

Camera's and sensors have rather plateaued in terms of image quality.  Any new camera you buy from these options is going to be good.  The items that appeal to me, may not be of use to the next person.
I'd be after stabilisation, as I'm not carting a tripod around the streets, and weather sealing, as if you are travelling you only get one chance at certain locations.
Even the above sentence has issues that related to how you are travelling and where.  If you have a rental car for the whole trip, then a tripod is not a problem, but if you are back packing then it may be.
If you are travelling to a dry climate then weather sealing (other than dust ingress) may not be an issue either.

I'm strongly considering an A6000 (now that the price is dropping and it has the new firmware update allowing X-AVCS 50mbs video) or the Olympus OM-D EM5 for it's excellent stabilisation ability and weather sealing.  The Sony is the best bang for the buck, but it feels quite budget/plastic in the hand, regardless of how strong it might actually be.  I'm after an interchangeable lens camera, and mirrorless offers tremendous value for money over equivalent priced DSLRs.  11fps burst shooting for example.  So many entry level DSLR's have crappy viewfinders (pentax mirror and not showing the full lens field of view etc) which is mostly the major difference between the two styles.