The XBox doesn't have a proper remote control, just a game controller. It doesn't have an RS232 port or interface for remote installation, and can't play SACD or DVD-Audio discs. If you have an older or lower spec receiver, it doesn't have the seven-channel analogue audio outputs, quality DACs, or the second HDMI output you might need. If you have a newer receiver, it won't support bitstreaming (currently). General compatibility aside, it doesn't operate in (near) total silence, as being a game unit this wasn't ever a design consideration. If you have a collection of movies on Blu-ray or DVD, it also can't upscale them to UHD, nor will it pass through other 4k sources.
It doesn't behave like a conventional player, so sitting down to listen to some music without having to turn the TV on is going to much more difficult. Being a gaming console, it has too many design compromises and limitations. It has Ultra HD support, but only as an afterthought, tacked onto a long list of other things it can do, and all within a small BOM/manufacturing budget. It hasn't been designed primarily as an Ultra HD player, and therefore its focus isn't on operation as one: it was designed as a game machine that also plays movies.