I alot of it comes down to the price people pay. If you are paying very little, then the server is likely to be hosting a lot of other sites on it, due to economics. One reason though sites load slowly, and perform worse on some servers, is that it has lots of plugins. Wordress sites for example can load poorly on shared hosting.  I would suggest using a host that uses cpanel, which doesnt use the older suphp, and avoid plesk. Also if you are using reseller hosting, performance I found can be worse, than using normal shared hosting. In some cases a VPS hosting plan can be an option for a more demanding website.