Saying that a Muslim ban isn't racist because it's a religion and not a race is just semantics. It is still stereotyping and vilifying an entire group of people based on one characteristic.
Trump himself might not necessarily be racist - I don't know that he holds any true beliefs other than he is the centre of the universe. However his policies and rhetoric are purposefully designed to appeal the to the racist nature of his supporters (either latent or explicit).
There is no greater unifier than a common enemy. Trump has painted the picture of outsiders as the enemy, and solidifies his support with oversimplified plans of giant walls to protect America from the evil outsiders.
Border security is necessary and important, but that is not what Trump's wall is about.