To be honest, I found that adding a subwoofer to a pair of fronts made a huge difference (before adding centre and surrounds).
I think these days it probably makes the most sense to get 5.1 speakers to start with, of at least reasonable quality, budget permitting.

As you get more money available, you can upgrade elements, like say the surround speakers, and re-use the displaced elements into, say, a second av setup in a bedroom.
I have full HT setup in both the living room, and the master bedroom. The living room has my best stuff, and living room has the older/smaller/cheaper stuff.

What amazes me most about my av setup over the years is how some stuff has lasted for 10+ years and performed really well, and a few things have not lasted well. Problem is it is hard to tell at the outset just what will last.
Good, high quality, long lasting brands of stuff I have personally used include:
Yamaha av amps; B&W speakers (big ones, and so-called Flat panel ones) Energy in-wall speakers, Yamaha sub woofers;
Things that haven't lasted well: 120" motorised screen - the mechanism failed shortly after installation. had to use almost manually; Some DVD players - started skipping badly after a couple of years use.

There is no end to it, and the joy of listening to a movie with a powerful soundtrack well reproduced in your home av setup is easily better than what you can experience at the movies. :)