The previous owner of my house liked telephone jackpoints, there seems to be a couple per each room.

I am currently on ADSL for another week and then fibre will be installed.
Once fibre is installed I want to remove all the phone jackpoints, is there anything that can go wrong / something I have not considered?

My plan is:

1. Unscrew and remove jackpoints.
2. Push cables back into wall.
3. Fill over hole.

Cheers.