It is pointless you don't enable it. SMS is unencrypted and insecure, this has full end to end encryption Google or not meaning Google can't read it. Your mobile carrier / law enforcement can read SMS messages.
So, based on your argument about having privacy you have close to zero with SMS. Just enable it and job done. Avoid SMS where possible and use anything else like RCS (https://jibe.google.com/), Signal (https://signal.org/en/) and even Whatsapp instead. RCS is rather good as it is automated with Android.
Also, by using Android you are being tracked in many more ways. You're literally using a phone with an operating system made by an ad company...