Yesterday I installed some Windows updates which were waiting to be installed then updated to the latest version of Edge but didn't have time to do anything else at that time. The problem was still present. Today I went back and rechecked the various settings which were correctly set. I changed some of these back and forth but couldn't see anything else I could try.
The next time I tried to open a PDF from within Outlook it opened with Acrobat and asked if I wanted make this the default handler. Since then each PDF opened in Outlook opened in Acrobat.
I'm really not sure what I did to fix the issue but it's all OK now. Thanks for the help.