Seems to point to the Netflix app or how the Netflix traffic is getting served to your TV.
You seem to have proved that the traffic can be fine (from Netflix to other devices) - though I am not sure if you said you managed to get 4K running on anything else (probably hard to test as the TV is likely the only client that is built for it).
I know that streaming sites often serve content from different sources depending on the client they detect.
Do other Orcon Fibre users get problems with 4K Netflix content?
The TV is probably too big to easily move to someone else's house to test on a different ISP.
I feel for you - Is it worth a call to Panasonic?