No idea if it is illegal. But definitely plain rude.
Tell your friend to not be a nosey sh1t, mind his own business, and re-position the camera so you can't see the neighbors property.
You can aim cameras to simply follow a fence line, or have the top of the image stop at the gate. It will still catch someone climbing the fence/gate and you aren't encroaching and peoples privacy.