Yet another post about fences

About 3 months ago my wife and I installed a rear garden fence – 6′ cedar, basic, no need for permits etc. The original fence was 6′ chain link, which we didn’t touch. We played it safe and had the fence installed INSIDE our property as we didn’t want the hassle of dealing with 5 different abutting neighbours, 2 of which are absent as their houses were on the market. We had recently had a survey done, so were confident the fence is well inside our property. Everything was going hunky dory until the new owner of one of the houses raised a complaint that we should have put the ‘pretty’ side of the fence facing them. They say it’s a requirement, yet I can’t find anything online that validates this. The way they are carrying on you’d have thought I’d cut their leg off. I’m not sure if this is a non-issue that I should generally ignore?