Remember: Even piracy doesn't offer absolution.
These properties rely on popular / universal awareness to achieve network effects and cement themselves within modern culture. When this happens, the memes and concepts from the property worm their way into everyday language ("he who cannot be named", "10 points for Gryffindor", etc) and help keep everyone else buying.
The only answer is to treat people talking about Harry Potter as you would someone who keeps talking about the greatness of R Kelly's music or Bill Cosby's comedy.

