Yeah, it's the gossipmongers that caused him to leave those previous jobs, not the fact that he's a creep.

Edit: Also, double-ew on the fact that the adjectives used to describe these young women are "lovely" and "gorgeous". Not that there's anything wrong with thinking your spouse is attractive, like, in general... but given the context....