To be fair, at four or five the kids will remember parts of the second move, and they may experience some distress at leaving friends and familiar places. But LW's wife is correct, and LW can hopefully set aside their misgivings to help the kids through it as needed.

Edit to add LW may want to discuss these feelings with someone, certainly before the second move.