The thing missing in that letter that might be a significant factor is whether LW is working from home at present, so Husband sees them 'not working' and wants to spend time together, when in fact LW is doing one of those lighter on phases (so, not at the computer, or on the phone, or whatever busy looks like). Particularly if LW is also inclined to noodle on the computer when not working.

Because 'down time at work where I can stare at the wall' and 'down time while working from home' -- only one of these is visible to the husband.