First, I basically agree with you: put the phone/tablet somewhere safe before you wipe yourself, and leave it there until you wash your hands, and you should be fine.

But I think a plausible reason for the greater concern is that people put their phones on or very close to their ears and mouths, and "don't touch your face" is one of the standard bits of advice on not spreading infection. That's specific to phones, though. "A plausible reason" because this hooks into a lot of other concern about "people are spending too much time with electronics," even though what I'm likely to be doing with my phone while on the bus is reading a book or the news, and a paperback book or printed newspaper wouldn't get the same reaction.