Many people do think that it'll delay speech - and anecdotally, I've found this ignorance everywhere in the Anglosphere.

Additionally, there's a strong push for immigrants to assimilate. This isn't just coming from Americans, but often from inside immigrant cultures as well, an idea that if they speak their home language they somehow won't speak English as well.