And it is also an idle act that someone says to the person sitting next to him, "Be quiet." Salman al-Farsi, may Allah be pleased with him, also narrated from the Prophet (peace and blessings of Allah be upon him) that when the Imam begins the sermon, one should remain silent.

Narrated Abu Huraira: Allah's Apostle (p.b.u.h) said, "When the Imam is delivering the Khutba, and you ask your companion to keep quiet and listen, then no doubt you have done an evil act."