And the clarification is that the Prophet, peace and blessings be upon him, only commanded the one praying towards a sutrah to prevent the passerby in front of him, and permitted him to fight him if he is praying towards a sutrah, not if he is praying without a sutrah.

Hazrat Abdur Rahman bin Abi Saeed narrates from his esteemed father, Sayyiduna Abu Saeed Khudri (may Allah be pleased with him), that he was offering prayer using a pillar as a sutrah, when a person from Banu Umayyah tried to pass in front of him. He stopped him, but the man tried to pass again, so he struck his chest. That man was from Banu Umayyah, and he reported this incident to Marwan (the governor). Marwan met Sayyiduna Abu Saeed (may Allah be pleased with him) and asked, "Why did you hit your nephew?" He replied, "Indeed, the Messenger of Allah (peace and blessings be upon him) said: 'When any one of you is praying using something as a sutrah, and someone tries to pass in front of him, he should stop him. If he refuses to stop, then he should fight him, for he is a devil.'" Indeed, I have only struck a devil.