Lord Caitanya Himself has stated that a devotee who is engaged in the chanting of the Holy Name of the Lord does not retaliate, even if he is rebuked or chastised. He says that such forbearance must be practiced by the devotee. Then He says something very, very important: This type of mood in the chanting of the Holy Name of the Lord is the behavior that solidly maintains devotional service for the devotee. It gives him the means by which he can actually be situated in devotional service, because he is so tolerant that the Lord cannot neglect! Others may neglect, but the Lord will not neglect!
MKV, Vol 8- Pg 23-25, How to Achieve the Ultimate Goal of Chanting- March 04, 2014 -Śrī Māyāpur Dhāma