When we begin to cultivate detachment, those who expect something of us become concerned. They may not understand our detachment and may be afraid that we will not live up to our material obligations. Therefore, especially in family relationships, which tend to be full of obligation, it is good to quitely cultivate detachment while appearing externally attentive. Those who renounce their household duties without experiencing true detachment are false renunciates.
LFD Vol 1, Spiritual Life of Grhasthas, Pg 380-381, last Paragraph