As Siddiq

It is written in the Conversations of Shams Tabriz that the hadith: "Visit me now and then" was intended for certain companions and their continuous presence, which made the Prophet (peace be upon him) weary. He did not intend this saying for everyone, especially not Abu Bakr r.a.

Shams Tabriz says that in one battle a warrior dashed onto the field, and the Muslims wouldn't fight him.
People asked, "Where are the volunteers, those who renounce their lives, those who seek death? (2:94). The companions answered, "the warrior who jumped into the field is Abu Bakr's son, the light of his eye. The Muslim warriors are ashamed to fight against him; this is the reason." Thus, Abu Bakr ra went onto the field. When his son saw his father's face, he immediately retreated. Abu Bakr r.a. retreated and returned to the Prophet (peace be upon him).

"The Prophet (peace be upon him) put his hands onto Abu Bakr's hands and said, "O Siddiq! Keep your nafs for us!" That is, "According to you, your nafs has no value, but for us it has great worth. Protect it for us. Don't go into battle, don't go into the fighting; don't separate from our companionship."

"How could the Prophet (peace be upon him) say to Abu Bakr, "Visit me from time to time?" For other believers, combat was an obligation; for Abu Bakr r.a. it would have been separation. The good deeds of the devout are sins for those who have attained closeness to God."

Conversation of Shams Tabriz