What is meant by a slave of gold and silver being cursed? A person is so engrossed in accumulating the wealth and riches of the world that he becomes negligent of his Salaah, fasting, and other acts of virtue, nor is he concerned of Halaal and Haraam thus he is distanced from the mercy of Allah Ta'ala. If a person had accumulated the wealth of the world and possess Taqwa as well then there isn’t any harm. It is mentioned in the Hadith,

لا باس بالغنى لمن اتقى الله عز وجل

There is no harm with having wealth for whoever fears Allah Ta'ala

From this it is understood that the ignorant Sufis that regard the pious wealthy people as people of Dunya and prevent them from earning a livelihood are incorrect in their practice. Khwaja Azizul Hassan Saheb Majzoob R.A says,

کسب دنیا کر ہوس کم کر                اس پے تو دین مقدم کر

Earn the world, decrease your desires. Give priority to Deen over Dunya.