Importance of Hajj

عَنْ عَلِي قَالَ: قَالَ رَسُولُ اللَّهِ صَلَّى اللهُ عَلَيْهِ وَسَلَّمَ : " مَنْ مَلَكَ زَادًا وَرَاحِلَةً يَبْلُعُ بِهِ إِلَى بَيْتِ اللَّهِ فَلَمْ يَحج فَلَا عَلَيْهِ أَنْ يَمُوتَ يَهُودِيَّا أو نَصْرَانِيًّا

"Whoever has the means of performing Hajj yet he doesn't perform Hajj, then he may pass away as a Jew or Christian. [Shua'bul Imaan Pg.443 Vol.5 / Tirmidhi Pg.167 Vol.3]
The same type of Hadith is reported by Hazrat Umar (Radiyallahu anhu) wherein he mentioned thrice: "he has the choice to die as a Jew if he so wishes or as a Christian." Another report from Hazrat Umar (Radiyallahu anhu) if he does not do so, then you can swear (by Allah) that he has died as a Christian or as a Jew." (Kanz)

Note: It is possible that this is Hazrat Umar (Radiyallahu anhu)'s own opinion. The ulama are agreed that not performing Hajj while being able to do so, does not make one a Kaafir. Kufr only establishes itself on a denial of it. 

Another saying by Hazrat Umar (Radiyallahu anhu) : "I would have liked to announce in all the towns under Islam that whoever has the means, yet does not go to perform Hajj, should be forced to pay the Jizya, because he is not a Muslim.' (jizya is a tax that the Muslim Ruler takes from the non-Muslim inhabitants of an Islamic country. It is never taken from a Muslim).

Fazaaile Hajj