Gandhi's Congress party trounced Prime Minister Atal Behari Vajpayee's Hindu nationalist Bharatiya Janata Party and his Democratic National Alliance.
"The post of prime [minister] has not been my aim," Gandhi said in an emotional address to a hectic Congress party meeting Tuesday. "I was always certain that if ever I found myself in the position I am in today I would follow my inner voice. I humbly decline the post."
Many at the party meeting begged Gandhi in a string of emotional speeches to reconsider her move, which took them by surprise.
"Please remain with us, you cannot betray the people of India," Member of Parliament Mani Shankar Aiyer said. "The inner voice of the people of India is that you should be the prime minister."
Incoming Congress MP Renuka Chowdhury asked Gandhi not to give in to Hindu nationalists' "machinations."
Gandhi, shouting to make herself heard over the unruly crowd of disappointed supporters, said, "I request you to accept my decision and to recognize that I will not reverse it."
Members of Vajpayee's outgoing party had vehemently opposed the election of a foreign-born leader, and had said they would boycott Gandhi's swearing-in ceremony if she became prime minister.
"A foreigner becoming the prime minister of the country will put national security and the country's self-respect in jeopardy," Uma Bharti, a former sports minister in the outgoing Vajpayee government, was quoted by the Press Trust of India news agency as saying.
According to media reports, Gandhi's children -- also politicians -- asked her not to accept the position, fearing for her safety. Gandhi's husband, Rajiv, and her mother-in-law, Indira, both served as prime minister and were assassinated.
"This is a woman whose husband was killed by terrorists, a woman whose mother-in-law died of gunshot wounds. Of course, there are security concerns," senior Congress party leader Jayanti Natarajan told Reuters.
Gandhi has not said who she will nominate as prime minister, although the favored politician appears to be Manmohan Singh, a Cambridge-educated economist who helped liberalize the nation's economy as finance minister from 1991 to 1996.
Singh accompanied Gandhi to meet with Indian President Abdul Kalam Tuesday, to discuss the process for forming the new government. She is due to meet with the president again on Wednesday.
Citizens throughout India staged protests to express their deep disappointment with Gandhi's decision. Some lay down in the street, while others burned effigies of Gandhi's political opponents. Hundreds gathered outside Gandhi's residence in New Delhi, where one man threatened to shoot himself if she did not reconsider her decision.
Gandhi would have been the first foreign-born leader of the world's largest democracy.