BJP president Venkaiah Naidu said, "As we have not got the mandate of the people, we have decided to sit in the opposition."
Vajpayee has been asked to continue until a new ruling coalition is formed headed by a new prime minister.
The election results show the main Congress party, led by Italian-born Sonia Gandhi, as winning over 200 seats in the 543-member parliament. Recent opinion polls had previously shown the Hindu nationalist BJP as being a clear favorite, so much so that Vajpayee called for the elections six months early.
"I could not have dreamed that Congress would come to power," said Samir Mondal, a private security guard in Calcutta. "All the polls said BJP alliance would be number one. It seems Congress has done some sort of magic. It is unreal."
Final vote counts indicated that the Congress party had won a 278, six more seats than the 272 needed to form a majority.
BJP had banked on recent economic gains, a burgeoning high-tech industry, and peace talks with Pakistan as ensuring victory. But the distribution of the newfound wealth has been slow to nonexistent, especially in rural areas where basic sanitation and access to electricity is still a problem.
"This is a lesson to most second-rung BJP leaders that they need to get out of TV screens and get on the roads," said H. Narasimhan, a retired bank employee in Delhi. "The rural masses have rejected the computer culture the [BJP] was propagating. To them reforms mean water, food and clothing, which the [BJP] has failed to deliver in the villages."
Supporters of the BJP worry that without Vajpayee in office India's healthy market could suffer, along with the progress made with Pakistan over the disputed Kashmir region.
"Without Vajpayee as the prime minister, the country will face a big blow," said retired professor Sudha Udeshi.
The upset marks the return to power of the Nehru-Gandhi dynasty. Congress leader Gandhi, the most likely choice to be the next prime minister, is the widow of late Prime Minister Rajiv Gandhi. Rajiv was the son of India's first female prime minister Indira Gandhi, and grandson of India's first prime minister Jawaharlal Nehru.