The Iraqi army secured the neighborhood, following a truce declared 10 days ago between the fighting Shiite factions, which paved the way for Tuesday's incursion. The truce called on the militia to lay down its arms and for the government to take over Sadr City.
"We are taking control of three quarters of (Sadr) city. What is left is the final quarter," said a spokesman for Iraqi security forces in Baghdad, Reuters reported.
Buildings were pockmarked from bullet holes and tank fire, signs of the earlier violence that had left hundreds dead.
Sadr City, populated by 2 million people, is the main stronghold of Sadr's Mahdi Army. Tuesday's operation marked the first time since the 2003 U.S.-led invasion that Iraqi forces pushed so far into the area. Previously, they had control of only the perimeter.
The security forces spokesman said the operation was coordinated with Sadr's movement to avoid bloodshed, according to Reuters.
The government of Prime Minister Nouri al-Maliki, himself a Shiite, is pushing to exert control over areas dominated by Sunni Arab insurgents or Shiite militias.
Iraqi troops took over a disused police station, while others moved into high-rise buildings and stationed snipers.
A U.S. military spokesman said the Army's action was Iraqi-planned and executed and no U.S. troops were involved.
Elsewhere in Iraq, the U.S. military reported that U.S. forces killed a senior al-Qaida leader in Samarra, north of Baghdad, and detained 20 others in different areas of Iraq on Monday and Tuesday, Reuters reported.