Palestinian security forces and hundreds of civilians immediately entered the evacuated Jewish settlements, raising the flags of both the Palestinian Authority and Hamas.
Mosque loudspeakers declared the "liberation" of Gaza while fireworks lit up the sky.
In the settlement of Neve Dekalim, Palestinians tore down aluminum window frames and metal ceiling fixtures from the main synagogue and lit fires inside. Palestinian police tried to stop them but were overwhelmed and finally stood by, Reuters reported.
The issue of what to do about the synagogues caused contentious debate in the Israeli cabinet up to the last minute. Israeli Prime Minister Ariel Sharon said his country was left with a no-win situation: tearing down the 20 temples or leaving them standing with the knowledge they might be desecrated. He voted with the majority to leave them in place.
Palestinian officials said the turnover was a success.
"These last 48 hours were a test and I think we passed," Palestinian cabinet minister Ghassan al-Khatib told Reuters.
"It cannot continue without elections. ... If there is a postponement or a canceling of elections I have no doubt this is going to be a very strong factor of instability."
The United States has said Palestinians must show they can form a stable government in Gaza before embarking on a peace "road map" toward a Palestinian state.
Palestinian President Mahmoud Abbas has pledged to control unrest in Gaza by the end of the year but said he will not try to disarm the powerful militant group Hamas.
Palestinian officials say it will be hard to control Gaza effectively if Israel is still occupying from the borders. Last week, Israel closed the Rafah crossing on Gaza's southern border with Egypt.
"Mainly the onus is on the Palestinians. ... But if Gaza is turned into a big prison, this will make our job in maintaining the rule of law very difficult," Palestinian chief negotiator Saeb Erekat said, according to Reuters.
Muhammad al-Najar, a 21-year-old student, walked nearly six miles to get to Neve Dekalim. He said he and others had "seen places we've never seen before, that we've heard about for years." Asked what surprised him the most, he said: "The clean streets and all the trees," according to the Times.
At the nearby Palestinian village of Mouassi, where some residents had worked for Neve Dekalim settlers, women in head scarves waved Palestinian flags, ululated and chanted thanks to God. The mukhtar of Mouassi, Daman Hajaj, said, "as long as we feel freedom, I feel wonderful, and I feel freedom today," he said, according to the International Herald Tribune.
"I can move easily," he said. "There are no checkpoints."